[Nix-dev] Improving the Developer Experience in the Nix Community

[Shea Levy]

On Jun 27, 2012, at 4:06 PM, Michael Raskin <7c6f434c at mail.ru> wrote:

>>> - There is a reasonable public place where I can see every package 
>>> expression used by any committer. So, if someone uses a git-head version
>>> of kernel, it would be nice to see what overrides were needed.
>> Do you mean some sort of repository of local overrides? That might be nice, do you have any ideas about how that could be structured and where to put it?
> 
> Frankly, from the technical point of view I would prefer a set of 
> overridable knobs gradually added to the packages in the main set. This
> gives easier discoverability which is nice. Of course, this clashes with
> the struggle for simplicity of the main expression or something else 
> like that. 
> 

Ah, ok. I think if we were to do this documentation would be essential: what does this knob do, how does it do it, how might it break if something were changed elsewhere, when would a user consider using it, etc. But it's a good idea (also another place for values to clash :))

> Or we could have an everything-goes overrides repository with attrsets 
> named just like the original packages.
> 
>>> - There is a reasonable way for partial overrides of packages. That way
>>> when I look at a private overrided package I can see what the changes 
>>> relative to trunk are. Better yet if the changes are semantically 
>>> separated (where applicable). Currently it is partially enough - 
>>> changing preBuild usually means copying and editing. 
>> Hm, I'm not sure I understand what you're going for here. Why do .override and overrideDerivation and friends not do enough for you?
> 
> It is better than it was a couple years ago. But still, it is way less 
> than what I would like to use. How do I split preConfigure into a part 
> setting a few variables and a part editing an .inc file so that it is 
> easy to insert code between them?
> 

Use nix's highly expressive string primitives ;).

In seriousness, I see what you mean. Does builderDefs solve that issue? How?

>>> - There is a way to add support for marginal improvements to the default
>>> builder without full rebuild (why would we want to need stdenv-updates
>>> just to add xz support?)
>> Agreed. Would you be interested in collaborating on a modular-stdenv (or something similar) branch to make this a reality?
> 
> I want someone else to make the basic design plan. It is obvious that
> whatever I propose here will be too similar to builderDefs (with all its
> problems) to be accepted. I have this experience, and am ready to help 
> to anyone who prepares a better design.
> 

Ok. Why was builderDefs rejected? What problems do you see with it, and what do others see?

>>> - The big packages used by many people are regularly and succesfully 
>>> built in a centralized way
>> Isn't this just hydra?
> 
> You asked about values. I am willing to concede the use of Hydra for
> many things, but I would like to be able to use LibreOffice from there.
> I see no loss in building my own Vim because Hydra builds some very 
> reduced version of it or something like that.
> 
> Yes, in the last stable state (SVN before migration) Hydra provided all
> binary packages I could expect and more than that (I speak of packages
> that I used here).
> 

Ah, ok, I assumed all of these were things you wanted to change. Yes, definitely having hydra still be useful is a good goal.

>>> - On the level of Nix as a package manager, there is a way to roll back
>>> everything but GC
>> What do you mean, exactly? You can roll back versions of nix, and if you use build.nix in the nix source tree you can build each component as its own derivation...
> 
> Again, it is a value well-satisfied now. I would prefer not to damage
> this property while solving other problems. 
> 
>>> - There is a simple way to declare something an acceptable substitute of
>>> something else.
>> Do you mean substitute in the sense nix uses it, i.e. a way to realise a path without building it? Or something else?
> 
> Yes, this. I mean something that can be specified from command line to
> nix-store, not by installing more Perl scripts.
> 

Mm, ok. Did you have an interface in mind?

>>> - There is a simple way to derive a version of package with dependency
>>> versions changed (for use with subsituters, mainly)
>> Can you expand on this here? How would such a feature help?
> 
> I claim (as system administrator) that glibc 2.x.y is a drop-in 
> replacement for 2.x.y-1, and I would like to test this. If I am OK
> with the results, I do a pseudo-rebuild to close the security hole and
> clean up later.
> 

Now that's a complicated but potentially very useful feature. Have you looked into hash-rewriting as a possible solution? You wouldn't even have to break any nix invariants this way.

Maybe I'll look into writing a 'replaceDependencyWithoutRebuild' function.

>>> - There is a way to replace subsituted package with a "better" 
>>> substitution or native build
>> I think I misunderstand what you mean by substitute in the above three issues :)
> 
> I have a Hydra-downloaded package. I notice that it has wrong 
> optimization flags because of a weird property of the build system.
> 
> I want to replace it with a local copy first and rebuild all the 
> depending packages later. (And do a proper fix after some thinking a 
> week later).
> 
> 
>