[Nix-dev] Overriding packages (was Re: Kernel headers)

[Andres Loeh]

>> I brainstormed with Andres a while ago about how to make Nixpkgs more
>> configurable without polluting it with zillions of options, which is related
>> to this.
>
> Who is Andres ?

That'd be me ...

[...]

> So why not making ~/.nixpkgs/config.nix a new part of
> all-packages.nix, instead of creating an attribute "override" to
> rewrite some parts of it ?

If you mean that the current purpose of ~/.nixpkgs/config.nix
subsumed by the "overrides" scheme, then you're right. Everything
could be done via overrides of all-packages.nix.

> This sound to be a bit redundant if the user wants to set an option of
> a package.  So why not writing things like:
>
> ## override
> pkgs: {
>  subversion = { drvFun = import .../subversion;
>    stdenv = pkgs.overrideGCC pkgs.stdenv pkgs.gcc34;
>    bdbSupport = true;
>    httpServer = true;
>    sslSupport = false;
>    ... more silly options ...
>  };
>
>  # this is the same except that there is no getConfig require. (override)
>  git = {
>    svnSupport = true;
>  };
> }
>
> Therefore with an "override" merge which take the latest definition of
> the attribute you would be able to keep the last definition before
> calling the derivation fun with its own arguments.  The merge could
> also substitute the subversion attribute by a call to its derivation
> function with its own set of arguments.
>
> evalAttrDrvFun = name: attr: listToAttr [{
>  name = name;
>  value = attr.drvFun attr;
> }];
>
> Therefore any reference to subversion attribute will refer to the
> derivation as it is currently.

I'm afraid anything involving listToAttr is too complicated for me.
At least I don't understand this ...

>>    pkgsFun = pkgs:
>>      { # <-- no rec
>>        kernelHeaders = pkgs.kernelHeaders_2_6_23;
>>        kernelHeaders_2_6_18 = ...;
>>        kernelHeaders_2_6_23 = ...;
>>        somePackage = ... {
>>          inherit (pkgs) kernelHeaders;
>>        };
>>      } // (getConfig "overrides" {}) pkgs;
>>
>>    # Tie the knot.  This works thanks to laziness.
>>    pkgs = pkgsFun pkgs;
>
> hum .. pkgs.lib.finalReference ?

I've looked this up:

finalReference = f: (rec { result = f result; }).result;

The usual name for this is "fix", but yes, that's the same
function. You can then say

pkgs = fix pkgsFun;

>> it would work.  But this has the downside that you have to write "pkgs" everywhere (so it would make all-packages.nix more verbose).
>
> Yes, this kind of configuration is quite verbose if you want to refer
> to the override attributes.

I don't see this as a problem. With the "..." extension Eelco has implemented,
most all-packages entries probably just become

pkgsFun = pkgs : {
  ...
  mypackage = (import mypackage.nix) pkgs;
  ...
}

anyway.

>> It might seem
>> that writing pkgsFun as "pkgsFun = pkgs: with pkgs; {...}" would solve this,
>> but then you get an infinite recursion as "with" isn't lazy enough.
>
> What cause this problem ? Can this be solved ?

Probably, "with pkgs; expr" does currently evaluate "pkgs" to weak head normal
form (i.e., evaluates it until it knows it's an attribute set and what the
attribute names are). It then adds the attributes to the environment
and continues
evaluating "expr". Fixing this probably forces environments themselves to become
lazy, and that's certainly some work to implement, and might also have impacts
on efficiency. Nevertheless, it might be worth trying.

>> Maybe we need some language extension to allow attributes in recs to be
>> overriden properly...
>
> I am not sure that creating directly a language extension would be a
> good solution because someone will always arise with a new issue.  May
> be we should concentrate our effort at findings the good solution with
> the current syntax instead of searching new syntax that could help for
> one specific kind of writing.  Otherwise, I can suggest something like
> "%%" which will substitute the older definition by the new definition
> at the opposite of "//" which will only mask the older definition with
> the new one.

I'm also not sure, but ultimately, I'd rather like a more powerful
merging operator built into the language and have all the reflection
operators (converting attribute sets to lists and vice versa) removed
instead.

But I still think that we can do almost everything one could want
with the current language and without reflection.

Cheers,
  Andres