[Nix-dev] Overriding packages (was Re: Kernel headers)

[Andres Loeh]

>> But I still think that we can do almost everything one could want
>> with the current language and without reflection.
>
> I do not agree.  Reflexivity can reduce redundancy and help to create
> new evaluation schemes in order to express things in a clear way.
>
> The reflexivity is one powerful feature that can be easily enable with
> the laziness and which can provide more ""user-land"" syntactic sugar.
>  I have one small example which is used by the option merger in
> pkgs/lib/default.nix.  This example may represents a service (in
> NixOS) which use another service only if it is already enable because
> it is not necessary.

Then we can agree to disagree. Granted, reflection is powerful, but
I simply don't see that it's needed for a domain-specific language
like Nix. Reflection on the other hand makes it very difficult to
judge what's going on. It becomes much harder to add a static type
system to the Nix language, or even perform some amounts of static
analysis on Nix expressions in order to guarantee that they behave
as expected.

Laziness usually gives you a lot of abstraction power, including the
possibility to define control flow operators. It seems strange that
Nix expressions of all language should actually require reflection,
where many general-purpose programming languages can do fine without.

> ------- thisService.nix
> config: pkgs:
> {
>  # extra definitions if the service is enable.
>  services = if ! config.services.thisService.enable then {} else {
>    requiredService = { enable = true; };
>    # this test will raise a circular definition.
>    extraService = if ! config.services.extraService.enable then {} else {
>      option1 = true;
>      option2 = false;
>      ...
>    };
>  };
> }

I'd like to understand this, but I don't ...
I have the feeling that if I'd understand the problem you're describing
here, I could maybe come up with a solution that doesn't need reflection.

> The second test verify that the "extraService" is enabled by the user
> or really required by another service. It will raise a circular
> evaluation because the condition evaluates an attribute which can be
> defined inside the "then" or "else" part.

Which attribute? What causes the circular evaluation? I can only
see references to "config", nothing circular. Is the whole thing
supposed to be passed to "fix" again?

> To skip this problem we can
> move the condition to each attribute definition.

Which condition?

> Therefore the
> circular evaluation can be caught and a function can try to spread the
> if statement on the attributes before trying to evaluate the set
> again.  Moving the if statement inside the set implies that we can
> detect if statements and act on them.

I don't understand this sentence.

> Currently we can create statements by writing nix-expressions but it
> could be much more interesting to match statements as well as
> attribute names.  I understand that "listToAttr" is not user friendly.
> May be we can replace this by a reflexive syntax that can manipulate
> all the language and easily support language extension like:
>
> spreadIfStatement = arg:
>
>  # match a "if then else" statement
>  match arg with  if cond then thenPart else elsePart  do
>
>    match thenPart with
>      # if the thenPart is an empty set
>      {} do ...
>
>      # if the thenPart is not an empty set.
>    | { $name = value; } // thenNextPart do
>        ( if elsePart ? name then
>            { $name = if cond then thenPart.$name else elsePart.$name; }
>          else
>            { inherit (thenPart) $name; }
>        ) // (spreadIfStatement (if cond then thenNextPart else elsePart));
>
> This function does not support all cases but the idea is to add the
> "match <expr> with <expr> do <expr> [ | <expr> do <expr> ]" statement
> in the language to handle the reflexivity directly in the language.
> The previous sample use another language extension which allow to
> replace an attribute name by an attribute evaluation.
>
> By the way, some primitive operation can be rewritten like this:
>
> head = list: match list with [head] ++ tail do head;
> tail = list: match list with [head] ++ tail do tail;
>
> Even if this is not efficient this could become a good documentation
> and the "match" statement can be make generic enough to support all
> future language extensions.

Don't get me wrong. Meta-programming like this is an interesting topic,
and developing languages with such features might be a worthwhile goal.
But it's not simple. And I just don't see why we need something like
this in Nix. I don't think that Nix has the requirement to be particularly
adaptable to lots of different situations, so that we need extensible
syntax etc. It has a specific and limited goal.

Cheers,
  Andres