[Nix-dev] applyAndFun again - kernel derivation proposal and example

[Marc Weber]

Hi @ll,

I've found a bug in my code.. And I have to admit that I had a hard time
reading my own code. Anyway I decided to simplify it and rewrite it.
I've tried giving the function a more meaningful name. Maye you like
this more?

To test it simply insert the code somewhere in lib.

changes:
  no longer using passthru, the "special" names are removed before the
  args are applied to the given funtion.
  The function contains the possibility to pass either a function
  (taking the previously supplied args) or attrs hardcoded now.

I even consider proposing using
  overridableDelayableArgs stdenv.mkDerivation { ..
  }
instead of stdenv.mkDerivation by default.

Scroll down to "res" to see examples getting more complex.

  # proposed replacement for applyAndFun (which has a bug calling the merge function twice when supplying attrs instead of a function)
  # the naming "overridableDelayableArgs" tries to express that you can
  # - override attr values which have been supplied earlier
  # - use attr values before they have been supplied by accessing the fix point
  #   name "fixed"
  # f: the (delayed overridden) arguments are applied to this
  #
  # initial: initial attrs arguments and settings. see defaultOverridableDelayableArgs
  #
  # returns: f applied to the arguments // special attributes attrs
  #     a) merge: merge applied args with new args. Wether an argument is overridden depends on the merge settings
  #     b) replace: this let's you replace and remove names no matter which merge function has been set
  #
  # examples: see test cases "res" below;
  overridableDelayableArgs =
          f :        # the function applied to the arguments
          initial :  # you pass attrs, the functions below are passing a function taking the fix argument
    let
        takeFixed = if (__isFunction initial) then initial else (fixed : initial); # transform initial to an expression always taking the fixed argument
        tidy = args : removeAttrs args (maybeAttr  "removeAttrs" [] args); # tidy up args before applying them
        fun = n : x :
             let newArgs = fixed :
                     let args = takeFixed fixed; 
                         mergeFun = __getAttr n args;
                     in if __isAttrs x then (mergeFun args x)
                        else assert __isFunction x;
                             mergeFun args (x ( args // { inherit fixed; }));
             in overridableDelayableArgs f newArgs;
    in
    (f (tidy (fix takeFixed))) // {
      merge   = fun "mergeFun";
      replace = fun "keepFun";
    };
  defaultOverridableDelayableArgs = f : 
      let defaults = {
            mergeFun = mergeAttrByFunc; # default merge function. merge strategie (concatenate lists, strings) is given by mergeAttrBy
            keepFun = a : b : { inherit (a) removeAttrs mergeFun keepFun mergeAttrBy; } // b; # even when using replace preserve these values
            inherit mergeAttrBy;
            removeAttrs = ["mergeFun" "keepFun" "mergeAttrBy" "removeAttrs" "fixed" ]; # before applying the arguments to the function make sure these names are gone
          };
      in (overridableDelayableArgs f defaults).merge;

  res = (removeAttrs (rec {
    res1 = defaultOverridableDelayableArgs id {};
    # {  }

    # merge tests:

    res2 = defaultOverridableDelayableArgs id { a = 7; };
    # { a = 7; }

    res3 = let x = defaultOverridableDelayableArgs id { a = 7; };
          in (x.merge) { b = 10; };
    # { a = 7; b = 10; }

    res4 = let x = defaultOverridableDelayableArgs id { a = 7; };
          in (x.merge) ( x: { b = 10; });
    # { a = 7; b = 10; }

    res5 = let x = defaultOverridableDelayableArgs id { a = 7; };
          in (x.merge) ( x: { a = __add x.a 3; });
    # { a = 10; }

    res6 = let x = defaultOverridableDelayableArgs id { a = 7; mergeAttrBy = { a = __add; }; };
               y = x.merge {};
          in (y.merge) { a = 10; };
    # { a = 17; }

    # using replace to remove an option
    resRem7 = res6.replace (a : removeAttrs a ["a"]);
    # {}

    resReplace6 = let x = defaultOverridableDelayableArgs id { a = 7; mergeAttrBy = { a = __add; }; };
                      x2 = x.merge { a = 20; }; # now we have 27
                  in (x2.replace) { a = 10; }; # and override the value by 10
    # { a = 10; }

    # fixed tests (delayed args): (when using them add some comments, please)
    resFixed1 = 
          let x = defaultOverridableDelayableArgs id ( x : { a = 7; c = x.fixed.b; });
              y = x.merge (x : { name = "name-${builtins.toString x.fixed.c}"; });
          in (y.merge) { b = 10; };
    # { a = 7; b = 10; c =10; name = "name-10"; }

  }.res6) [ "merge" "replace" ]).a;






real world example :
===========================================================

note how fixed is used instead of rec

kernelOld = defaultOverridableDelayableArgs stdenv.mkDerivation (a: let inherit (a) fixed; in {
  version = "2.6.28;

  name = "kernel-${fixed.version}";

  src = fetchurl {
    url = "mirror://kernel.org/kernel-${fixed.version}";
    sha256 = fixed.hash;
  };
  hash = "2304921aoeuaoeu09a8e; # this way its easier to override
  removeAttrs = [ "availiblePatches" "hash"]; # but don't add it to the builder env

  availiblePatches = {
    tuxOnIce = ( ...);
    bar = (..);
    foo = (..);
  };

  patches = [ fixed.availiblePatches.bar ]; # [2]
  
  config = "
    allow = m
    boot =y
  ";

  # default merge actions:
  mergeAttrBy = {
    config = a : b : "${a}\n${b}"; # concatenate config lines by default
    patches = a : b : pkgs.lib.uniq (a ++ b);
  }
});

change the source and version:

  kernelNew = kernelOld.merge {
    version = "2.6.3000-extra";
    sha256 = "new hash";
    config = "
      additional config line
    ";
  }

unsure about wether adding a config line works? So make sure only your configfile is used

  kernelOldMyConfig = kernelOld.replace {
    config = ./my-config;
  }

I've used the kernel example cause most of us already have encountered the
limitations of the default kernel derivation(s). The one written by
MichaelRaskin beeing the most flexible. However you still have to duplicate the
sources ? You alread can guess how easiy it is to enable tuxOnIce:

myKernelTuxOnIce = defaultKernel.merge (a: let inherit (a) fixed; in {
  patches = [ fixed.availiblePatches.tuxOnIce ];
});

Of course you can use replace again to throw away the default patch list etc.
You could even start replacing the patches by different ones:

myKernelTuxOnIce = defaultKernel.merge (a: {
  availiblePatches = a.fixed.availiblePatches // {
    tuxOnIce = ( new patch version );
    bar = ( new patch version); # << will replace the patch within the default patch list above [2]
  };
});

I know its getting more complex.. but I still think it perfectly fits some use
cases..

Of course you can use merge and replace as often as you like:
((d.replace {}).merge {})....

Sincerly
Marc Weber