[Nix-dev] Re: [Nix-commits] SVN commit: nix - 18351 - viric - in nixpkgs/branches/stdenv-updates/pkgs: development/compilers/gcc-4.3 development/libraries/glibc-2.9 os-specific/linux/kernel-headers stdenv/linux top-level

[Nicolas Pierron]

Hi list,

On Sun, Nov 15, 2009 at 06:28, Llus Batlle <viriketo at gmail.com> wrote:
> @@ -2701,6 +2730,11 @@
>
>   bison = bison23;
>
> +  bisonArm = import ../development/tools/parsing/bison/bison-2.3.nix {
> +    inherit fetchurl m4;
> +    stdenv = stdenvCross "armv5tel-unknown-linux-gnueabi";
> +  };
> +
>   bison1875 = import ../development/tools/parsing/bison/bison-1.875.nix {
>     inherit fetchurl stdenv m4;
>   };

As I've reported on IRC, I don't think having multiple stdenv is a
good choice.  This small parts of all-packages highlight the problem.
You don't want to duplicate each package for each target.

On IRC, I've suggested to consider the usual compilation as a special
case of the cross compilation and not considering the
cross-compilation as a special case of the usual compilation.  This
suggestion implies that stdenv have to be modified to handle the host
and the target of the compilation.  Such information will be provided
as argument of all-packages.nix and will be used to update stdenv.
Cross-compiled programs need to have 2 sets of inputs, to express the
host inputs (build time dependencies) and the target inputs (run time
dependencies).  Such distinctions could help us to get information to
package developers by reporting any build time dependency leaking in
the run time.  Thus a derivation will look like:

{stdenv, fetchurl, foo, bar, baz}:

stdenv.mkDerivation {
  name = "..";
  src = fetchurl { .. };
  buildInputs = [ foo bar ];
  runInputs = [ bar baz ];

  ... /* hidden complexity */ ...
}

In order to make this derivation working for most packages, we should
change stdenv like that:

stdenvCross = stdenv: stdenv // {
  mkDerivation = {buildInputs, runInputs, ...}@args: let
    hostInputs = map (drv: drv.host) buildInputs;
    targetInputs = map (drv: drv.target) runInputs;
    hostDrv = stdenv.mkDerivation (args // {..});
    targetDrv = stdenv.mkDerivation (args // {..});
  in hostDrv // {
    inherit hostDrv tagertDrv;
  };
  host = system;
  target = targetSystem;
};

Such stdenv modifier will add two attributes to the resulting
derivation which define the host-derivation and the target-derivation.
 We need the host-derivation if some programs are needed to generate
code for the target (make, bison, gcc, bash, ..) and we need the
target derivation to make the program working on the target (glibc,
python, ...).  When this stdenv is not used for cross-compiling, the
host-derivation and the target-derivation would be identical which
should avoid extra duplication.

Package arguments don't need to be duplicated because each derivation
contains a hostDrv & targetDrv attributes for each need.  So we will
not end-up with

  foo = import ../path/to/foo {
    inherit stdenv fetchurl bison;
  };

  fooArm = import ../path/to/foo {
    inherit fetchurl;
    stdenv = stdenvCross "armv5tel-unknown-linux-gnueabi";
    bison = bisonArm; # because bison is a run time dependency of foo.
  };

  foox86_64 = import ../path/to/foo {
    inherit fetchurl;
    stdenv = stdenvCross "x86_64-pc-linux";
    bison = bisonx86_64; # because bison is a run time dependency of foo.
  };

  fooi686 = import ../path/to/foo {
    inherit fetchurl;
    stdenv = stdenvCross "i686-pc-linux";
    bison = bisoni686; # because bison is a run time dependency of foo.
  };

which leak package information into all-packages.nix and cause a lot
of duplications to handle a lot of cases which are similar. (let's
factor that)

Of course many programs may not handle cross-compilation, and this can
be handle easily inside each derivation by failing before even
starting the compilation:

{stdenv, ...}:

assert stdenv.host == stdenv.target;

stdenv.mkDerivation {
  ...
}

Thus, with such solution we avoid the duplication of all-packages.nix,
and we add support for cross compilation in each package without
changing packages arguments.

-- 
Nicolas Pierron
http://www.linkedin.com/in/nicolasbpierron - http://nbp.name/
If you are doing something twice then you should try to do it once.