[Nix-dev] Re: [Nix-commits] SVN commit: nix - 18398 - viric - in nixpkgs/branches/stdenv-updates/pkgs: development/libraries/readline stdenv top-level
Nicolas Pierron
nicolas.b.pierron at gmail.com
Wed Nov 18 18:49:04 CET 2009
On Tue, Nov 17, 2009 at 23:58, Llus Batlle <viriketo at gmail.com> wrote:
> Author: viric
> Date: 2009-11-17 22:58:48 +0000 (Tue, 17 Nov 2009)
> New Revision: 18398
>
> You can view the changes in this commit at:
> https://svn.nixos.org/viewvc/nix?rev=18398&view=rev
>
> Modified:
> nixpkgs/branches/stdenv-updates/pkgs/development/libraries/readline/readline6.nix
> nixpkgs/branches/stdenv-updates/pkgs/stdenv/adapters.nix
> nixpkgs/branches/stdenv-updates/pkgs/top-level/all-packages.nix
>
> Log:
> I made the whole nixpkgs dependencies available to the cross compiler, no
> needing to keep a new tree of expressions apart for the expressions to get
> cross-compiled.
Some of the design choices have added a few duplication inside
all-packages.nix. (see comments of the code)
> I changed the whole way of using cross compilation with nixpkgs, which before
> was done through a simple adapter.
As I said many times, cross-compilation should be the default case,
and non-cross-compilation should be the particular case where some
optimization could be done. This caused you some trouble because your
stdenvNoCross does not provide the same feature of the stdenvCross.
The trouble I am talking are what you describe in the following
paragraph:
> Now the adapter became complex, and I've tried to avoid the most obvious
> recursivities. For example, the fetchurl expression should
> never be cross-compiled, as the gmp, mpfr, and some others, like
> some ncurses, perl, ... I made overrided copies of those necessary as
> perlNoCross, ncursesNoCross, as stdenvNoCross, keeping in mind that
> the stdenv (capable of cross compilation) is built upon stdenvNoCross using
> an adapter.
This copies are not necessary and highlight my previous point.
> So, to cross compile, instead of building using "nixpkgs/default.nix",
> you should build with your
> own "myarchiteture.nix", which should have contents like these, for example:
>
> import /etc/nixos/nixpkgs/default.nix
> {
> crossSystem = {
> config = "armv5tel-unknown-linux-gnueabi";
> bigEndian = false;
> arch = "arm";
> float = "soft";
> };
> }
>
>
>
> Changes:
>
> Modified: nixpkgs/branches/stdenv-updates/pkgs/development/libraries/readline/readline6.nix
> ===================================================================
> --- nixpkgs/branches/stdenv-updates/pkgs/development/libraries/readline/readline6.nix 2009-11-17 21:14:57 UTC (rev 18397)
> +++ nixpkgs/branches/stdenv-updates/pkgs/development/libraries/readline/readline6.nix 2009-11-17 22:58:48 UTC (rev 18398)
> @@ -8,7 +8,7 @@
> sha256 = "1pn13j6f9376kwki69050x3zh62yb1w31l37rws5nwr5q02xk68i";
> };
>
> - propagatedBuildInputs = [ncurses];
> + propagatedHostInputs = [ncurses];
>
> patchFlags = "-p0";
> patches =
>
> Modified: nixpkgs/branches/stdenv-updates/pkgs/stdenv/adapters.nix
> ===================================================================
> --- nixpkgs/branches/stdenv-updates/pkgs/stdenv/adapters.nix 2009-11-17 21:14:57 UTC (rev 18397)
> +++ nixpkgs/branches/stdenv-updates/pkgs/stdenv/adapters.nix 2009-11-17 22:58:48 UTC (rev 18398)
> @@ -109,16 +109,38 @@
>
> # Return a modified stdenv that adds a cross compiler to the
> # builds.
> - makeStdenvCross = stdenv: binutilsCross : gccCross: stdenv //
> - { mkDerivation = args: stdenv.mkDerivation (args // {
> -
> - buildInputs =
> - (if args ? buildInputs then args.buildInputs else [])
> - ++ [ gccCross binutilsCross ];
> + makeStdenvCross = stdenv: cross: binutilsCross: gccCross: stdenv //
Why cross does not contains binutils & gcc? Atfer all stdenv contains
gcc and other bin-utils.
And add:
# filter buildInputs and hostInputs based on the system on which they
should be used.
makeStdenvDispatchCross = stdenv:
if stdenv.buildSystem == stdenv.crossSystem then
stdenv
else {
..
buildDrv = (makeStdenvCross stdenv stdenv.buildSystem).mkDerivation { .. }
hostDrv = stdenv.mkDerivation { .. }
}
where
makeStdenvCross provides support for cross compilation in any cases.
> + { mkDerivation = {name, buildInputs ? null, hostInputs ? null,
> + propagatedBuildInputs ? null, propagatedHostInputs ? null, ...}@args: let
{ mkDerivation = {name, buildInputs ? [], hostInputs ? [],
propagatedBuildInputs ? [], propagatedHostInputs ? [],
...}@args: let
# propagatedBuildInputs is meaning less.
> + buildInputsList = if (buildInputs != null) then
> + buildInputs else [];
> + hostInputsList = if (hostInputs != null) then
> + hostInputs else [];
> + propagatedBuildInputsList = if (propagatedBuildInputs != null) then
> + propagatedBuildInputs else [];
> + propagatedHostInputsList = if (propagatedHostInputs != null) then
> + propagatedHostInputs else [];
No longer necessary.
> + buildInputsDrvs = map (drv: drv.buildDrv) buildInputsList;
> + hostInputsDrvs = map (drv: drv.hostDrv) hostInputsList;
> + propagatedBuildInputsDrvs = map (drv: drv.buildDrv) propagatedBuildInputsList;
> + propagatedHostInputsDrvs = map (drv: drv.buildDrv) propagatedHostInputsList;
> + buildDrv = stdenv.mkDerivation (args // {
> + buildInputs = buildInputsDrvs ++ hostInputsDrvs;
hostInputsDrvs contain the list of runtime dependencies, so buildDrv
should not see any packages compiled for the host because buildDrv
will be run on the current system, this is an error except in case of
a cross-compiler where the compiler should be aware that it compiles
against host libraries.
> + propagatedBuildInputs = propagatedBuildInputsDrvs ++
> + propagatedHostInputsDrvs;
> + });
> + hostDrv = if (cross == null) then buildDrv else
> + stdenv.mkDerivation (args // {
> + name = name + "-" + cross.config;
> + buildInputs = buildInputsDrvs
> + ++ [ gccCross binutilsCross ];
>
> + crossConfig = cross.config;
> + });
buildDrv & hostDrv should not have the same stdenv. buildDrv should
have the an stdenv which has (buildSys == hostSys) where the hostDrv
may have an stdenv which has (buildSys != hostSys). By the way
gccCross and binutilsCross should be provided by the stdenv, which
means on at this stage.
> + in hostDrv // {
This may cause a lot of breakage. because packages are used inside strings like:
buildPhase = "
${foo}/bin/foo ...
";
and people are used to install software with nix-env -Ai foo which is
expected to the hostDrv. As the cross compilation is not the most
frequent usage of Nix, I recommend to change this by the buildDrv
knowing that there is no difference when you are not cross-compiling.
> + inherit hostDrv buildDrv;
> + };
> + } // { inherit cross; };
> - crossConfig = gccCross.cross.config;
> - });
> - };
>
> /* Modify a stdenv so that the specified attributes are added to
> every derivation returned by its mkDerivation function.
>
> Modified: nixpkgs/branches/stdenv-updates/pkgs/top-level/all-packages.nix
> ===================================================================
> --- nixpkgs/branches/stdenv-updates/pkgs/top-level/all-packages.nix 2009-11-17 21:14:57 UTC (rev 18397)
> +++ nixpkgs/branches/stdenv-updates/pkgs/top-level/all-packages.nix 2009-11-17 22:58:48 UTC (rev 18398)
> @@ -33,6 +33,7 @@
> # argument. Otherwise, it's read from $NIXPKGS_CONFIG or
> # ~/.nixpkgs/config.nix.
> config ? null
> +, crossSystem ? null
, buildSystem ? ...
, crossSystem ? buildSystem
This have many impact
> }:
>
>
> @@ -215,7 +216,7 @@
>
> defaultStdenv = allStdenvs.stdenv;
>
> - stdenv =
> + stdenvNoCross =
> if bootStdenv != null then bootStdenv else
> let changer = getConfig ["replaceStdenv"] null;
> in if changer != null then
> @@ -225,6 +226,10 @@
> }
> else defaultStdenv;
>
> + stdenv = if (bootStdenv != null || crossSystem == null) then stdenvNoCross else
> + makeStdenvCross stdenvNoCross crossSystem (binutilsCross crossSystem)
> + (gccCrossStageFinal crossSystem);
> +
All stdenv are cross-compiling (even if buildSystem == hostSystem).
You should not do this check here because stdenvNoCross lack a way to
handle cross compiled extensions.
> @@ -239,10 +244,7 @@
> useDietLibC useKlibc makeStaticBinaries addAttrsToDerivation
> keepBuildTree cleanupBuildTree addCoverageInstrumentation makeStdenvCross;
>
> - stdenvCross = cross : makeStdenvCross stdenv (binutilsCross cross)
> - (gccCrossStageFinal cross);
>
> -
> ### BUILD SUPPORT
>
> attrSetToDir = arg : import ../build-support/upstream-updater/attrset-to-dir.nix {
> @@ -304,7 +306,8 @@
> # from being built.
> fetchurl = useFromStdenv "fetchurl"
> (import ../build-support/fetchurl {
> - inherit curl stdenv;
> + curl = curlNoCross;
This line should be removed. curl should either be put inside the
buildInputs of fetchurl or used as usual inside a string. There is no
need to use a curl version which have been declared with stdenvLocal,
because stdenvLocal will ask curl which have been compiled for the
buildSystem.
> + stdenv = stdenvNoCross;
if you provide buildDrv insitead of hostDrv as defualt, the problem is
solved without patching anything.
> });
>
> # fetchurlBoot is used for curl and its dependencies in order to
> @@ -638,13 +641,19 @@
> inherit fetchurl stdenv;
> };
>
> - curl = import ../tools/networking/curl {
> + curl = makeOverridable (import ../tools/networking/curl) {
> fetchurl = fetchurlBoot;
> inherit stdenv zlib openssl;
> zlibSupport = ! ((stdenv ? isDietLibC) || (stdenv ? isStatic));
> sslSupport = ! ((stdenv ? isDietLibC) || (stdenv ? isStatic));
> };
>
> + curlNoCross = curl.override {
> + stdenv = stdenvNoCross;
> + zlib = zlib.override { stdenv = stdenvNoCross; };
> + openssl = opensslNoCross;
> + };
> +
Thus you don't need to duplicate every expression and this is not necessary.
> curlftpfs = import ../tools/networking/curlftpfs {
> inherit fetchurl stdenv fuse curl pkgconfig zlib glib;
> };
> @@ -1066,10 +1075,14 @@
> readline nettools lsof procps;
> };
>
> - lzma = import ../tools/compression/lzma {
> + lzma = makeOverridable (import ../tools/compression/lzma) {
> inherit fetchurl stdenv;
> };
>
> + lzmaNoCross = lzma.override {
> + stdenv = stdenvNoCross;
> + };
> +
> xz = import ../tools/compression/xz {
> inherit fetchurl stdenv lib;
> };
> @@ -1883,13 +1896,17 @@
> gcc43 = useFromStdenv "gcc" gcc43_real;
>
> gcc43_real = lowPrio (wrapGCC (makeOverridable (import ../development/compilers/gcc-4.3) {
> - inherit fetchurl stdenv texinfo gmp mpfr noSysDirs;
> + inherit fetchurl gmp mpfr noSysDirs;
> + stdenv = stdenvNoCross;
> + texinfo = texinfoNoCross;
> profiledCompiler = true;
> }));
>
> gcc43_realCross = cross : makeOverridable (import ../development/compilers/gcc-4.3) {
> #stdenv = overrideGCC stdenv (wrapGCCWith (import ../build-support/gcc-wrapper) glibc_multi gcc);
> - inherit stdenv fetchurl texinfo gmp mpfr noSysDirs cross;
> + inherit fetchurl gmp mpfr noSysDirs cross;
> + stdenv = stdenvNoCross;
> + texinfo = texinfoNoCross;
> binutilsCross = binutilsCross cross;
> glibcCross = glibcCross cross;
> profiledCompiler = false;
> @@ -2326,7 +2343,8 @@
> import ../build-support/gcc-cross-wrapper {
> nativeTools = false;
> nativeLibc = false;
> - inherit stdenv gcc binutils libc shell name cross;
> + inherit gcc binutils libc shell name cross;
> + stdenv = stdenvNoCross;
> };
>
> # FIXME: This is a specific hack for GCC-UPC. Eventually, we may
> @@ -2428,7 +2446,7 @@
> impureLibcPath = if stdenv.isLinux then null else "/usr";
> };
>
> - perl510 = import ../development/interpreters/perl-5.10 {
> + perl510 = makeOverridable (import ../development/interpreters/perl-5.10) {
> inherit stdenv;
> fetchurl = fetchurlBoot;
> impureLibcPath = if stdenv.isLinux then null else "/usr";
> @@ -2436,6 +2454,11 @@
>
> perl = if system != "i686-cygwin" then perl510 else sysPerl;
>
> + perlNoCross = perl.override
> + {
> + stdenv = stdenvNoCross;
> + };
> +
> # FIXME: unixODBC needs patching on Darwin (see darwinports)
> phpOld = import ../development/interpreters/php {
> inherit stdenv fetchurl flex bison libxml2 apacheHttpd;
> @@ -2708,22 +2731,13 @@
> });
>
> binutilsCross = cross : import ../development/tools/misc/binutils {
> - inherit stdenv fetchurl cross;
> + inherit fetchurl cross;
> + stdenv = stdenvNoCross;
> noSysDirs = true;
> };
>
> bison = bison23;
>
> - bisonArm = import ../development/tools/parsing/bison/bison-2.3.nix {
> - inherit fetchurl m4;
> - stdenv = stdenvCross {
> - config = "armv5tel-unknown-linux-gnueabi";
> - bigEndian = false;
> - arch = "arm";
> - float = "soft";
> - };
> - };
> -
> bison1875 = import ../development/tools/parsing/bison/bison-1.875.nix {
> inherit fetchurl stdenv m4;
> };
> @@ -2859,11 +2873,15 @@
>
> m4 = gnum4;
>
> + m4NoCross = m4.override {
> + stdenv = stdenvNoCross;
> + };
> +
> global = import ../development/tools/misc/global {
> inherit fetchurl stdenv;
> };
>
> - gnum4 = import ../development/tools/misc/gnum4 {
> + gnum4 = makeOverridable (import ../development/tools/misc/gnum4) {
> inherit fetchurl stdenv;
> };
>
> @@ -3055,10 +3073,16 @@
> inherit fetchurl stdenv ncurses;
> };
>
> - texinfo = import ../development/tools/misc/texinfo {
> + texinfo = makeOverridable (import ../development/tools/misc/texinfo) {
> inherit fetchurl stdenv ncurses lzma;
> };
>
> + texinfoNoCross = texinfo.override {
> + stdenv = stdenvNoCross;
> + ncurses = ncursesNoCross;
> + lzma = lzmaNoCross;
> + };
> +
> texi2html = import ../development/tools/misc/texi2html {
> inherit fetchurl stdenv perl;
> };
> @@ -3534,7 +3558,8 @@
> };
>
> glibc29Cross = cross : makeOverridable (import ../development/libraries/glibc-2.9) {
> - inherit fetchurl stdenv cross;
> + inherit fetchurl cross;
> + stdenv = stdenvNoCross;
> binutilsCross = binutilsCross cross;
> gccCross = gccCrossStageStatic cross;
> kernelHeaders = kernelHeadersCross cross;
> @@ -3587,7 +3612,9 @@
> };
>
> gmp = import ../development/libraries/gmp {
> - inherit fetchurl stdenv m4;
> + inherit fetchurl;
> + stdenv = stdenvNoCross;
> + m4 = m4NoCross;
> };
>
> # `gmpxx' used to mean "GMP with C++ bindings". Now `gmp' has C++ bindings
> @@ -3609,7 +3636,8 @@
>
> #GMP ex-satellite, so better keep it near gmp
> mpfr = import ../development/libraries/mpfr {
> - inherit fetchurl stdenv gmp;
> + inherit fetchurl gmp;
> + stdenv = stdenvNoCross;
> };
>
> gst_all = recurseIntoAttrs (import ../development/libraries/gstreamer {
> @@ -4320,11 +4348,17 @@
> inherit fetchurl stdenv;
> };
>
> - ncurses = composedArgsAndFun (import ../development/libraries/ncurses) {
> + ncurses = makeOverridable (composedArgsAndFun (import ../development/libraries/ncurses)) {
> inherit fetchurl stdenv;
> - unicode = (system != "i686-cygwin");
> + # The "! (stdenv ? cross)" is for the cross-built arm ncurses, which
> + # don't build for me in unicode.
> + unicode = (system != "i686-cygwin" && ! (stdenv ? cross));
> };
>
> + ncursesNoCross = ncurses.override {
> + stdenv = stdenvNoCross;
> + };
> +
> neon = neon026;
>
> neon026 = import ../development/libraries/neon/0.26.nix {
> @@ -4404,11 +4438,16 @@
> pkgconfig;
> };
>
> - openssl = import ../development/libraries/openssl {
> + openssl = makeOverridable (import ../development/libraries/openssl) {
> fetchurl = fetchurlBoot;
> inherit stdenv perl;
> };
>
> + opensslNoCross = openssl.override {
> + stdenv = stdenvNoCross;
> + perl = perlNoCross;
> + };
> +
> ortp = import ../development/libraries/ortp {
> inherit fetchurl stdenv;
> };
> @@ -4757,7 +4796,7 @@
> inherit ncurses flex bison autoconf automake m4 coreutils;
> };
>
> - zlib = import ../development/libraries/zlib {
> + zlib = makeOverridable (import ../development/libraries/zlib) {
> fetchurl = fetchurlBoot;
> inherit stdenv;
> };
> @@ -5508,7 +5547,9 @@
> kernelHeaders = kernelHeaders_2_6_28;
>
> kernelHeadersCross = cross : import ../os-specific/linux/kernel-headers/2.6.28.nix {
> - inherit fetchurl stdenv perl cross;
> + inherit fetchurl cross;
> + stdenv = stdenvNoCross;
> + perl = perlNoCross;
> };
>
> kernelHeaders_2_6_18 = import ../os-specific/linux/kernel-headers/2.6.18.5.nix {
> @@ -6101,10 +6142,6 @@
> inherit fetchurl stdenv unzip;
> };
>
> - ubootArm = uboot.override {
> - stdenv = stdenvCross "armv5tel-unknown-linux-gnueabi";
> - };
> -
> uclibc = import ../os-specific/linux/uclibc {
> inherit fetchurl stdenv kernelHeaders;
> };
>
--
Nicolas Pierron
http://www.linkedin.com/in/nicolasbpierron - http://nbp.name/
If you are doing something twice then you should try to do it once.
More information about the nix-dev
mailing list