[Nix-dev] A Journey into our brand-new Haskell infrastructure: Part II

Thomas Hunger tehunger at gmail.com
Mon Jan 12 00:28:56 CET 2015


Thanks, that's super useful!

One more question: How would I get older versions of certain packages (e.g.
I need optparse-applicative 0.10 for elm-make) into hackage-packages.nix?

~

On 11 January 2015 at 19:41, Peter Simons <simons at cryp.to> wrote:

> The topic of today's posting is: Fixing Build Failures!
>
>
> I know all about Cabal builds. How can I override a Nix build environment?
> --------------------------------------------------------------------------
>
>   Every Haskell expression expects an argument called "mkDerivation" -- the
>   function that builds the derivation from the build description. You can
>   modify the environment of a build "foo" by replacing mkDerivation with a
>   version that applies some function "f" to the expression first:
>
>   | foo.override (args: args // {
>   |   mkDerivation = expr: args.mkDerivation (expr // f expr);
>   | })
>
>   All Haskell configuration modules import the 'lib' library
>   <
> https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/haskell-modules/lib.nix
> >
>   that defines a bunch of neat little helper functions on top of this basic
>   mechanism. Just check out out the configuration-XYZ.nix files to see some
>   examples of how these functions are used.
>
>
> My build fails with "the following dependencies are missing"!
> --------------------------------------------------------------
>
>   The most common build error is that the configure phase complains about
>   missing dependencies. <http://hydra.cryp.to/build/354149/nixlog/1/raw>,
> for
>   example, reporting these packages as missing:
>
>   | Configuring AbortT-transformers-1.0.1...
>   | Setup: At least the following dependencies are missing:
>   | QuickCheck >=2.4 && <2.6,
>   | test-framework ==0.6.*,
>   | test-framework-hunit ==0.2.*,
>   | test-framework-quickcheck2 ==0.2.*
>
>   "Missing" means that these libraries are available in the build
> environment,
>   but the build isn't happy about their versions. Usually, the Cabal files
>   specifies upper bounds that our versions exceed, i.e. our packages are
> *too
>   new*.
>
>   Now, the big question is whether those upper bounds are justified or
> whether
>   they exist solely because upstream hasn't updated the Cabal file
> recently? An
>   easy to test this is to add a "jailbreak" and to run the build again:
>   <
> https://github.com/peti/nixpkgs/commit/0dd413458ef1a5c05e64bee2462de2edfe0fe620
> >,
>   If it succeeds now, then this fact should be communicated upstream:
>   <https://github.com/gcross/AbortT-transformers/issues/1>. If you commit
> the
>   override afterwards, then please include a reference to the upstream
> ticket
>   in a comment, or at least add a brief comment explaining briefly why that
>   jailbreak was added!
>
>   A trickier case was reported on the nix-dev mailing list in
>   <http://permalink.gmane.org/gmane.linux.distributions.nixos/15526>:
>
>   | Configuring cabal-test-quickcheck-0.1.2...
>   | Setup: At least the following dependencies are missing:
>   | Cabal ==1.20.*
>
>   First of all, let's check which Cabal version we have during the build:
>
>   | $ nix-shell '<nixpkgs>' -A haskellngPackages.cabal-test-quickcheck.env
> --command "ghc-pkg list Cabal"
>   |
> /nix/store/a3dj9sjg5lh9wxl90lj4shp9s3brlscd-ghc-7.8.4/lib/ghc-7.8.4/package.conf.d
>   |   Cabal-1.18.1.5
>
>   'Cabal' is available, indeed, but our version doesn't meet the "==1.20.*"
>   constraint specified by 'cabal-test-quickcheck'. Curiously enough, the
>   package's Cabal file imposes the following constraints on the library:
>
>   | library
>   |   [...]
>   |   build-depends:
>   |     Cabal      >= 1.16.0 && < 1.21,
>
>   This is strange, right? Our Cabal library fits into that range! It turns
> out
>   that there is a test suite -- which we enable by default --, and that
> says:
>
>   | test-suite example
>   |   [...]
>   |   build-depends:
>   |     Cabal      >= 1.20 && < 1.21,
>
>   The test suite imposes a narrower constraint than the library itself. An
> easy
>   way out of this situation is to just disable the testing phase:
> constraint:
>   <
> https://github.com/NixOS/nixpkgs/commit/7fa32aecd183f4fcede821a2b0c60a925c0e2ef5
> >.
>
>   Similarly, libraries may imposes additional restrictions when certain
>   optional features are enabled during the build, i.e. features controlled
> by a
>   Cabal flag. The "aeson" library uses this mechanism to choose between
> "time"
>   versions before and after 1.5:
>
>   | if flag(old-locale)
>   |   build-depends: time < 1.5, old-locale
>   | else
>   |   build-depends: time >= 1.5
>
>   We have '-fold-locale' hard-coded for that build in hackage-packages.nix
> (for
>   reasons that will be addressed in a later installment), so aeson wouldn't
>   build with ghc-7.9.x, because that compiler ships time 1.5.0.1. The
> solution
>   in this case is to disable the "old-locale" for the the 7.9.x branch:
>   <
> https://github.com/NixOS/nixpkgs/blob/2ff8d1940f0986b572760edf1923539687f41ac8/pkgs/development/haskell-modules/configuration-ghc-7.9.x.nix#L52
> >.
>
>
> I tried "jailbreak", but that broke the build even more than before!
> --------------------------------------------------------------------
>
>   Removing dependency restrictions from a Cabal file sensibly is hard, and
>   jailbreak-cabal sometimes doesn't succeed doing that. In these cases, it
> may
>   be worth trying to patch the offending constraints out of Cabal file
>   manually. For example, the build of "darcs" cannot be fixed by jailbreak
>   alone, so instead we added a custom "patchPhase" that uses 'sed' to show
> the
>   build who's the boss:
>   <
> https://github.com/NixOS/nixpkgs/blob/7fa32aecd183f4fcede821a2b0c60a925c0e2ef5/pkgs/development/haskell-modules/configuration-common.nix#L91
> >.
>
>
> If a "jailbroken" build succeeds, does this mean the resulting binary is
> okay?
>
> ------------------------------------------------------------------------------
>
>   This is almost always the case. Haskell is a strongly typed language,
> right?
>   So a successful compiler run actualy means something. In some cases,
> however,
>   package authors consciously exclude certain versions of the dependencies
> for
>   other reasons, i.e. because these versions have a bug that their software
>   triggers. In this case, a jailbroken build would succeed, but the
> binaries
>   that come out of it might be broken in subtle and non-obvious way.
>
>   Generally speaking, jailbreaking is fine if, and only if, you report the
> fact
>   that you had to do that upstream to give them a chance to comment.
>
>
> I tried jailbreaking, and the package doesn't compile with the newer
> dependency.
>
> --------------------------------------------------------------------------------
>
>   Report this issue upstream. Ask the package authors to, please, release
> a new
>   version of the library that supports the latest versions of its
> respective
>   dependencies. Refer them to <http://packdeps.haskellers.com/> if they
> don't
>   believe you. If they refuse, tell them that they are a bunch of mindless
>   jerks who will be the first against the wall when the revolution comes.
>
>   If that doesn't help, then it's also possible to build the package with
> those
>   old versions that they want, apparently, but doing that isn't easy. The
> magic
>   keyword is "deep override". This subject will be covered in a separate
>   installment.
>
>
> My build failed while "Preprocessing test suite XYZ"?
> -----------------------------------------------------
>
>   This is almost certainly a bug in the package: the test suite tries to
> import
>   a Haskell module that's missing from the release tarball, i.e. because
> the
>   Cabal file doesn't mention that file as a required source code. This
> happens
>   when developers build their code in their Git checkout (which contains
> the
>   file) but don't whether the build still succeeds in solely from the
> release
>   tarball that's generated by "cabal sdist". A real-life example of such an
>   issue is <http://hydra.cryp.to/build/349188/nixlog/2/raw>. The proper
> way to
>   fix this bug is to report it upstream. I've done that at
>   <https://github.com/techtangents/ablist/issues/1>. If the author fixes
> the
>   problem and releases an update, then the new version will find its way
> into
>   our package database automatically with 1-2 days.
>
>   Now, if you can't wait, then it's possible to disable the test suite for
> this
>   particular build in
> pkgs/development/haskell-modules/configuration-common.nix:
>   <
> https://github.com/peti/nixpkgs/commit/1d223754dee2dab61d4d143c1cbdf17289613a10
> >.
>
>   If you add an override, then please *add a comment* that refers to an
>   upstream ticket, or at least describe what the problem was that this
> override
>   is supposed to solve. This is important, because manually added overrides
>   tend to become unnecessary after a while, and then it's very hard to
> figure
>   out why they were added in the first place and whether it's safe to
> remove
>   them or not. This bit of extra information can be very helpful!
>
>
> My build fails because of a "missing dependency on a foreign library"!
> ----------------------------------------------------------------------
>
>   Consider the build failure <
> http://hydra.cryp.to/build/350912/nixlog/2/raw>
>   of the package ALUT:
>
>   | Setup: Missing dependency on a foreign library:
>   | * Missing C library: alut
>
>   The corresponding build expression in "hackage-packages.nix" was:
>
>   | "ALUT" = callPackage
>   |   ({ mkDerivation, alut, base, OpenAL, OpenGL }:
>   |    mkDerivation {
>   |      pname = "ALUT";
>   |      version = "2.3.0.2";
>   |      sha256 = "02kfyb4g7sfjfzqlddxqbjffrj4a0gfrzkisdpbj2lw67j1gq5dp";
>   |      buildDepends = [ base OpenAL OpenGL ];
>   |      extraLibraries = [ alut ];
>   |      configureFlags = [ "-fusenativewindowslibraries" ];
>   |      homepage = "https://github.com/haskell-openal/ALUT";
>   |      description = "A binding for the OpenAL Utility Toolkit";
>   |      license = stdenv.lib.licenses.bsd3;
>   |    }) { alut = null; };
>
>   The package specifies "alut" in its extraLibraries section, but
> unfortunately
>   that argument is later hard-coded to "null"! The reason for that "alut =
> null"
>   override is that the hackage2nix utility -- which generates this
> expression
>   -- couldn't find any package called "alut" in Nixpkgs. Instead, we have
>   "freealut". This issue is best be fixed in the hackage2nix source code:
>   <
> https://github.com/NixOS/cabal2nix/commit/dd0e5c08cb8c5bd471db1843e387467eb4e2abd2
> >.
>   With that change applied to the code, hackage2nix generates the change
>   <
> https://github.com/peti/nixpkgs/commit/b362dc8fdf038db64c48218a88f207231937bfab
> >,
>   which fixes the build.
>
>   Hackage2nix will be discussed in-depth in a later installment. If you run
>   into an issue like this one, then feel free to open a ticket on Github at
>   <https://github.com/NixOS/cabal2nix/issues/new>.
>
>   Another case was "sfml-audio", which failed with the following error:
>
>   | Setup: Missing dependency on a foreign library:
>   | * Missing (or bad) header file: al.h
>
>   The library that provides this header -- openal -- was part of the build
>   environment, but Cabal still wouldn't find the header because it lives
> in a
>   sub-directory that the package author clearly didn't expect. This problem
>   could be solved by adding the appropriate directory to the header search
> path:
>   <
> https://github.com/peti/nixpkgs/blob/0dd413458ef1a5c05e64bee2462de2edfe0fe620/pkgs/development/haskell-modules/configuration-common.nix#L40
> >.
>
>
> That's it for today!
>
> Have fun,
> Peter
>
> _______________________________________________
> nix-dev mailing list
> nix-dev at lists.science.uu.nl
> http://lists.science.uu.nl/mailman/listinfo/nix-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.science.uu.nl/pipermail/nix-dev/attachments/20150111/5c0378c5/attachment-0001.html 


More information about the nix-dev mailing list