[Nix-dev] A Journey into our brand-new Haskell infrastructure: Part II
Thomas Hunger
tehunger at gmail.com
Mon Jan 12 00:28:56 CET 2015
Thanks, that's super useful!
One more question: How would I get older versions of certain packages (e.g.
I need optparse-applicative 0.10 for elm-make) into hackage-packages.nix?
~
On 11 January 2015 at 19:41, Peter Simons <simons at cryp.to> wrote:
> The topic of today's posting is: Fixing Build Failures!
>
>
> I know all about Cabal builds. How can I override a Nix build environment?
> --------------------------------------------------------------------------
>
> Every Haskell expression expects an argument called "mkDerivation" -- the
> function that builds the derivation from the build description. You can
> modify the environment of a build "foo" by replacing mkDerivation with a
> version that applies some function "f" to the expression first:
>
> | foo.override (args: args // {
> | mkDerivation = expr: args.mkDerivation (expr // f expr);
> | })
>
> All Haskell configuration modules import the 'lib' library
> <
> https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/haskell-modules/lib.nix
> >
> that defines a bunch of neat little helper functions on top of this basic
> mechanism. Just check out out the configuration-XYZ.nix files to see some
> examples of how these functions are used.
>
>
> My build fails with "the following dependencies are missing"!
> --------------------------------------------------------------
>
> The most common build error is that the configure phase complains about
> missing dependencies. <http://hydra.cryp.to/build/354149/nixlog/1/raw>,
> for
> example, reporting these packages as missing:
>
> | Configuring AbortT-transformers-1.0.1...
> | Setup: At least the following dependencies are missing:
> | QuickCheck >=2.4 && <2.6,
> | test-framework ==0.6.*,
> | test-framework-hunit ==0.2.*,
> | test-framework-quickcheck2 ==0.2.*
>
> "Missing" means that these libraries are available in the build
> environment,
> but the build isn't happy about their versions. Usually, the Cabal files
> specifies upper bounds that our versions exceed, i.e. our packages are
> *too
> new*.
>
> Now, the big question is whether those upper bounds are justified or
> whether
> they exist solely because upstream hasn't updated the Cabal file
> recently? An
> easy to test this is to add a "jailbreak" and to run the build again:
> <
> https://github.com/peti/nixpkgs/commit/0dd413458ef1a5c05e64bee2462de2edfe0fe620
> >,
> If it succeeds now, then this fact should be communicated upstream:
> <https://github.com/gcross/AbortT-transformers/issues/1>. If you commit
> the
> override afterwards, then please include a reference to the upstream
> ticket
> in a comment, or at least add a brief comment explaining briefly why that
> jailbreak was added!
>
> A trickier case was reported on the nix-dev mailing list in
> <http://permalink.gmane.org/gmane.linux.distributions.nixos/15526>:
>
> | Configuring cabal-test-quickcheck-0.1.2...
> | Setup: At least the following dependencies are missing:
> | Cabal ==1.20.*
>
> First of all, let's check which Cabal version we have during the build:
>
> | $ nix-shell '<nixpkgs>' -A haskellngPackages.cabal-test-quickcheck.env
> --command "ghc-pkg list Cabal"
> |
> /nix/store/a3dj9sjg5lh9wxl90lj4shp9s3brlscd-ghc-7.8.4/lib/ghc-7.8.4/package.conf.d
> | Cabal-1.18.1.5
>
> 'Cabal' is available, indeed, but our version doesn't meet the "==1.20.*"
> constraint specified by 'cabal-test-quickcheck'. Curiously enough, the
> package's Cabal file imposes the following constraints on the library:
>
> | library
> | [...]
> | build-depends:
> | Cabal >= 1.16.0 && < 1.21,
>
> This is strange, right? Our Cabal library fits into that range! It turns
> out
> that there is a test suite -- which we enable by default --, and that
> says:
>
> | test-suite example
> | [...]
> | build-depends:
> | Cabal >= 1.20 && < 1.21,
>
> The test suite imposes a narrower constraint than the library itself. An
> easy
> way out of this situation is to just disable the testing phase:
> constraint:
> <
> https://github.com/NixOS/nixpkgs/commit/7fa32aecd183f4fcede821a2b0c60a925c0e2ef5
> >.
>
> Similarly, libraries may imposes additional restrictions when certain
> optional features are enabled during the build, i.e. features controlled
> by a
> Cabal flag. The "aeson" library uses this mechanism to choose between
> "time"
> versions before and after 1.5:
>
> | if flag(old-locale)
> | build-depends: time < 1.5, old-locale
> | else
> | build-depends: time >= 1.5
>
> We have '-fold-locale' hard-coded for that build in hackage-packages.nix
> (for
> reasons that will be addressed in a later installment), so aeson wouldn't
> build with ghc-7.9.x, because that compiler ships time 1.5.0.1. The
> solution
> in this case is to disable the "old-locale" for the the 7.9.x branch:
> <
> https://github.com/NixOS/nixpkgs/blob/2ff8d1940f0986b572760edf1923539687f41ac8/pkgs/development/haskell-modules/configuration-ghc-7.9.x.nix#L52
> >.
>
>
> I tried "jailbreak", but that broke the build even more than before!
> --------------------------------------------------------------------
>
> Removing dependency restrictions from a Cabal file sensibly is hard, and
> jailbreak-cabal sometimes doesn't succeed doing that. In these cases, it
> may
> be worth trying to patch the offending constraints out of Cabal file
> manually. For example, the build of "darcs" cannot be fixed by jailbreak
> alone, so instead we added a custom "patchPhase" that uses 'sed' to show
> the
> build who's the boss:
> <
> https://github.com/NixOS/nixpkgs/blob/7fa32aecd183f4fcede821a2b0c60a925c0e2ef5/pkgs/development/haskell-modules/configuration-common.nix#L91
> >.
>
>
> If a "jailbroken" build succeeds, does this mean the resulting binary is
> okay?
>
> ------------------------------------------------------------------------------
>
> This is almost always the case. Haskell is a strongly typed language,
> right?
> So a successful compiler run actualy means something. In some cases,
> however,
> package authors consciously exclude certain versions of the dependencies
> for
> other reasons, i.e. because these versions have a bug that their software
> triggers. In this case, a jailbroken build would succeed, but the
> binaries
> that come out of it might be broken in subtle and non-obvious way.
>
> Generally speaking, jailbreaking is fine if, and only if, you report the
> fact
> that you had to do that upstream to give them a chance to comment.
>
>
> I tried jailbreaking, and the package doesn't compile with the newer
> dependency.
>
> --------------------------------------------------------------------------------
>
> Report this issue upstream. Ask the package authors to, please, release
> a new
> version of the library that supports the latest versions of its
> respective
> dependencies. Refer them to <http://packdeps.haskellers.com/> if they
> don't
> believe you. If they refuse, tell them that they are a bunch of mindless
> jerks who will be the first against the wall when the revolution comes.
>
> If that doesn't help, then it's also possible to build the package with
> those
> old versions that they want, apparently, but doing that isn't easy. The
> magic
> keyword is "deep override". This subject will be covered in a separate
> installment.
>
>
> My build failed while "Preprocessing test suite XYZ"?
> -----------------------------------------------------
>
> This is almost certainly a bug in the package: the test suite tries to
> import
> a Haskell module that's missing from the release tarball, i.e. because
> the
> Cabal file doesn't mention that file as a required source code. This
> happens
> when developers build their code in their Git checkout (which contains
> the
> file) but don't whether the build still succeeds in solely from the
> release
> tarball that's generated by "cabal sdist". A real-life example of such an
> issue is <http://hydra.cryp.to/build/349188/nixlog/2/raw>. The proper
> way to
> fix this bug is to report it upstream. I've done that at
> <https://github.com/techtangents/ablist/issues/1>. If the author fixes
> the
> problem and releases an update, then the new version will find its way
> into
> our package database automatically with 1-2 days.
>
> Now, if you can't wait, then it's possible to disable the test suite for
> this
> particular build in
> pkgs/development/haskell-modules/configuration-common.nix:
> <
> https://github.com/peti/nixpkgs/commit/1d223754dee2dab61d4d143c1cbdf17289613a10
> >.
>
> If you add an override, then please *add a comment* that refers to an
> upstream ticket, or at least describe what the problem was that this
> override
> is supposed to solve. This is important, because manually added overrides
> tend to become unnecessary after a while, and then it's very hard to
> figure
> out why they were added in the first place and whether it's safe to
> remove
> them or not. This bit of extra information can be very helpful!
>
>
> My build fails because of a "missing dependency on a foreign library"!
> ----------------------------------------------------------------------
>
> Consider the build failure <
> http://hydra.cryp.to/build/350912/nixlog/2/raw>
> of the package ALUT:
>
> | Setup: Missing dependency on a foreign library:
> | * Missing C library: alut
>
> The corresponding build expression in "hackage-packages.nix" was:
>
> | "ALUT" = callPackage
> | ({ mkDerivation, alut, base, OpenAL, OpenGL }:
> | mkDerivation {
> | pname = "ALUT";
> | version = "2.3.0.2";
> | sha256 = "02kfyb4g7sfjfzqlddxqbjffrj4a0gfrzkisdpbj2lw67j1gq5dp";
> | buildDepends = [ base OpenAL OpenGL ];
> | extraLibraries = [ alut ];
> | configureFlags = [ "-fusenativewindowslibraries" ];
> | homepage = "https://github.com/haskell-openal/ALUT";
> | description = "A binding for the OpenAL Utility Toolkit";
> | license = stdenv.lib.licenses.bsd3;
> | }) { alut = null; };
>
> The package specifies "alut" in its extraLibraries section, but
> unfortunately
> that argument is later hard-coded to "null"! The reason for that "alut =
> null"
> override is that the hackage2nix utility -- which generates this
> expression
> -- couldn't find any package called "alut" in Nixpkgs. Instead, we have
> "freealut". This issue is best be fixed in the hackage2nix source code:
> <
> https://github.com/NixOS/cabal2nix/commit/dd0e5c08cb8c5bd471db1843e387467eb4e2abd2
> >.
> With that change applied to the code, hackage2nix generates the change
> <
> https://github.com/peti/nixpkgs/commit/b362dc8fdf038db64c48218a88f207231937bfab
> >,
> which fixes the build.
>
> Hackage2nix will be discussed in-depth in a later installment. If you run
> into an issue like this one, then feel free to open a ticket on Github at
> <https://github.com/NixOS/cabal2nix/issues/new>.
>
> Another case was "sfml-audio", which failed with the following error:
>
> | Setup: Missing dependency on a foreign library:
> | * Missing (or bad) header file: al.h
>
> The library that provides this header -- openal -- was part of the build
> environment, but Cabal still wouldn't find the header because it lives
> in a
> sub-directory that the package author clearly didn't expect. This problem
> could be solved by adding the appropriate directory to the header search
> path:
> <
> https://github.com/peti/nixpkgs/blob/0dd413458ef1a5c05e64bee2462de2edfe0fe620/pkgs/development/haskell-modules/configuration-common.nix#L40
> >.
>
>
> That's it for today!
>
> Have fun,
> Peter
>
> _______________________________________________
> nix-dev mailing list
> nix-dev at lists.science.uu.nl
> http://lists.science.uu.nl/mailman/listinfo/nix-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.science.uu.nl/pipermail/nix-dev/attachments/20150111/5c0378c5/attachment-0001.html
More information about the nix-dev
mailing list