[Nix-dev] A Journey into our brand-new Haskell infrastructure: Part II
Peter Simons
simons at cryp.to
Sun Jan 11 20:41:26 CET 2015
The topic of today's posting is: Fixing Build Failures!
I know all about Cabal builds. How can I override a Nix build environment?
--------------------------------------------------------------------------
Every Haskell expression expects an argument called "mkDerivation" -- the
function that builds the derivation from the build description. You can
modify the environment of a build "foo" by replacing mkDerivation with a
version that applies some function "f" to the expression first:
| foo.override (args: args // {
| mkDerivation = expr: args.mkDerivation (expr // f expr);
| })
All Haskell configuration modules import the 'lib' library
<https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/haskell-modules/lib.nix>
that defines a bunch of neat little helper functions on top of this basic
mechanism. Just check out out the configuration-XYZ.nix files to see some
examples of how these functions are used.
My build fails with "the following dependencies are missing"!
--------------------------------------------------------------
The most common build error is that the configure phase complains about
missing dependencies. <http://hydra.cryp.to/build/354149/nixlog/1/raw>, for
example, reporting these packages as missing:
| Configuring AbortT-transformers-1.0.1...
| Setup: At least the following dependencies are missing:
| QuickCheck >=2.4 && <2.6,
| test-framework ==0.6.*,
| test-framework-hunit ==0.2.*,
| test-framework-quickcheck2 ==0.2.*
"Missing" means that these libraries are available in the build environment,
but the build isn't happy about their versions. Usually, the Cabal files
specifies upper bounds that our versions exceed, i.e. our packages are *too
new*.
Now, the big question is whether those upper bounds are justified or whether
they exist solely because upstream hasn't updated the Cabal file recently? An
easy to test this is to add a "jailbreak" and to run the build again:
<https://github.com/peti/nixpkgs/commit/0dd413458ef1a5c05e64bee2462de2edfe0fe620>,
If it succeeds now, then this fact should be communicated upstream:
<https://github.com/gcross/AbortT-transformers/issues/1>. If you commit the
override afterwards, then please include a reference to the upstream ticket
in a comment, or at least add a brief comment explaining briefly why that
jailbreak was added!
A trickier case was reported on the nix-dev mailing list in
<http://permalink.gmane.org/gmane.linux.distributions.nixos/15526>:
| Configuring cabal-test-quickcheck-0.1.2...
| Setup: At least the following dependencies are missing:
| Cabal ==1.20.*
First of all, let's check which Cabal version we have during the build:
| $ nix-shell '<nixpkgs>' -A haskellngPackages.cabal-test-quickcheck.env --command "ghc-pkg list Cabal"
| /nix/store/a3dj9sjg5lh9wxl90lj4shp9s3brlscd-ghc-7.8.4/lib/ghc-7.8.4/package.conf.d
| Cabal-1.18.1.5
'Cabal' is available, indeed, but our version doesn't meet the "==1.20.*"
constraint specified by 'cabal-test-quickcheck'. Curiously enough, the
package's Cabal file imposes the following constraints on the library:
| library
| [...]
| build-depends:
| Cabal >= 1.16.0 && < 1.21,
This is strange, right? Our Cabal library fits into that range! It turns out
that there is a test suite -- which we enable by default --, and that says:
| test-suite example
| [...]
| build-depends:
| Cabal >= 1.20 && < 1.21,
The test suite imposes a narrower constraint than the library itself. An easy
way out of this situation is to just disable the testing phase: constraint:
<https://github.com/NixOS/nixpkgs/commit/7fa32aecd183f4fcede821a2b0c60a925c0e2ef5>.
Similarly, libraries may imposes additional restrictions when certain
optional features are enabled during the build, i.e. features controlled by a
Cabal flag. The "aeson" library uses this mechanism to choose between "time"
versions before and after 1.5:
| if flag(old-locale)
| build-depends: time < 1.5, old-locale
| else
| build-depends: time >= 1.5
We have '-fold-locale' hard-coded for that build in hackage-packages.nix (for
reasons that will be addressed in a later installment), so aeson wouldn't
build with ghc-7.9.x, because that compiler ships time 1.5.0.1. The solution
in this case is to disable the "old-locale" for the the 7.9.x branch:
<https://github.com/NixOS/nixpkgs/blob/2ff8d1940f0986b572760edf1923539687f41ac8/pkgs/development/haskell-modules/configuration-ghc-7.9.x.nix#L52>.
I tried "jailbreak", but that broke the build even more than before!
--------------------------------------------------------------------
Removing dependency restrictions from a Cabal file sensibly is hard, and
jailbreak-cabal sometimes doesn't succeed doing that. In these cases, it may
be worth trying to patch the offending constraints out of Cabal file
manually. For example, the build of "darcs" cannot be fixed by jailbreak
alone, so instead we added a custom "patchPhase" that uses 'sed' to show the
build who's the boss:
<https://github.com/NixOS/nixpkgs/blob/7fa32aecd183f4fcede821a2b0c60a925c0e2ef5/pkgs/development/haskell-modules/configuration-common.nix#L91>.
If a "jailbroken" build succeeds, does this mean the resulting binary is okay?
------------------------------------------------------------------------------
This is almost always the case. Haskell is a strongly typed language, right?
So a successful compiler run actualy means something. In some cases, however,
package authors consciously exclude certain versions of the dependencies for
other reasons, i.e. because these versions have a bug that their software
triggers. In this case, a jailbroken build would succeed, but the binaries
that come out of it might be broken in subtle and non-obvious way.
Generally speaking, jailbreaking is fine if, and only if, you report the fact
that you had to do that upstream to give them a chance to comment.
I tried jailbreaking, and the package doesn't compile with the newer dependency.
--------------------------------------------------------------------------------
Report this issue upstream. Ask the package authors to, please, release a new
version of the library that supports the latest versions of its respective
dependencies. Refer them to <http://packdeps.haskellers.com/> if they don't
believe you. If they refuse, tell them that they are a bunch of mindless
jerks who will be the first against the wall when the revolution comes.
If that doesn't help, then it's also possible to build the package with those
old versions that they want, apparently, but doing that isn't easy. The magic
keyword is "deep override". This subject will be covered in a separate
installment.
My build failed while "Preprocessing test suite XYZ"?
-----------------------------------------------------
This is almost certainly a bug in the package: the test suite tries to import
a Haskell module that's missing from the release tarball, i.e. because the
Cabal file doesn't mention that file as a required source code. This happens
when developers build their code in their Git checkout (which contains the
file) but don't whether the build still succeeds in solely from the release
tarball that's generated by "cabal sdist". A real-life example of such an
issue is <http://hydra.cryp.to/build/349188/nixlog/2/raw>. The proper way to
fix this bug is to report it upstream. I've done that at
<https://github.com/techtangents/ablist/issues/1>. If the author fixes the
problem and releases an update, then the new version will find its way into
our package database automatically with 1-2 days.
Now, if you can't wait, then it's possible to disable the test suite for this
particular build in pkgs/development/haskell-modules/configuration-common.nix:
<https://github.com/peti/nixpkgs/commit/1d223754dee2dab61d4d143c1cbdf17289613a10>.
If you add an override, then please *add a comment* that refers to an
upstream ticket, or at least describe what the problem was that this override
is supposed to solve. This is important, because manually added overrides
tend to become unnecessary after a while, and then it's very hard to figure
out why they were added in the first place and whether it's safe to remove
them or not. This bit of extra information can be very helpful!
My build fails because of a "missing dependency on a foreign library"!
----------------------------------------------------------------------
Consider the build failure <http://hydra.cryp.to/build/350912/nixlog/2/raw>
of the package ALUT:
| Setup: Missing dependency on a foreign library:
| * Missing C library: alut
The corresponding build expression in "hackage-packages.nix" was:
| "ALUT" = callPackage
| ({ mkDerivation, alut, base, OpenAL, OpenGL }:
| mkDerivation {
| pname = "ALUT";
| version = "2.3.0.2";
| sha256 = "02kfyb4g7sfjfzqlddxqbjffrj4a0gfrzkisdpbj2lw67j1gq5dp";
| buildDepends = [ base OpenAL OpenGL ];
| extraLibraries = [ alut ];
| configureFlags = [ "-fusenativewindowslibraries" ];
| homepage = "https://github.com/haskell-openal/ALUT";
| description = "A binding for the OpenAL Utility Toolkit";
| license = stdenv.lib.licenses.bsd3;
| }) { alut = null; };
The package specifies "alut" in its extraLibraries section, but unfortunately
that argument is later hard-coded to "null"! The reason for that "alut = null"
override is that the hackage2nix utility -- which generates this expression
-- couldn't find any package called "alut" in Nixpkgs. Instead, we have
"freealut". This issue is best be fixed in the hackage2nix source code:
<https://github.com/NixOS/cabal2nix/commit/dd0e5c08cb8c5bd471db1843e387467eb4e2abd2>.
With that change applied to the code, hackage2nix generates the change
<https://github.com/peti/nixpkgs/commit/b362dc8fdf038db64c48218a88f207231937bfab>,
which fixes the build.
Hackage2nix will be discussed in-depth in a later installment. If you run
into an issue like this one, then feel free to open a ticket on Github at
<https://github.com/NixOS/cabal2nix/issues/new>.
Another case was "sfml-audio", which failed with the following error:
| Setup: Missing dependency on a foreign library:
| * Missing (or bad) header file: al.h
The library that provides this header -- openal -- was part of the build
environment, but Cabal still wouldn't find the header because it lives in a
sub-directory that the package author clearly didn't expect. This problem
could be solved by adding the appropriate directory to the header search path:
<https://github.com/peti/nixpkgs/blob/0dd413458ef1a5c05e64bee2462de2edfe0fe620/pkgs/development/haskell-modules/configuration-common.nix#L40>.
That's it for today!
Have fun,
Peter
More information about the nix-dev
mailing list