[Nix-dev] A Journey into our brand-new Haskell infrastructure: Part II

Peter Simons simons at cryp.to
Sun Jan 11 20:41:26 CET 2015


The topic of today's posting is: Fixing Build Failures!


I know all about Cabal builds. How can I override a Nix build environment?
--------------------------------------------------------------------------

  Every Haskell expression expects an argument called "mkDerivation" -- the
  function that builds the derivation from the build description. You can
  modify the environment of a build "foo" by replacing mkDerivation with a
  version that applies some function "f" to the expression first:

  | foo.override (args: args // {
  |   mkDerivation = expr: args.mkDerivation (expr // f expr);
  | })

  All Haskell configuration modules import the 'lib' library
  <https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/haskell-modules/lib.nix>
  that defines a bunch of neat little helper functions on top of this basic
  mechanism. Just check out out the configuration-XYZ.nix files to see some
  examples of how these functions are used.


My build fails with "the following dependencies are missing"!
--------------------------------------------------------------

  The most common build error is that the configure phase complains about
  missing dependencies. <http://hydra.cryp.to/build/354149/nixlog/1/raw>, for
  example, reporting these packages as missing:

  | Configuring AbortT-transformers-1.0.1...
  | Setup: At least the following dependencies are missing:
  | QuickCheck >=2.4 && <2.6,
  | test-framework ==0.6.*,
  | test-framework-hunit ==0.2.*,
  | test-framework-quickcheck2 ==0.2.*

  "Missing" means that these libraries are available in the build environment,
  but the build isn't happy about their versions. Usually, the Cabal files
  specifies upper bounds that our versions exceed, i.e. our packages are *too
  new*.

  Now, the big question is whether those upper bounds are justified or whether
  they exist solely because upstream hasn't updated the Cabal file recently? An
  easy to test this is to add a "jailbreak" and to run the build again:
  <https://github.com/peti/nixpkgs/commit/0dd413458ef1a5c05e64bee2462de2edfe0fe620>,
  If it succeeds now, then this fact should be communicated upstream:
  <https://github.com/gcross/AbortT-transformers/issues/1>. If you commit the
  override afterwards, then please include a reference to the upstream ticket
  in a comment, or at least add a brief comment explaining briefly why that
  jailbreak was added!

  A trickier case was reported on the nix-dev mailing list in
  <http://permalink.gmane.org/gmane.linux.distributions.nixos/15526>:

  | Configuring cabal-test-quickcheck-0.1.2...
  | Setup: At least the following dependencies are missing:
  | Cabal ==1.20.*

  First of all, let's check which Cabal version we have during the build:

  | $ nix-shell '<nixpkgs>' -A haskellngPackages.cabal-test-quickcheck.env --command "ghc-pkg list Cabal"
  | /nix/store/a3dj9sjg5lh9wxl90lj4shp9s3brlscd-ghc-7.8.4/lib/ghc-7.8.4/package.conf.d
  |   Cabal-1.18.1.5

  'Cabal' is available, indeed, but our version doesn't meet the "==1.20.*"
  constraint specified by 'cabal-test-quickcheck'. Curiously enough, the
  package's Cabal file imposes the following constraints on the library:

  | library
  |   [...]
  |   build-depends:
  |     Cabal      >= 1.16.0 && < 1.21,

  This is strange, right? Our Cabal library fits into that range! It turns out
  that there is a test suite -- which we enable by default --, and that says:

  | test-suite example
  |   [...]
  |   build-depends:
  |     Cabal      >= 1.20 && < 1.21,

  The test suite imposes a narrower constraint than the library itself. An easy
  way out of this situation is to just disable the testing phase: constraint:
  <https://github.com/NixOS/nixpkgs/commit/7fa32aecd183f4fcede821a2b0c60a925c0e2ef5>.

  Similarly, libraries may imposes additional restrictions when certain
  optional features are enabled during the build, i.e. features controlled by a
  Cabal flag. The "aeson" library uses this mechanism to choose between "time"
  versions before and after 1.5:

  | if flag(old-locale)
  |   build-depends: time < 1.5, old-locale
  | else
  |   build-depends: time >= 1.5

  We have '-fold-locale' hard-coded for that build in hackage-packages.nix (for
  reasons that will be addressed in a later installment), so aeson wouldn't
  build with ghc-7.9.x, because that compiler ships time 1.5.0.1. The solution
  in this case is to disable the "old-locale" for the the 7.9.x branch:
  <https://github.com/NixOS/nixpkgs/blob/2ff8d1940f0986b572760edf1923539687f41ac8/pkgs/development/haskell-modules/configuration-ghc-7.9.x.nix#L52>.


I tried "jailbreak", but that broke the build even more than before!
--------------------------------------------------------------------

  Removing dependency restrictions from a Cabal file sensibly is hard, and
  jailbreak-cabal sometimes doesn't succeed doing that. In these cases, it may
  be worth trying to patch the offending constraints out of Cabal file
  manually. For example, the build of "darcs" cannot be fixed by jailbreak
  alone, so instead we added a custom "patchPhase" that uses 'sed' to show the
  build who's the boss:
  <https://github.com/NixOS/nixpkgs/blob/7fa32aecd183f4fcede821a2b0c60a925c0e2ef5/pkgs/development/haskell-modules/configuration-common.nix#L91>.


If a "jailbroken" build succeeds, does this mean the resulting binary is okay?
------------------------------------------------------------------------------

  This is almost always the case. Haskell is a strongly typed language, right?
  So a successful compiler run actualy means something. In some cases, however,
  package authors consciously exclude certain versions of the dependencies for
  other reasons, i.e. because these versions have a bug that their software
  triggers. In this case, a jailbroken build would succeed, but the binaries
  that come out of it might be broken in subtle and non-obvious way.

  Generally speaking, jailbreaking is fine if, and only if, you report the fact
  that you had to do that upstream to give them a chance to comment.


I tried jailbreaking, and the package doesn't compile with the newer dependency.
--------------------------------------------------------------------------------

  Report this issue upstream. Ask the package authors to, please, release a new
  version of the library that supports the latest versions of its respective
  dependencies. Refer them to <http://packdeps.haskellers.com/> if they don't
  believe you. If they refuse, tell them that they are a bunch of mindless
  jerks who will be the first against the wall when the revolution comes.

  If that doesn't help, then it's also possible to build the package with those
  old versions that they want, apparently, but doing that isn't easy. The magic
  keyword is "deep override". This subject will be covered in a separate
  installment.


My build failed while "Preprocessing test suite XYZ"?
-----------------------------------------------------

  This is almost certainly a bug in the package: the test suite tries to import
  a Haskell module that's missing from the release tarball, i.e. because the
  Cabal file doesn't mention that file as a required source code. This happens
  when developers build their code in their Git checkout (which contains the
  file) but don't whether the build still succeeds in solely from the release
  tarball that's generated by "cabal sdist". A real-life example of such an
  issue is <http://hydra.cryp.to/build/349188/nixlog/2/raw>. The proper way to
  fix this bug is to report it upstream. I've done that at
  <https://github.com/techtangents/ablist/issues/1>. If the author fixes the
  problem and releases an update, then the new version will find its way into
  our package database automatically with 1-2 days.

  Now, if you can't wait, then it's possible to disable the test suite for this
  particular build in pkgs/development/haskell-modules/configuration-common.nix:
  <https://github.com/peti/nixpkgs/commit/1d223754dee2dab61d4d143c1cbdf17289613a10>.

  If you add an override, then please *add a comment* that refers to an
  upstream ticket, or at least describe what the problem was that this override
  is supposed to solve. This is important, because manually added overrides
  tend to become unnecessary after a while, and then it's very hard to figure
  out why they were added in the first place and whether it's safe to remove
  them or not. This bit of extra information can be very helpful!


My build fails because of a "missing dependency on a foreign library"!
----------------------------------------------------------------------

  Consider the build failure <http://hydra.cryp.to/build/350912/nixlog/2/raw>
  of the package ALUT:

  | Setup: Missing dependency on a foreign library:
  | * Missing C library: alut

  The corresponding build expression in "hackage-packages.nix" was:

  | "ALUT" = callPackage
  |   ({ mkDerivation, alut, base, OpenAL, OpenGL }:
  |    mkDerivation {
  |      pname = "ALUT";
  |      version = "2.3.0.2";
  |      sha256 = "02kfyb4g7sfjfzqlddxqbjffrj4a0gfrzkisdpbj2lw67j1gq5dp";
  |      buildDepends = [ base OpenAL OpenGL ];
  |      extraLibraries = [ alut ];
  |      configureFlags = [ "-fusenativewindowslibraries" ];
  |      homepage = "https://github.com/haskell-openal/ALUT";
  |      description = "A binding for the OpenAL Utility Toolkit";
  |      license = stdenv.lib.licenses.bsd3;
  |    }) { alut = null; };

  The package specifies "alut" in its extraLibraries section, but unfortunately
  that argument is later hard-coded to "null"! The reason for that "alut = null"
  override is that the hackage2nix utility -- which generates this expression
  -- couldn't find any package called "alut" in Nixpkgs. Instead, we have
  "freealut". This issue is best be fixed in the hackage2nix source code:
  <https://github.com/NixOS/cabal2nix/commit/dd0e5c08cb8c5bd471db1843e387467eb4e2abd2>.
  With that change applied to the code, hackage2nix generates the change
  <https://github.com/peti/nixpkgs/commit/b362dc8fdf038db64c48218a88f207231937bfab>,
  which fixes the build.

  Hackage2nix will be discussed in-depth in a later installment. If you run
  into an issue like this one, then feel free to open a ticket on Github at
  <https://github.com/NixOS/cabal2nix/issues/new>.

  Another case was "sfml-audio", which failed with the following error:

  | Setup: Missing dependency on a foreign library:
  | * Missing (or bad) header file: al.h

  The library that provides this header -- openal -- was part of the build
  environment, but Cabal still wouldn't find the header because it lives in a
  sub-directory that the package author clearly didn't expect. This problem
  could be solved by adding the appropriate directory to the header search path:
  <https://github.com/peti/nixpkgs/blob/0dd413458ef1a5c05e64bee2462de2edfe0fe620/pkgs/development/haskell-modules/configuration-common.nix#L40>.


That's it for today!

Have fun,
Peter



More information about the nix-dev mailing list