[Nix-dev] Haskell packages in the binary cache broken?

Wed May 15 22:00:25 CEST 2013

Okay, here it goes.

I used:
* nixpkgs rev 880a386c5793269a30e052521b60bdf60b09a406
* nixos rev 4f71bce69150bdea42a9374663e1eaaa5a40b7fe

Haskell and Haskell-related packages in configuration.nix:

~~~~
haskellPackages.xmonad
haskellPackages.xmonadContrib
haskellPackages.xmonadExtras
haskellPackages.xmobar
haskellPackages.ghc
haskellPackages.ghcMod
haskellPackages.hlint
haskellPackages.Agda
haskellPackages.AgdaExecutable
haskellPackages.cabalInstall
haskellPackages.cabalDev
haskellPackages.cabal2nix
emacs24Packages.haskellMode
haskellPackages.Agda
haskellPackages.lhs2tex
gitAndTools.gitAnnex
~~~~

Let us call this package set "PackA".
"PackB" is the same set without ghcMod and gitAnnex packages.

What I did:

# Stage 1

* in configuration.nix:
  * turned binary caches on
  * set up PackA
* in nixpkgs:
  * patched it with `nixpkgs.kernel.patch` and
    `nixpkgs.cabal-hack.patch` (see attachments)
* `nixos-rebuild switch -I /etc/nixos` no errors, but the stuff like

~~~~
Reading package info from "/nix/store/p0f4nxm3hka2a3lh8gg6rlyh80bvfrw7-haskell-yesod-default-ghc7.4.2-1.1.3.2/lib/ghc-pkgs/ghc-7.4.2/yesod-default-1.1.3.2.conf" ... done.
yesod-default-1.1.3.2: dependency "base-4.5.1.0-66f22db3dfcd87541c9c7e50e7095d26" doesn't exist (ignoring)
yesod-default-1.1.3.2: dependency "bytestring-0.9.2.1-503e91bb155301fdb1956cb5c26ce6e9" doesn't exist (ignoring)
yesod-default-1.1.3.2: dependency "data-default-0.5.3-f33230611d35c41f344389ab4c476cef" doesn't exist (ignoring)
yesod-default-1.1.3.2: dependency "directory-1.1.0.2-90bfc47dbd5ac1313aec9ce76cc22338" doesn't exist (ignoring)
yesod-default-1.1.3.2: dependency "hamlet-1.1.7.1-5ffd5408241a9a0144b161973befc7c9" doesn't exist (ignoring)
yesod-default-1.1.3.2: dependency "network-conduit-1.0.0-fc421e22f5536e3396342a474f605274" doesn't exist (ignoring)
yesod-default-1.1.3.2: dependency "safe-0.3.3-2e88224aa98500bc4dcbbdfb58ba6a17" doesn't exist (ignoring)
yesod-default-1.1.3.2: dependency "shakespeare-css-1.0.5.1-01356a2b7c933a23f07754d676e43dc4" doesn't exist (ignoring)
yesod-default-1.1.3.2: dependency "shakespeare-js-1.1.4-df8d20a192aaf25a188095cbf6730024" doesn't exist (ignoring)
yesod-default-1.1.3.2: dependency "template-haskell-2.7.0.0-0279fb7127ab32769a6ce931c5de0612" doesn't exist (ignoring)
yesod-default-1.1.3.2: dependency "text-0.11.2.3-db61832d0c4660614c4ceff234ed4abb" doesn't exist (ignoring)
yesod-default-1.1.3.2: dependency "transformers-0.3.0.0-8e66ecc7d4dae2b07b2b5406908c70e4" doesn't exist (ignoring)
yesod-default-1.1.3.2: dependency "unix-2.5.1.1-4e7768e25de45a8e5884eadf2508728d" doesn't exist (ignoring)
yesod-default-1.1.3.2: dependency "unordered-containers-0.2.3.1-55f822141197e7a6076afa5f7137f232" doesn't exist (ignoring)
yesod-default-1.1.3.2: dependency "wai-1.4.0-fcd64a81573994b7d1a26e1d346e5991" doesn't exist (ignoring)
yesod-default-1.1.3.2: dependency "wai-extra-1.3.4-f9061a440972497b8e6b2b0423dbe4a6" doesn't exist (ignoring)
yesod-default-1.1.3.2: dependency "warp-1.3.8.1-87b52bd0ef7fb361869a63acb97471f4" doesn't exist (ignoring)
yesod-default-1.1.3.2: dependency "yaml-0.8.2.4-33015ed4519c8e42e3c9081898486443" doesn't exist (ignoring)
yesod-default-1.1.3.2: dependency "yesod-core-1.1.8.3-dd99bf53d710a7854c79e24db3462ba4" doesn't exist (ignoring)
Writing new package config file... done.
post-installation fixup
stripping (with flags -S) in  /nix/store/p0f4nxm3hka2a3lh8gg6rlyh80bvfrw7-haskell-yesod-default-ghc7.4.2-1.1.3.2/lib  /nix/store/p0f4nxm3hka2a3lh8gg6rlyh80bvfrw7-haskell-yesod-default-ghc7.4.2-1.1.3.2/bin 
strip:/nix/store/p0f4nxm3hka2a3lh8gg6rlyh80bvfrw7-haskell-yesod-default-ghc7.4.2-1.1.3.2/lib/ghc-pkgs/ghc-7.4.2/yesod-default-1.1.3.2.conf: File format not recognized
strip:/nix/store/p0f4nxm3hka2a3lh8gg6rlyh80bvfrw7-haskell-yesod-default-ghc7.4.2-1.1.3.2/lib/ghc-pkgs/ghc-7.4.2/yesod-default-1.1.3.2.installedconf: File format not recognized
strip:/nix/store/p0f4nxm3hka2a3lh8gg6rlyh80bvfrw7-haskell-yesod-default-ghc7.4.2-1.1.3.2/lib/yesod-default-1.1.3.2/ghc-7.4.2/Yesod/Default/Config.hi: File format not recognized
strip:/nix/store/p0f4nxm3hka2a3lh8gg6rlyh80bvfrw7-haskell-yesod-default-ghc7.4.2-1.1.3.2/lib/yesod-default-1.1.3.2/ghc-7.4.2/Yesod/Default/Main.hi: File format not recognized
strip:/nix/store/p0f4nxm3hka2a3lh8gg6rlyh80bvfrw7-haskell-yesod-default-ghc7.4.2-1.1.3.2/lib/yesod-default-1.1.3.2/ghc-7.4.2/Yesod/Default/Handlers.hi: File format not recognized
strip:/nix/store/p0f4nxm3hka2a3lh8gg6rlyh80bvfrw7-haskell-yesod-default-ghc7.4.2-1.1.3.2/lib/yesod-default-1.1.3.2/ghc-7.4.2/Yesod/Default/Util.hi: File format not recognized
patching ELF executables and libraries
patching script interpreter paths
~~~~

shown up quite a lot in the log.

I have only a part of this log (I started recording it too late and
internet connection broke a couple of times on the way), but some
quite readable parts of it are in the attachements (stage-1*.log).

* `nix-store --verify --check-contents` no errors
* `reboot`
* `nix-collect-garbage -d`
* resulting `/etc/nix/nix.conf` is in attachement (stage-1.nix.conf).

This gave me the system with all Haskell packages build locally and no
other Haskell packages in the nix store.

After that I

# Stage 2

* reverted `nixpkgs.cabal-hack.patch` in nixpkgs
* set up PackB in configuration.nix
* `nixos-rebuild build -I /etc/nixos`  

This installed everything that is to be built with cabal except for
ghcMod and gitAnnex from the binary cache, no errors. See stage-2.log.

* `nix-store --verify --check-contents` no errors

Then again I

# Stage 3

* used `nixpkgs.cabal-hack.patch` on nixpkgs
* and set up PackA in configuration.nix
* but this time I set `use-binary-caches = false` in configuration.nix
* `nixos-rebuild switch -I /etc/nixos` no package rebuilds, no errors,
* resulting `/etc/nix/nix.conf` is in attachement (stage-3.nix.conf).

This gave me Stage 1 system with binary caches turned off.

And finally, I

# Stage 4

* reverted `nixpkgs.cabal-hack.patch` in nixpkgs again
* `nixos-rebuild build -I /etc/nixos --keep-going`

This rebuild PackA with everything except for ghcMod, gitAnnex, and
their dependencies (quite a lot of them), installed from the binary
cache.

This failed as I expected. See stage-4.log.

* `nix-store --verify --check-contents` shows no errors, by the way

On Wed, 15 May 2013 05:02:22 +0000
Jan Malakhovski <oxij at oxij.org> wrote:

> Hi Peter,
> 
> No, I'm running NixOS, and yes, I don't use chroot bulids.
> I'm rebuilding everything twice currently to show you the logs.
> It's going to take till evening or even tomorrow I think.
> 
> Cheers,
>   Jan Malakhovski
> 
> On Tue, 14 May 2013 21:57:18 +0200
> Peter Simons <simons at cryp.to> wrote:
> 
> > Hi Karn,
> > 
> >  > Yes, I meant the ABI hashes (visible in the GHC package database).
> > 
> > ah, okay, that makes sense.
> > 
> > Personally, I'm having trouble believing that this particular feature of
> > GHC is causing those build errors, though, because I've run thousands of
> > Haskell builds with Nix, and I've not run into this issue once. My guess
> > is that Jan uses Nix inside of another Linux distribution -- i.e. he
> > probably doesn't use NixOS nor chroot() builds --, and that there is
> > some other impurity causing these issues. Anyway, it's probably moot to
> > speculate, it would be best if we had a concrete build log to try to
> > figure out what's going on.
> > 
> > Take care,
> > Peter
> > 
> > _______________________________________________
> > nix-dev mailing list
> > nix-dev at lists.science.uu.nl
> > http://lists.science.uu.nl/mailman/listinfo/nix-dev
> 
> 
> _______________________________________________
> nix-dev mailing list
> nix-dev at lists.science.uu.nl
> http://lists.science.uu.nl/mailman/listinfo/nix-dev

-------------- next part --------------
A non-text attachment was scrubbed...
Name: nixpkgs.kernel.patch
Type: application/octet-stream
Size: 720 bytes
Desc: not available
Url : http://lists.science.uu.nl/pipermail/nix-dev/attachments/20130515/978d00ea/attachment-0008.obj 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: nixpkgs.cabal-hack.patch
Type: application/octet-stream
Size: 574 bytes
Desc: not available
Url : http://lists.science.uu.nl/pipermail/nix-dev/attachments/20130515/978d00ea/attachment-0009.obj 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: stage-1.nix.conf
Type: application/octet-stream
Size: 757 bytes
Desc: not available
Url : http://lists.science.uu.nl/pipermail/nix-dev/attachments/20130515/978d00ea/attachment-0010.obj 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: stage-3.nix.conf
Type: application/octet-stream
Size: 783 bytes
Desc: not available
Url : http://lists.science.uu.nl/pipermail/nix-dev/attachments/20130515/978d00ea/attachment-0011.obj 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: stage-4.log.gz
Type: application/octet-stream
Size: 303054 bytes
Desc: not available
Url : http://lists.science.uu.nl/pipermail/nix-dev/attachments/20130515/978d00ea/attachment-0012.obj 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: stage-1.1.log.gz
Type: application/octet-stream
Size: 54528 bytes
Desc: not available
Url : http://lists.science.uu.nl/pipermail/nix-dev/attachments/20130515/978d00ea/attachment-0013.obj 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: stage-1.2.log.gz
Type: application/octet-stream
Size: 97953 bytes
Desc: not available
Url : http://lists.science.uu.nl/pipermail/nix-dev/attachments/20130515/978d00ea/attachment-0014.obj 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: stage-2.log.gz
Type: application/octet-stream
Size: 33953 bytes
Desc: not available
Url : http://lists.science.uu.nl/pipermail/nix-dev/attachments/20130515/978d00ea/attachment-0015.obj 