[Nix-dev] haskell dev environment, ghc non-determinism and binary cache interaction

[Miguel Negrão]

Hi Peter,

Peter Simons <simons <at> cryp.to> writes:

> 
> Hi Miguel,
> 
>  > package binary-0.7.4.0 is broken due to missing package
>  >   array-0.5.0.0-470385a50d2b78598af85cfe9d988e1b,
>  >   base-4.7.0.2-bfd89587617e381ae01b8dd7b6c7f1c1,
>  >   bytestring-0.10.4.0-d6f1d17d717e8652498cab8269a0acd5,
>  >   ...
> 
> these errors occur sometimes, when Haskell libraries from different
> sources are mixed, i.e. some libraries come from hydra.cryp.to, others
> from hydra.nixos.org, and some were compiled locally. This is a design
> problem in GHC. Check out [1] for further details (including how to
> recover from this situation).

Ok, because of this I have decided to not use binary cache in case I have to
build packages locally later. After a lot of tryouts and looking at the
output of nix-env ... --dry-run I think I found a commit on 12 of April
where almost everything that I need seems to be building. I will test this
tonight (takes a couple of hours to build locally).

This takes to me to the next question. My current method is to checkout
commits of nixpkg, see the output of nix-env with the hydra.crypt.to binary
cache enabled, and if most of the packages have a binary available I then
build locally without hydra.crypo.to. This is a very cumbersome and time
consuming process. Even if most packages have binaries there are always some
that don't, and I won't know if they build or not until I do it locally,
which can take a while. If they don't build and I don't know how fix them
which is most often the case given my level of expertise, my only option is
to start the process all over again. Some questions:

 - Has someone already made a script to automate the process of looking for
a commit with the maximum amount of packages building in hidra for a
specific expression to be installed ? Or even better, looking for a commit
where all packages that are to be installed are guaranteed to build based on
information from hydra.
 - Maybe I misunderstand how the new haskell infrastructure works, but it
seems to me that the haskell packages are now essentially not curated since
they come from hackage, and patches or changes of version are done manually
only after a while. Doesn't this mean that we will never have a consistent
set of haskell packages in master ? A lot of the search of nixpkgs commits
that I have been doing is related with changes introduced from getting new
versions from hackage that leaves the haskell packages in an inconsistent
state (some don't build). For instance, when diagrams was updated to 1.3
recently, IHaskell was broken because it depends on 1.2. Maybe by the time
the IHaskell maintainer fixes this some other package will update and will
break IHaskell again. Wouldn't it make sense to try to have a consistent set
of haskell packages like stackage ? Perhaps mirroring the versions in
stackage ? (even if this is minimal set of packages compared to the whole of
hackage, it would be a start...). Perhaps I'm misunderstanding things, but
in any case I would like to learn a workflow which is easier then what I'm
currently doing...


best regards,
Miguel