[Nix-dev] Re: overview about hack-nix and how it works

[Peter Simons]

Hi Marc,

thank you very much for the detailed explanation. I have a few
questions, but I guess it's probably a good idea for me to explain in a
little more detail where I am coming from on this subject.

For the last 12 months or so, I maintained the Haskell repository for
ArchLinux. That effort consisted of two parts:

 - HABS, a repository of binary packages for Pacman, which contained
   approximately 228 popular Haskell packages, and

 - AUR, a repository of build instructions users could use to compile
   these packages themselves, which contained another 2,000 Haskell
   packages.

A team of people developed a whole set of tools to automate that
process, and our experience has been that there are two more or less
independent problems that need to be solved:

 1) Cabal files need to be converted into PKGBUILD files (the ArchLinux
    equivalent of Nix expressions).

 2) The constraints specified in the Cabal files must be resolved to
    figure out a conflict-free set of packages.

Part (1) is quite easy, really. The Cabal library exports functions that
allow anyone to parse Cabal files, and once you have a access to the
underlying data type, build instructions can be dumped out. There is a
bit of an effort involved in fixing mis-specified dependencies and
converting the names of system dependencies to the proper packages of
the package manager, but all things considered that part amounted to
less than 10% of the time we spent maintaining that repository.

Part (2), however, was what took the remaining 90% of the time.
Oftentimes, we performed a trivial upgrade of a small package. That
upgrade, however, led to build failures because some other package
wouldn't accept the newer version. Then we e-mailed upstream and hoped
for an update. When the update arrived, we ran into the next build
failure because some other version restriction had now been broken, etc.
I remember "yesod" being a particularly unpleasant package to maintain,
because it has a plethora of dependencies, and it expects very specific
versions of those, too.

Now, my understanding of hack-nix is as follows:

 > A) generate package pool

That package pool appears to correspond to the task I described under (1).

 > B) use the pool (the magic?)
 > On top of the pool brute force solving of dependencies takes place.

That would be part (2) then. What do you mean by "brute force solving of
dependencies"? How exactly does hack-nix determine which versions can
co-exist?

Also, it is not obvious to me why this whole procedure requires
end-users to run a special tool. Why don't you generate plain Nix
expressions that can be checked into nixpkgs? That way, users of the
repository wouldn't have to bother with hack-nix. Instead, they would be
able to build and install Haskell packages using the standard nix-env
interface, which seems to be very desirable. Personally, I wouldn't want
to use a separate tool to install Haskell packages. Besides, such a tool
already exists, namely cabal-install.

Am I missing something?


 > Its still your task to [...] select additional not latest version
 > [7], or add additional C dependencies (mysql, gtk, ..), see
 > [default.nix] line 216.

So when a package cannot be built the very latest version of its
dependencies, I'll have to choose the version that ought be used
manually. Is that right?

Take care,
Peter