[Nix-dev] Re: overview about hack-nix and how it works

[Peter Simons]

Hi Marc,

 >> It is not obvious to me why this whole procedure requires
 >> end-users to run a special tool.
 >
 > Because if you upgrade (or force a specific package version) all
 > dependencies can change.

yes, that is true. However, that property is not at all specific to
Haskell packages. Every other package may change its required build
inputs from one version to the next. Obviously, Nix can deal with that
just fine. So why would you want to treat Haskell packages differently?


 > TARGET -> P1,P2
 >
 > P1 -> B 2.0 or 1.0 -> C
 >
 > P2 -> B 1.0 -> C
 >
 > Let's have a look at what the solver does when trying to build TARGET:
 >
 > It finds TARGET in the pool, then continues resolving P1 first, then P2.
 > When resolving P1 it'll also visit deps finding B-2.0 and C.

In Nix, that kind of choice is statically encoded in the package
database. In hack-nix, however, it seems like dependency resolution
takes place dynamically, i.e. when hack-nix is called. That would be a
fundamentally different system. Now, I'm sure a package manager that
works like this can be very useful, but I don't see how this has
anything to do with Haskell, nor do I believe that this kind of thing
should be written in the Nix language. Nix is great for specifying build
instructions, but it's not a general purpose programming language.


 > That's why I don't try to generate on set of .nix files. Patching
 > cabal to dump .nix expression was my first attempt. But I gave up on
 > it soon. Instead this all is done on the fly as needed.

I don't see how that follows. There is a finite number of ways in which
the package set can be permuted, so it is clearly possible to configure
all those combinations in static Nix files. Builds in Nix are
*functions*, so they are really good at being re-used in different
configurations. Doing these things "on the fly" doesn't gain anything,
it just causes the tool to be slow and inefficient, because it makes the
same decisions over and over again, every time it's called.


 > I agree that it most cases you can get along with having one set of
 > packages (cabal-packages.nix demonstrates it).

I trust you mean haskell-packages.nix? Yes, that function works really
well. It supports a significant number of packages in several
architectures and several different versions of GHC.


 > But as soon as you want to escape you have to start from scratch. And
 > that's when hack-nix starts paying you back each minute you've spend
 > on it.

Why would anyone want to "escape" that? What would you like to do that
haskell-packages.nix doesn't allow you to?


 >> Why don't you generate plain Nix expressions that can be checked into nixpkgs?
 >
 > So the short answer is: Because in the worst case you would have to
 > create cabal-packages.nix for each target package and maintain them
 > all because one needs newer bytestring, but the other don't compile
 > with that newer one.

I beg to differ. haskell-packages.nix is a *function*. That expression
is extremely generic, and you can customize it for all kinds of
different situations without ever editing the source code. In the
presence of override, you can change virtually anything about that
package set without editing haskell-packages.nix itself.


 > The interesting point to note is than nixpkgs is built around the
 > assumption that there is one set of packages you can build almost
 > everything with it.

I don't think that is true. Nixpkgs has built-in support for overrides
that allow you to customize the package set to your liking.


 > Eelco Dolstra told me on irc once that there is no dependency solver
 > by design. And I have to agree that it works pretty good in almost
 > all cases.

What you mean by "almost all cases"? What would be a use case that
doesn't work?


 >> [User should be] able to build and install Haskell packages using
 >> the standard nix-env interface, which seems to be very desirable.
 >
 > You can! Have a look at the [default.nix] in the overlay file and look for
 > xmonad,darcs,hackNix itself, .. The function exeByName which is used
 > will calculate different dependencies for each (which may look like a
 > waste of resources - but it was the only way I came up supporting
 > everything possible.)

Well, what I meant that I would like users to be able to install Haskell
packages without having to worry about hack-nix. That tool sounds like
it's intended for package set maintainers, but not for end users. So the
existence of hack-nix should not be exposed to end users. They shouldn't
have configure it or run it or even know that it exists.

Maybe I misunderstand the situation, but I have the impression that I
would have to install hack-nix before I could install any of the
packages in that pool.


 >> [A separate tool to install Haskell packages] already exists, namely
 >> cabal-install.
 >
 > Yes - but it does not know about Nix. And if dependencies are wrong on
 > hackage you're doomed.

I believe you are overstating the magnitude of that problem a bit. Cabal
files on Hackage are usually quite accurate. And if they aren't, then
you can always ask upstream to fix them. And if that doesn't work, then
you can upload a fixed version yourself. If a Cabal file has bugs, then
you are by no means "doomed"; there are plenty of ways to remedy the
situation.


 >> So when a package cannot be built the very latest version of its
 >> dependencies, I'll have to choose the version that ought be used
 >> manually. Is that right?
 >
 > Correct. You add it to ~/.hack-nix/config and run hack-nix to regenerate
 > the package pool. Then you should be fine (hopefully).

Well, if that is true, then hack-nix is not particularly helpful.
Figuring out those dependencies is by far the most difficult part of the
whole task. What would anyone need a dependency resolver for if it
doesn't resolve those dependencies? Doing that manually is what we're
doing in haskell-packages.nix already. We don't need a tool that allows
us to continue to do that manually -- we need a tool that does that for
us.

Take care,
Peter