[Nix-dev] Re: overview about hack-nix and how it works

[Peter Simons]

Hi Marc,

 >> Now, I'm sure a package manager that works like this can be very
 >> useful, but I don't see how this has anything to do with Haskell,
 >> nor do I believe that this kind of thing should be written in the
 >> Nix language.
 >
 > Its the cabal system making the difference.

you suggest that Cabal makes a huge difference between Haskell and other
packages, but I don't believe that is true. Most languages have some
custom format for build meta data. Perl has it, Python has it, Ruby has
it, LaTeX has it, and so on. One could even argue that the package
databases published by Debian, Gentoo, or ArchLinux constitute build
meta data that can be used to auto-generate Nix expressions. As far as I
can tell, Haskell is not at all unique in that regard.


 > You're right, it should not have been written in Nix. What do you
 > suggest as alternative?

I don't know. Personally, I would have used Prolog because constraint
resolving is pretty much for free in that language, but that is entirely
a matter of taste.


 > Or what about making Nix call into Haskell via C and reuse cabal's
 > solver?

I would just call "cabal install --dry-run" and parse the output it
generates:

  $ cabal install --dry-run leksah
  Resolving dependencies...
  In order, the following would be installed (use -v for more details):
  binary-shared-0.8.1
  cairo-0.12.0
  glib-0.12.0
  gio-0.12.0
  hslogger-1.1.4
  pango-0.12.0
  gtk-0.12.0
  gtksourceview2-0.12.2
  ltk-0.10.0.4
  process-leksah-1.0.1.4
  leksah-server-0.10.0.4
  regex-tdfa-1.1.8
  strict-0.3.2
  leksah-0.10.0.4


 >> Nix is great for specifying build instructions, but it's not a
 >> general purpose programming language.
 >
 > I totally agree. And I told you that its a night mare previously.
 > This doesn't change the fact that the work has been done and that it
 > works. If you have time to rewrite it.. go on and do it.

Actually, that effort is well underway. I was just worried because you
seemed to be angry at me for doing that.


 >> There is a finite number of ways in which the package set can be
 >> permuted, so it is clearly possible to configure all those
 >> combinations in static Nix files.
 >
 > Yes, you can also write a nix interpreter in brainfuck!

I am sorry, but that is not a helpful response. Do you want to have a
serious conversation or not? I spent a lot of time thinking about this
thread because you asked me to. If it turns out that you really don't
care and that you are just wasting my time, then I'll be really pissed.


 >> Builds in Nix are *functions*, so they are really good at being
 >> re-used in different configurations. Doing these things "on the fly"
 >> doesn't gain anything,
 >
 > You're wrong, it does.

Do you have any specific examples to illustrate that assertion?


 > I know that I don't break anything by changing configurations. I know
 > that I don't forget to remove dependency inputs when upgrading
 > packages, ...

I don't understand how hack-nix could possibly guarantee these traits in
a way that's superior to, or even different from the mechanisms in Nix.
You said that the package set is determined by a configuration file
that's written manually?


 >> [A dynamically generated configuration] just causes the tool to be
 >> slow and inefficient, because it makes the same decisions over and
 >> over again, every time it's called.
 >
 > Unless you want to start testing different configurations (flag
 > settings) and different ghc versions.

GHC is a parameter to haskell-packages.nix. All the expressions in there
can be instantiated with any other version of GHC. I don't see how
hack-nix improves the situation on that.


 >> I trust you mean haskell-packages.nix? Yes, that function works really
 >> well. It supports a significant number of packages in several
 >> architectures and several different versions of GHC.
 >
 > Yet it failed on scion.

No, that is not true. Scion doesn't build because the package is
hopelessly outdated upstream. That doesn't have anything to do with the
Nix language. On the contrary, it's really quite amazing that Nix still
*can* build Scion, because we still do have ghc 6.12.3 available.

Can hack-nix build Scion?


 > Same for leksah, .. so does it really perform that well?

Again, this build problem is totally unrelated to the issue we are
discussing. The build for leksah used to fail because our version of
gtksourceview2 was too old. That is not even a Haskell package, so I
reckon hack-nix had the exact same problem. Yet, you imply that hack-nix
would have been able to build leksah two days ago. I doubt, though, that
this is the case.


 >> Why would anyone want to "escape" that? What would you like to do
 >> that haskell-packages.nix doesn't allow you to?
 >
 > Because sometimes you want dev packages. And their dependencies may
 > be totally different (eg scion no longer depending on syb, ..).

Could you please be a little more specific? What would you like to do
that haskell-packages.nix doesn't allow you to?


 >> Nixpkgs has built-in support for overrides that allow you to
 >> customize the package set to your liking.
 >
 > I'm not talking about svnSupport.

Neither am I.

 > I'm talking about: "rebuild everything and replace gtk libs-2.6 with
 > gtk libs-2.8 in the whole dependency chain".

As you know, that is quite possible.


 >>> Eco Dolstra told me on irc once that there is no dependency solver
 >>> by design. And I have to agree that it works pretty good in almost
 >>> all cases.
 >>
 >> What you mean by "almost all cases"? What would be a use case that
 >> doesn't work?
 >
 > I didn't say "doesn't work" I said "being pretty good at". Try to
 > package all version of all ghc and packages on hackage manually.

Marc, please refrain from giving me orders such as "go try that". That
is an arrogant and condescending thing to do. You imply that I don't
know what I am talking about because I lack practical experience. I feel
insulted, because I already explained in some detail that I *have been*
packaging Hackage for ArchLinux for the last 12 months. I have already
done what you suggest I should try, and if you've been reading my
messages, then you should know that.

So, would you like to answer my question about a specific use case, or
are you just being evasive?


 >> Well, what I meant that I would like users to be able to install
 >> Haskell packages without having to worry about hack-nix. That tool
 >> sounds like it's intended for package set maintainers, but not for
 >> end users.
 >
 > That's why I wrote the solver in Nix. Its run automatically in those
 > cases printing some lines so that you know what its doing because
 > nix-* evaluations do take some additional seconds.

Do I understand it correctly that I can install a Haskell package via
hack-nix without having to worry about hack-nix at all? That would be
great, indeed. I'd be eager to try that out. Could you please post an
example of a command line I could execute to install, say, pandoc?


 >> The existence of hack-nix should not be exposed to end users.
 >
 > There are two types of users: [...]

Marc, please don't be silly. There are not just two types of users. That
notion is ridiculous. There are hundreds of different types of users,
because every user is a unique human being with unique intentions and
goals.


 >>>> So when a package cannot be built the very latest version of its
 >>>> dependencies, I'll have to choose the version that ought be used
 >>>> manually. Is that right?
 >>>
 >>> Correct. You add it to ~/.hack-nix/config and run hack-nix to regenerate
 >>> the package pool. Then you should be fine (hopefully).
 >>
 >> Well, if that is true, then hack-nix is not particularly helpful.
 >> Figuring out those dependencies is by far the most difficult part of
 >> the whole task. What would anyone need a dependency resolver for if
 >> it doesn't resolve those dependencies?
 >
 > Automate what can be automated: fetch hashes, write .nix code, ..

Huh? You say that everything that can be automated should be, but you
say that in response to my complaint that dependency resolution has
*not* been automated. What does that mean?

Now, if hack-nix would be able to resolve build input dependencies
automatically, then it would be very useful indeed.

Take care,
Peter