[Nix-dev] distutils2nix/python4nix

[Florian Friesdorf]

On Fri, 23 Sep 2011 23:12:14 +0200, Marc Weber <marco-oweber at gmx.de> wrote:
> Excerpts from Peter Simons's message of Fri Sep 23 21:21:16 +0200 2011:
> > One issue that you might want to consider is that the "pythonPackages"
> > set needs to support several different versions of Python, i.e. the
> > python interpreter needs to be an argument to that expression.
> > Currently, we cannot build our python packages with Python 3.x, for
> > example, which is really a shame.
> 
> And that's the issue: The python packages don't say "I'm tested with
> python-2.7". There are some wired automatic conversion tools.
> 
> Before we start writing cabal2nix, perl2nix, X2nix: How should the
> result look like?
> There are 20.000 ruby packages!
> And I personally don't like seeing more and more individual "let me
> update just this package and then that cause I need it right now and I
> don't know what this might break.." There is no choice unless we start
> supporting constraints.
> 
> In theory I've packaged all of those ruby packages. But do we really
> want to checkout such masses of .nix files to get only 10 packages?
> 
> Or would it be much better to query PyPi package database, cpan, ..
> lazily (which would require changing nix?)

I would first create all thix X2nix and Y4nix tools, once we have them
we can think about whether and how to use them on-the-fly. Solving
constraints would happen in those tools (maybe via a hook) and could be
implemented as a generic, language independent solver (in prolog?, as
Peter suggested). Or it might be feasible to use a language's own
package manager to handle the resolving. But at the time a nix
expression is being installed, that resolving has happened already and
eg. python setup.py install is not allowed to fetch any dependencies.

In any case nix knows how to handle nix expressions, so let's feed it
nix expressions and use the magic to create them.
 
> For python there is another issue: You have to run the setup.py file in
> most cases to learn about package dependencies.

You as a person generating nix expressions currently have to, changes
towards a declarative format are ahead.

> I already have written kind python 2 nix and its good enough to install
> some packages on the fly but still needs much more love.
> 
> If you want to learn about this way (which perfectly work for all ruby
> versions) I'll point you to the code and explain how to use it.

You were showing me all these tools, but I failed in seeing the
currently relevant pieces. As said before, I am not searching an
integrated new solution, but pieces to extend/adjust nix.
 
> I personally believe that the final solution should take version
> constraints into account so that you can fail early.

+1 at the time of nix expression generation

> How should this all look in the future (there are vim, emacs, perl,
> ruby, python, eclipse, zsh, haxe, java, scala, ... ) packages to package.
> And eg for scala, java it almost impossible to keep up to date using the
> current manual way.

I'm not sure what you mean with "current manual way". As far as I
understand Peter uses cabal2nix and haskell4nix to semi-automatically
maintain the haskell expressions. cabal2nix generates a nix expression
and haskell4nix keeps nix haskell tree up-to-date. You run haskell4nix,
check whether things look good, install something, run some tests, and
check it in.

Maybe Peter can explain in more detail what manual work he currently
needs to do and where he sees potential for further automisation.

I prefer a "manually" maintained tree over on-the-fly. Having explicit
nix expressions allows for somebody/something to test these and building
a database of known good sets of packages. If we generate the
expressions on-the-fly, we would need a database of checksums in order
to know whether the on-the-fly generated expressions are what we want
them to be (as brought up by Michael).

> It may be possible to tune the tools (sbt, maven,
> ..) to use the nix store as cache at least..
> And I'm pretty sure that I've forgotten tons of package solutions (eg
> ocaml)
> 
> Thoughts?

Are your tools providing the functionality of distutils2nix/python4nix
described so far?

-- 
Florian Friesdorf <flo at chaoflow.net>
  GPG FPR: 7A13 5EEE 1421 9FC2 108D  BAAF 38F8 99A3 0C45 F083
Jabber/XMPP: flo at chaoflow.net
IRC: chaoflow on freenode,ircnet,blafasel,OFTC
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 835 bytes
Desc: not available
Url : http://lists.science.uu.nl/pipermail/nix-dev/attachments/20110924/f18e750c/attachment-0001.bin