[Nix-dev] Re: Python 3

[Marc Weber]

On Wed, Feb 18, 2009 at 08:23:45PM +0100, Peter Simons wrote:
> Hi Marc,
> 
>  >> We have two different expressions for Python 2.5 in Nix. That state
>  >> is not a particularly satisfactory.
>  >
>  > My view on this: I'm using composableDerivation. [...]
> 
> well, for me personally that is an implementation detail that I don't care
> about much; I care for having a working Python installation, and I'd like to
> be able to install Python 2.5 and 3.0.1 simultaneously. I mean, Nix was
> designed to support this particular use-case, so I'd like to use Nix for what
> it was designed to do!
> 
> Can your new-and-improved python expression support that? Can I install two
> different Python versions with the corresponding set of libraries?
How should it look like?
I mean you can install to different profiles and you're fine. You can
also change the final python wrapper name to
python-2.5 and python-3.0 and you're fine as well. Adding just a prefix
should be easy

>  > If you fix the default expressions thats fine with me.
> What do you mean by "fix"? What is broken about the current default
> expression?
That time I wanted to make pitivi compile. I do no longer remember where
I had most trouble. One major problem is that  a lot of python libraries
check for its dependencies. Only using PYTHONPATH isn't nough in most
cases, cause the import directories do cantain .pth files or such
listing more directories to be added to the import path. Those are
ignored by using PYTHONPATH only. So some of those setup.py install
scripts complained about missing libs. There is a similar issue about
py-gtk. You use

        import pygtk; pygtk.require('2.0')
        import gtk
        import gconf

The beast is pygtk.require('2.0') cause it causes depending libraries to
import different subdirs at runtime. But they all choose the default
python install location which doesn't exist in nix. That's why I finally
gave up and just did put all (pygobject, pygtk, gnome-python,
gnome-desktop, gnome-python-extras) into the same location.
Of course the problem is gone when installing everything into a final
~/.nix-profile/* location. However its not gone while compiling
libraries. Manually skimming the lib path to add all the required dirs
to PYTHONPATH is no way cause the mantainance effort would be to big.
In the pygtk case patching the library to add all <any nix python lib>/2.0/
to path would add a lot of directories to the sys.path which would take
time cycling through. the pygtk.require('2.0') doesn't add one location
but 2 or more (I do no longer recall the amount). So if you have 100
packages one day You would have 400 search locations and a sys.path
which can no longer be read ..

There is another choice:

myPythonLib = mkDerivation {
  buildInputs = [ (buildEnv { path = [ .... ] }) ];
}

This would make the lib find its dependencies at build time (cause they
can all be found within a temporary profile). I don't
know how many libraries do compile in the path of their deps somehow. So
this could cause some other problems..

I hope I was able to illustrate some of the possible problems. Also read my
comment in pkgs/development/interpreters/python-new/2.5/default.nix.

NIX_PYTHON_SITES (added by a small patch) basically does the same as
PYTHONUSERBASE but it allows you to pass multiple directories.
Maybe that all has changed in python-3 anyway (?) I don't know. I
haven't looked into it.

Sincerly
Marc Weber