[Nix-dev] python application vs python environment

[Florian Friesdorf]

Hi,

sorry for failing to express this in less words:

As far as I understand: a profile in nix is a derivation. A derivation
is build by linking in other derivations. By that installing a python
interpreter and additional python modules into a profile results in
these being linked from their respective derivations.

A python application (eg. offlineimap) installed in a profile is
isolated from the python environment that might also be installed there,
but uses the built time interpreter directly and PYTHONPATH to find the
needed modules in their derivations.

On the other hand, if I install python itself and additional modules
into a python profile, I would expect this profile to behave like a
native python installation or virtualenv environment with additional
modules.

With the patch I recently posted (Subject: python packages and
dependencies) python packages will not clutter their site-packages
directory with easy_install.pth and site.py files, but instead for every
distribution a .pth file is created that loads it.

Given that the python installed into the profile links over
pythonX.Y/site.py (buildEnv succeeds, nix-env fails so far) and that
python itself is not a symlink (see below), python then finds all
packages.

If python is not a symlink the sys.prefix and therfore search mechanisms
for site-packages works fine. If it is a symlink (happens if there is
more than one derivation supplying content for bin/) python will follow
it and find itself in its derivation, ending up with sys.prefix pointing
to the derivation instead of the profile.

By wrapping the python interpreter (see pythonhomeWrapper) and setting
PYTHONHOME to the profile, this can be prevented and site-packages are
correctly available. Doing the same for virtualenv (not committed yet)
enables virtualenv which otherwise complains about mismatching prefixes.

This would need to be done for all python shell commands that should be
connected with the environment they are installed in (like virtualenv,
ipdb, ipython).

A profile corresponds to what others call /usr and a python installed
there I would expect to feel at home there: -> sys.prefix set to the
profile.

I think we should have a python executable that feels at home in
whatever profile it is used.

Patching python itself not to resolve symlinks for finding its prefix, I
think is not an option, because that is what python does and what users
might expect.

Instead we need a python that stops resolving symlinks once it reached
its home profile (not continuing into its derivation).

A solution would be, not to link python and pythonX.Y from the
derivation, but instead to create two wrappers that set the PYTHONHOME
and call then the python in the derivation.

Do we break something by doing that?
How can I influence how a derivation is merged into another derivation?

regards
-- 
Florian Friesdorf <flo at chaoflow.net>
  GPG FPR: 7A13 5EEE 1421 9FC2 108D  BAAF 38F8 99A3 0C45 F083
Jabber/XMPP: flo at chaoflow.net
IRC: chaoflow on freenode,ircnet,blafasel,OFTC
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 835 bytes
Desc: not available
Url : http://lists.science.uu.nl/pipermail/nix-dev/attachments/20110923/4fcad9e4/attachment.bin