[Nix-dev] Upcoming PyPi URL Scheme Change

[Graham Christensen]

Profpatsch <mail at profpatsch.de> writes:

> Maybe it’s time to automate what we can? Similar to Hackage?

I'm not too familiar with what is done with Hackage, but I think what
we have for Python isn't a good approach. I have a meeting with dstufft
to try and come up with a better solution. It might be helpful to
understand what we have with Hackage to do a better job.

I know Domen has specific expertise here, and probably has some really
valuable feedback.

My problem with the current system is we have many arbitrarily versioned
python packages referencing each other in a haphazardly developed graph
of dependencies. Upgrading one package has a nasty cascading effect of
needing to upgrade each of the other ones depending on it. This has
stymied at least one of my attempts at contributing fixes.

For development dependencies, more fully automating it is probably the
best approach.

I think for applications, it would be more beneficial to take an
approach similar to Bundix, Npm2Nix, etc. The community's current tools
for Python (pypi2nix, pip2nix, others?) seem to work on some types of
packages, and sometimes not on others.

I have a prototype of an alternative method which leans harder on pip
to do the work than nix. Instead of building each python dependency in
its own derivation:

 1. it creates a fake Pypi mirror of all the dependent packages
 2. installs all of the packages at once with `pip install -r
 requirements.txt`

This avoids issues like circular dependencies and other complexities of
how python packaging works, but is a much heavier-weight installation
mechanism. I'll have to test more before saying it is good or not.

Best,
Graham