[Nix-dev] Re: Tahoe-LAFS in NixOS

[Ludovic Courtès]

Hi Zooko,

Zooko Wilcox-O'Hearn <zooko at zooko.com> writes:

> Here is the instructions that I recently gave someone who wanted to
> set up an OpenBSD buildslave:
>
> http://allmydata.org/pipermail/tahoe-dev/2009-December/003279.html

At first sight it seems like a bit of overhead I’d like to avoid these
days.

>> In addition, most of Tahoe’s direct dependencies are defined in [3].
>>
>> [3] https://svn.nixos.org/repos/nix/nixpkgs/trunk/pkgs/top-level/
>> python-packages.nix
>
> How was this file generated?

Heh, it wasn’t generated.

> Since Tahoe-LAFS's dependencies are already declared in a
> machine-readable format [1], I think it would be cool if there were a
> generic tool to translate those dependencies from that Python format
> into NixOS format.  I contributed some patches to such a tool to
> translate from Python format to Debian format [2].

Indeed, that seems to be an interesting approach.  However, is it 100%
automatic or 90%ish?  If the latter (which I suspect from my experience
packaging several Python libs), I wonder whether it’s a big win in terms
of the time spent packaging a given piece of software.  What do you
think?

>> The “doCheck = false” in [0] means that tests aren’t run.  I just
>> checked what prompted me to disable them; here’s the end of the log:
> ...
>> The following error occurred while trying to add or remove files in
>> the
>> installation directory:
>>
>>     [Errno 2] No such file or directory: 'support/lib/python2.5/
>> site-packages/test-easy-install-15898.pth'
>>
>> The installation directory you specified (via --install-dir, --
>> prefix, or
>> the distutils default setting) was:
>>
>>     support/lib/python2.5/site-packages
>>
>> This directory does not currently exist.  Please create it and try
>> again, or
>> choose a different installation directory (using the -d or --
>> install-dir
>> option).
> ...
>> It seems that ‘setuptools’ is trying to write something in a place
>> it isn’t allowed to.  Note that “python setup.py test” here is run
>> before Tahoe-LAFS is installed.  Is that a problem?  Any other idea?
>
> It is okay to run "python setup.py test" before anything else.  (It is
> configured to execute the build step before it does the test step,  in
> the Makefile style -- see line 48 of setup.cfg [3]).
>
> I don't think that error message above says that the build was
> attempting to write to a disallowed location.  The location it is
> trying to write to is a relative path below the base dir (i.e.
> without '..').  That is allowed by the NixOS build system right?

Writing in $top_srcdir/.. is allowed, yes (but it’s bad style).

> I think what you see there is an example of a bug in our buildsystem
> which manifests on some platforms.  I am going to be trying to fix
> that bug in our buildsystem and looking at the buildbot results [4]
> to see if my attempted fixes cause it to build and run tests properly
> on more platforms without causing a regression on any platforms.  The
> sooner NixOS is included in that list of platforms the sooner I will
> be able to make this part work correctly on NixOS.

OK, then I guess I’ll just stay tuned.  :-)

>> (Ironically, it may be that build failure is related to the fact
>> that NixOS builders follow POLA whereas ‘setuptools’ expects some
>> form of ambient authority.)
>
> This part is super cool!  Do the NixOS docs explicitly cite POLA?

No, because it approaches things differently, I guess.  Still, I think
it’d make sense to explicitly cite it.

> What paper or doc should I read for the top-level justification for
> NixOS's design?  I would like to show this to the POLA theorists.  :-)

The introduction of
http://www.st.ewi.tudelft.nl/~dolstra/pubs/nixos-icfp2008-final.pdf (and
probably other papers at http://nixos.org/docs/papers.html) is probably
enough to get a feel of how POLA is applied to the build system.

Section 5 of the Nix manual is a good introduction too:
http://hydra.nixos.org/job/nix/trunk/tarball/latest/download-by-type/doc/manual .

Thanks,
Ludo’.