[Nix-dev] Nix and Plash

[Mark Seaborn]

ludovic.courtes at inria.fr (Ludovic Courtès) wrote:

> Mark Seaborn <mrs at mythic-beasts.com> writes:

> > Obviously the Debian packaging system has problems.  However, it does
> > provide one way to specify a version of "libpoppler0" that "evince"
> > works with: the "Packages" file, which lists a set of .deb packages
> > and includes their hashes.  If I create a Debian package repository
> > based on that Packages file, debootstrap from it and "apt-get install
> > evince" inside the resulting chroot, I will get a reproducible set of
> > packages installed.  This is awkward, yes, but it is reproducible.
> > The Plash package tools make it easier to set up an environment just
> > to run Evince without having to use chroots.
> >
> > Library versions L1 and L2 don't have the same name from the point of
> > view of a Packages file, which names them by hash.  The fact that L1
> > and L2 can't be installed at the same time in the same chroot is a
> > limitation which is unrelated to reproducibility.
> 
> He he, sure you can do anything with `.debs', provided you are careful
> enough.  The thing is, a Debian `control' file names dependencies using
> human-readable and human-assigned names, such as `libpoppler0'---it does
> *not* name dependencies using hashes.  Thus, a `control' file alone is
> ambiguous because it depends on a naming context provided by the
> `Packages' file.

I view the dependendies in the "Depends" field in the control file as
being more like interface dependencies than implementation
dependencies.


> > As I see it, there are two issues for reproducibility:
> >  * Recording sets of binary packages so that a deployment setup can be
> >    reproduced.  With Debian you can do this with a Packages file.
> 
> That's the build-time issue that I described above.
> 
> >  * Recording how a binary package was built so that the binary package
> >    can be reproduced.  Debian does not provide a way to do this, but
> >    it could do this by referring back to an immutable Packages file
> >    listing the packages that were installed when the package was
> >    built.
> 
> That's the run-time dual of the first aspect.

(I'd have thought these would be the other way around: the first is
run-time, the second is build-time.)

> Under Debian, `evince' is linked under `libpoppler.so.0'; at
> run-time, whatever library under `/usr/lib' whose name matches is
> picked up and used as `libpoppler0', regardless of whether this
> shared object was the one used at build-time.  Here Debian relies,
> again, on human-maintained library naming: upstream developers are
> expected to change SO names when there's ABI breakage.
>
> Under NixOS, `evince' refers unambiguously to the very `libpoppler.so.0'
> that was used at build-time (through its `RPATH').  The corollary is
> that NixOS does not exploit ABI-compatibility at all: when `libpoppler'
> is upgraded, `evince' keeps using the old `libpoppler', or you have to
> rebuild `evince' to use the new one---but that is a deliberate choice.

I would prefer to be able to choose between:
 a) running the existing Evince with the old libpoppler.so.0
    (while other executables can use the new libpoppler.so.0)
 b) running the existing Evince with the new libpoppler.so.0
    (while other executables can use the old libpoppler.so.0)
 c) building and running a new Evince with the new libpoppler.so.0

I would not expect that building and linking against different
versions of a library that are accidentally ABI-incompatible would be
a common source of problems.  It seems more likely that bugs are
removed or introduced in new versions of a library without the ABI
changing, and that this would be a more common source of problems.

From my point of view, having worked on a project (Plash) which
involves changing libraries underneath applications, it seems like a
bad trade-off to not support (b) and to require (c) instead.


> > Zero-Install handles libraries this way by setting LD_LIBRARY_PATH.
> 
> Nix uses the `RPATH' for librairies.

BTW, do you know what causes the RPATH field to be added to
executables and libraries?  It looks like "ld" only adds it when an
explicit "-rpath" argument is given.  Is it libtool that is adding the
"-rpath" argument and how does it decide whether to add it?  Based on
<http://wiki.debian.org/RpathIssue>, it looks like libtool adds it by
default but it is switched off with --disable-rpath.  How does Nix
deal with packages that don't use libtool?

> > If Nix did the same, substituting hashes inside binary files might
> > become unnecessarly.
> 
> No: hash substitution has nothing to do with the fact that Nix uses
> `RPATH'.  As I mentioned, hash substitution is used to rewrite
> self-references in files resulting from a build with the actual hash of
> the output modulo self-references.

From the "Secure Sharing" paper, it seems Nix has a concept of
packages being equivalent if they were built from the same input even
if the outputs were not exactly identical because the build is not
completely deterministic.  As I understand it, hash substitution is
used to substitute references to packages that are build-equivalent
but not identical, so it's not just used to deal with self-reference.

Regards,
Mark