[Nix-dev] Dependency Semantics

[Bryce L Nordgren]

On Thu, Jul 12, 2012 at 12:40 PM, Mathijs Kwik <mathijs at bluescreen303.nl>wrote:

> That's not what I was talking about, the python expression didn't change
> in my example. Just a (made up) underlying graphics library.
> That triggers a python rebuild, which can't use the new gfx version, so
> it builds without bindings for it.
> Then a package on top of that gets triggered to rebuild, and its
> upstream configure/build script detects the missing bindings and fails.
>
> How often do you expect to run into this? I can see explicitly specifying
something like --with-ImageThingy=/nix/store/{hash}-lib/blah.so on python's
command line (or more prominently, PHP's. PHP has an awful lot of optional
components). But I'd expect the python build to fail if you explicitly
specify to use something and it can't find what it needs from that
something.

But for the sake of agreement, let's just say you don't use weaker
dependencies for environments where it's likely to cause problems (TBD). If
the environment is deemed unlikely to cause these issues (e.g., optional
components are explicitly specified, not autodetected; build fails when
specified component doesn't provide expected interface) give it enough
rope to hang itself. Each of the environments will have to be evaluated on
its own merits.


> I agree on java. I think java builds a solid abstraction which really
> shields the host system off from packages using it.
>
+1


> But I don't believe the same can be said about any other environment.
>
Well, I suspect that each environment (even Java) is going to land
somewhere between "perfectly isolated" and "fully integrated to the host".
It will boil down to whether the savings in time, CPU cycles, bandwidth and
disk space is worth the extra human effort to root out and handle cases
where isolation was violated. Even if the default for an environment is
"rebuild on version bump", any individual package could be given the
stronger dependency (always rebuild). The CPU cycles are then focused on
code which has proven to break isolation. I am not preaching a one size
fits all solution here.


> So, this would mean adding an exception for glibc (force trigger
> rebuilds for weak deps when glibc is involved), or having maintainers
> specify that their package might use ipv6, and flag glibc versions with
> "ipv6 capable" so to trigger just those rebuilds. Sounds like a lot of
> work to me, and probably stuff will slip through.
>

Instead of tracking down every exception, what about being able to manually
force rebuilds for weak deps on a specific environment? Again, each
environment needs to be evaluated on its own (de)merits, but being able to
instigate a full rebuild of all python code at a whim may give the
maintainers confidence to "try it and see how much of a problem it
is". Smell a problem? Hit the panic button.

For that matter, whether or not all the dependent code needs to be rebuilt
when the environment is could be a property of the environment. Ruby could
be "rebuild always". Java could be "rebuild on version bump". Python can be
whatever it turns out that python needs to be. It can be changed as time
goes on, experience is gained, and the maintainers get a feel for how much
caution is actually necessary.

There are many reasonable impediments which could have led Eelco never to
try installing each package into a separate content-addressable directory
with a hash in the name. But he did try it, and it seems to work so far.
Still, there's no guarantees that tomorrow's library format/linkers will be
compatible with such a scheme, as it's not very common. Even without such a
guarantee, it seems like a worthwhile strategy to pursue.

Bryce
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.science.uu.nl/pipermail/nix-dev/attachments/20120712/046cde20/attachment.html