[Nix-dev] Dependency Semantics

[Evgeny Egorochkin]

On Вторник 10 июля 2012 16:28:28 Bryce L Nordgren wrote:
> Hi list,
> 
> I want to suggest something to see what you think.
> 
> Nix currently supports a single meaning for its dependency relation, which
> essentially can be summed up as "Depends on exact build of". As in, GDAL
> <depends on exact build of> GEOS(hash here). This is an enormously
> successful model, but it does force you to completely omit real
> dependencies (and to trick Nix into not detecting them). For instance, to
> avoid recompiling all Java source to bytecode simply because some change
> triggers the rebuilding of the java compiler (Note we're not even talking
> about upgrading Java here: we're saying, i.e., a sound library was upgraded
> so the same JDK was rebuilt.), it has been suggested to NOT depend on the
> jdk and to refer to the java interpreter only by an environment variable so
> that Nix won't pick up the dependency (e.g., $JAVA_HOME/bin/java).
> Likewise, "hash rewriting" has also been suggested as a manual workaround
> for those situations where the "Depends on exact build of" semantics are
> inappropriate. Certainly "Depends on exact build of" will produce a
> workable result given enough time and CPU cycles, but there are situations
> where it is extremely wasteful. Consider the following additional
> semantics:
> 
> * "requires environment" : Requires the presense of the interpreter
> software/VM (e.g. bash/python/perl/java/ruby) but does not force the
> dependent package to be rebuilt (and stored again) if the interpreter/VM is
> replaced. The target of the dependency does not represent a specific build,
> but an environment description.
> * "implements environment" : Declares that the package is designed to
> provide a specific environment. Two implementations of Ruby 1.8.7 could be
> the reference implementation of Ruby or Ruby-Enterprise. The target of the
> dependency represents an environment description.
> * "depends on executables from" : Refers to a Nix source expression instead
> of a specific build. The dependency is on a command line utility which has
> already been linked. The dependant package is not rebuilt when the target
> is. Also: any build instantiated from the target expression should be
> considered interchangable. For example: a dependency on the "dot"
> executable from GraphVis should not cause Doxygen to be rebuilt if a
> rebuild of GraphVis is triggered; conversely, any build of GraphVis
> (version X) present on the system should satisfy the requirement.
> 
> I understand that there is some resistance to these suggestions because
> they appear too similar to the semantics present in other package
> management systems (which completely lack "depends on exact build of").
> This is not an effort to make Nix more "familiar" or to remove "depends on
> exact build of". It's intended to eliminate wasteful and needless
> rebuilding (while reducing the number of missed/masked dependencies) by
> allowing the packager to correctly express relationships between packages
> which are not "depends on exact build of". Additionally, this would allow
> "nix-env" "nix-build" and Hydra to correctly process these directives.
> 
> Thoughts?

As a former Gentoo user, my practice tells me that hidden breakage due to 
minor incompatibilities due to different package versions or different build 
options happen often enough. Also packages tend to have workarounds for bugs 
of specific versions of their deps. All this ends up in really nasty, nearly 
impossible to catch bugs, which aren't reproducible.

When you talk about waste, you have to understand that really it's a trade-off 
between wasting cpu cycles and developer time. It's much easier to double 
hydra cpu power than double developer effort for nix* right now.

"recompile everything" approach does get in the way when you need to urgently 
apply a trivial patch(meaning an actual developer can check that there's goin 
to be no breakage) to a core(stdenv) dependency, eg to fix a vulnerability, but 
this time it's not so much about wasting cycles as it is about time it takes 
to fix the issue. Also "recompile everything" gets in the way when you debug 
core packages because you need to recompile often, which again is mostly a 
problem of time, not cpu cycles

...and the best solution seems to be allowing temporary manual hash overrides 
to get a quick and dirty temporary fix and do a full recompile later to restore 
purity

-- 
Evgeny