[Nix-dev] Re: [Nix-commits] SVN commit: nix - 22282 - MarcWeber - nixpkgs/branches/stdenv-updates/pkgs/stdenv/generic

[Peter Simons]

Hi Ludovic,

 >> [One problem is] that certain builds take ages (OpenOffice, boost, gcc, qt,
 >> firefox etc.) when performed on a single core.
 >
 > That builds take ages doesn’t matter as long as you’re getting pre-built
 > binaries. For those who don’t use pre-built binaries, the problem is the
 > same as for Hydra (above).

unfortunately, it is not that simple. Inter-build parallelism scales fairly
well if there's a great variety of mostly independent build jobs in the queue.
That may be the case for Hydra, but it's not the case for end users, like me,
who run

  nix-env --always -u \*

every so often. What happens in my experience is that some minor package got
updated, and this update triggers a rebuild of, say qt, and that triggers a
rebuild of two or three other packages that depend on qt. In that case, qt is a
bottle neck, because until qt is built, no other build can be performed. That
qt build, however, takes hours. The same applies to GHC. Once GHC has been
re-built, there's lots and lots of Haskell packages to compile in parallel, but
re-building GHC takes hours.

I'm pretty sure that the same problem occurs on Hydra, too. It's may be less
significant, because Hydra's profile a far more extensive that mine, but the
basic problem exists whenever a big package is a precondition for others. Until
that big, fat package is built, nothing else will be.


 >> [Impurities] are undesirable in general. In this particular case, however,
 >> I don't see a problem. Passing the number of available cores to Make is a
 >> fairly insignificant impurity that allows for fairly significant benefits,
 >> i.e. significantly reduced build times.
 >
 > It’s impossible to know with certainty whether a package is
 > parallel-build-safe, i.e., deterministic regardless of the level of
 > parallelism. Thus, its build output can depend on build task scheduling, in
 > which case passing the number of cores is indeed an impurity.

Oh, please, come on! "It's impossible to know with certainty whether a package
is parallel-build-safe"? What are you talking about? Are you referring to some
profound existentialist Gödel-Escher-Bach diagonal argument that shows there's
nothing we can know for sure?

Parallel builds are being routinely performed for, I dunno, a decade, maybe?
You cannot possible have missed that fact. Where have you been? ;-)

Take care,
Peter