[Nix-dev] parallel make (was: Again: Why don't these people havecommit access)

[Michael Raskin]

>Hi Michael,
>
> > To find out if a makefile is parallel-safe, we need to try it with
> > parallel building enabled. This can only be done package-per-package,
> > I cannot quickly build everything with parallel builds on and then
> > blacklist what fails.
>
>the rationale behind that choice is that you cannot easily tell whether a
>package compiles with parallel make jobs enabled or not. Under-specified
>make dependencies lead to race conditions, and these may or may not screw
>up your build. Whether an unsafe build actually fails depends on any
>number of factors that influence the order in which make schedules the
>jobs. You can -- in general -- not enable parallel building, run the build
>once, and conclude that it's safe! The number of false positives is going
>to be large.

So the packages where you can safely turn on parallel building are the
packages where upstream says all is fine _and_ many people tried and got
no problem. As there is no easy way to try parallel builds for 
everything, we end up experimenting on Hydra, and finding problems a few
weeks later, just as you decribed.

>Because of this situation, many NixOS contributors preferred the opt-in
>solution that we have today. The aim of that the double opt-in is *not* to
>maximize the number of builds that run with parallelism enabled, it is to
>minimize the number of non-deterministic failures that someone has to
>debug and worry about.

Different people have different priorities. I would sometimes try to
build my system with maximum parallel building enabled just to record 
what breaks this time if I could do it without abandoning the stdenv
I already have. Multiple people said multiple times that there could be
different levels of opt-in (opt-in for a whitelist or opt-in for
everything but a blacklist). But solutions offering multiple opt-in
levels (I still think these are the only ones that allow us to have
reasonable data before adding to a whitelist) were declared too risky.

Actually, I wonder if there is any package where parallel build succeeds
with a significantly wrong result; as for printing a warning on build 
failure with full opt-in (so user is reminded that the configuration is
a risky one; and the failure log to submit is clearly tainted) --- 
I will implement it if it is the real objection here...