[Nix-dev] Re: How do parallel builds scale?

[Vladimír Čunát]

On 4 March 2011 02:06, Marc Weber <marco-oweber at gmx.de> wrote:
> If you need more speed you could run the check phase separately
Certainly, especially extensive check phases on critical paths (if
there are any) are better to be separated.

By the way, do you know how much hydra waits because of waiting for
dependencies? I think it shouldn't be too difficult to always find
something else to build on hydra, but e. g. for package maintainers
it's certainly good to be able to build  more in parallel as they
usually have less packages to build.

>  foo would refer to the build
>  foo.checked would refer to the checked version which depends on all
>      map (x: x.checked) (foo.buildInputs foo.propagatedBuildInputs)
Yes, but here is IMO a small problem that the checked and unchecked
versions are build separately. Sharing them probably wouldn't be so
easy in nix.
A similar that occurred to me several times are debugging builds. In
many distributions the packages are built with -ggdb flag and then the
binaries are stripped, adding a link to the debugging symbols. The
symbols can then be installed separately (e. g. in the moment the
application crashes). It would be nice to have something like this,
but it probably isn't relevant now, with the number of users NixOS
has.

> Probably there are not enough check phases justifying the effort yet.
I feel that we should utilize automatic testing as much as possible.
This can make NixOS so much more reliable, detecting for example typos
in scripts that only affect little used features. I wonder if other
distros only use upstream test or they have some other, too.

Using stats from hydra, nix could one day estimate the build time
automatically. I don't see this as a priority now, as the usual users
are IMHO going to prefer binaries from hydra in most cases.

Vlada