[Nix-dev] make -jx within builds?

[David Brown]

On Sat, May 29, 2010 at 01:13:58AM +0200, Marc Weber wrote:

>So if you have 4 cores you recommend this:
>make -j 8 -l 4
>where -j is an upper bound ?

It depends a lot on what is being built.  But, with just a bare '-j',
it will fork until the load average catches up.  I've typically seen
it get to 20 or 30 on something like the kernel.

It's also important to remember that there are some packages out there
that will need to be able to force -j 1, since their dependencies
aren't fully specified, and the builds will randomly fail if build
multi-threaded.

>We can't make the kernel take care of this, can we?
>Eg ask it to only run 1 out of four threads if cpus are busy (eg more
>than one build are running)

Being 'nice' is pretty close to what you want here, already.

>So I vote for Nix defining NUM_CORES and builders using
>
>make -j $NUM_CORES -l $NUM_CORES or similar

Seems reasonable to me, as long as we have a mechanism for packages to
be able to override it, and specify NUM_CORES=1.

David