[Nix-dev] Re: Another approach at parallelizing in-build jobs

[Lluís Batlle i Rossell]

On Sat, Jun 19, 2010 at 08:04:39PM +0200, Peter Simons wrote:
> Hi Lluís,
> 
>  > Michael Raskin proposed the idea of using substituters to achieve
>  > in-build parallelization.
> 
> unfortunately, I don't know much about substituters and their role in the build
> process. Can you (or Michael, anyone else) please post a quick explanation of
> how that approach would work? When are those "substituters" run? What exactly
> do they do?
Substituters. There are now (at least) 'download-from-manifest' (take builds
available in a manifest through curl and nix-store --import), and
'copy-from-other-stores' (take builds from a remote nix store mounted in the
system file tree):

>  > As we have little means to achieve reproduceability once we have the
>  > parameters "-j" with a number greater than one in 'make' [...].
> 
> I have heard this claim several times before, but so far no-one has been able
> to give any reasoning whatsoever to back it up. Because of that, I tend to
> believe that this claim is based on irrational fears that are rooted in a lack
> of understanding.

For my part, I think I know how make works, and yes I have a fear (not necesarily
irrational) that not many people wrote good Makefiles (in terms of
reproduceability of results in parallel builds).

Sorry for that many parentheses,
Lluís.