[Nix-dev] -J NUM_CORES patch - better alternative.. (for future releases ?)

[Marc Weber]

>  [...]

Thanks for the feedback.. I guess it's more important to also gather
information wether the results are likely to differ (even if tey
shouldn't) see my other post. I've got another strange idea. Haven't
thought about it in detail yet.. and it would be a lot more work than
hardcoding passing NUM_CORES.. Basically we could create two hashes per
package:
a pure one: computed by all inputs but the  AEF23459A
impure ones: NUM_CORES, doCheck etc.         234 (less vars so a smaller hash would suffice)

Then each package could have two store paths:
/nix/store/AEF23459A-234
/nix/store/AEF23459A (symlinks to AEF23459A-234)

If now nix figures out that a derivation would need adependency
/nix/store/AEF23459A-999 (999 because this time NUM_CORES is 19 and doCheck is "Sunday")
it could create another symlink
ln -s /nix/store/AEF23459A-234 /nix/store/AEF23459A-999
Then the the "Sunday" NUM_CORES=19 build would be substituted by the /nix/store/AEF23459A-234 build..

This is really nice because you can find all false friends (impure
symlinks) easily.. Just look for all "pure" links being symlinks, remove
them, tell nix to not create them again and you'll have a perfectly
pure (but maybe unbootable) system..
So we could implement a nix-store --purify ~:-] to keep it bootable..

All you have to do is classify inputs as
a) should not cause different result but may (eg doCheck=1)
b) will change the result for sure

It would be much more transparent because using my patch the only
permantent note will show up in /nix/var/nix/logs/drv which is not
traceable if you've done some remote buliding.. (or are build logs
copied as well?) BTW: What about gzipping them?

Unfortunately it's not predictable.. because running make -j 4 could
cause different results each time you try again.. So maybe we even need
3 parts:
A-B-C 
where A = hash computed from inputs which *should* determine the result
      B = hash computed from inputs which should not change the result
        (such as doCheck=1 which means either it fails or it doesn'?
         fine - let's take another example: Think of a check program
        which removes all files not passing the checks (is this used
        anywhere?) Then doCheck=1 may give a different result than
        doCheck=0 but usually will not.
      C = enable random generator.. Even when passing the same inputs
          you'll get different results (this is likely to happen if you
        use parallel builds (-j option) )
Not sure wether this last thought really makes much sense.. the first
hash made up of two parts does.

Anyway there is more which could be done than I have time to do..
so thanks to anyone having contributed to this project in the past..

Sincerly
Marc Weber