[Nix-dev] Binary cache substituter

[Michael Raskin]

>Hi,
>
>On 19/09/12 14:49, Michael Raskin wrote:
>
>> Hm. What about hash-of-name -> list-of-binary patches lists? It would be
>> just as bad as MANIFEST from mutability point of view, but without 
>> downloading a huge chunk of data in one sitting.
>
>Yeah, we'll probably need some manifest-lite for the patches.  And keying it on
>the package name makes sense.  At least with patches it's not fatal it the
>manifest is not in sync - you just get full binaries instead of patches.

Keying by path means that you can afford to redownload every time.

My checklist of conditions:

1) There is not needed path locally.
2) There is another local path that differs only in hash and version.
3) Binary cache carries the needed path.
4) Patch list file is more than 15 minutes old.
5) Full path is larger than some threshold.

There is a possibility of quite a strange optimization that could help 
why keeping narinfo immutable (although loaded with junk info).

Narinfo could carry timestamp of the oldest same-name path present at
build time. Patch list would have timestamps on files.

Most of the time if full reverts of some change happen, they happen
inside two-three days of initial change. So, there is very little chance
that a path can appear, disappear for noticeable time and appear again.
So if you have any patch list and it says that the oldest upgradeable 
path is timestamped X or if the fresh narinfo you got says that it was 
created when the oldest upgradeable path was X and you have no instances
of path newer than X-(5 days), you could download patch list in 
background with low priority, and just get the binary version in the 
meantime.

Positive heuristics is that if you know the patch path, it either works
or you are likely out of luck.

I think that these 2 reckless heuristics (optional, of course) applied
after the 6 obvious hard rules would lead to most binary patch list
downloaded being used (most as in 51%).