[Nix-dev] Sharing the Nix store between systems

[Rickard Nilsson]

Den 2011-12-13 09:20:32 skrev Lluís Batlle i Rossell <viric at viric.name>:

> On Tue, Dec 13, 2011 at 12:39:40AM +0100, Rickard Nilsson wrote:
>> one x86_64). Unfortunately, the database got severly corrupted when both
>> machines was building things at the same time. I guess this is a feature
>> of sqlite+NFS+file locks.
>
> Afaik, NFS 'file locks' are rarely well implemented, and often work bad.  
> If you
> expect that they will automatically work in your setup, you may be wrong.

Yes, this was what I suspected from the very start. But I will at least  
try with NFS4, too.


>> One idea could be if slave build machines didn't build directly into the
>> store used by the host itself, but instead wrote to a temporary store,  
>> and
>> also to a temporary database (or maybe it doesn't need to write to any  
>> db
>> at all). The master host could then pick up the build results and put  
>> them
>> into its database and into the shared store. I think that, in theory,  
>> this
>> scheme could work. How about in practice?
>
> I don't know how much remote can NIX_REMOTE be.

NIX_REMOTE can actually be very remote. I have verified that it is no  
problem to tunnel the local Nix worker socket via TCP (with socat) to a  
Nix worker socket on another host. It actually works very good. However,  
this only solves half of my problem. You can delegate builds perfectly to  
one master host in this way, but if you want to mix in build slaves (which  
you need if you have mixed CPU architectures), then you get into the  
shared store/db problem. Maybe I should just settle for dedicated build  
slaves with local store and db.


   / Rickard