[Nix-dev] Sharing the Nix store between systems

[Marc Weber]

> I had it figured out quite well, with one "master" machine that could  
> perform all writes to the store and to the Nix database, but then I  
> discovered that I could not get this master machine to cross-build ARM for  
> a complete NixOS system. So, then I tried to share both the store and the  
> Nix database with write permissions to two different systems (one ARM and  
> one x86_64). Unfortunately, the database got severly corrupted when both  
> machines was building things at the same time. I guess this is a feature  
> of sqlite+NFS+file locks.
NFS can be used in different ways. If you use the kernel build in or
such locking does not work.

I'm interested in this topic as well - because it is the fastest way to
distribute builds without copying store paths first.

> So, now I'm looking for suggestions on what to try next. I guess the  
> sqlite problem could be solved if a PostgreSQL or MySQL backend was  
No, you're probably not going to solve the issue because there are not
only sqlite locks but also locks on .drv files and such ? I'm not 100%
sure about it - but you should double check it..

Instead of using MySQL you could also create a binary protocol and run
all sqlite operations on the host. AFAIK the nixpkgs code does separate
the store implementation from rest - thus it should be easy to add
another protocol based implementation. But this does not solve locking
issues on .drv files and such as noted above.

Thus the easiest way is to get the locking working ...
AFAIK NFS does support locking if configured correctly.
Eg see:
http://nfs.sourceforge.net/ 
http://www.centos.org/docs/5/html/Deployment_Guide-en-US/ch-nfs.html (-> rpc.lockd)
Looks like since V4 those additional lock protocols are no longer
required? So maybe enure that V4 is used?
http://tldp.org/HOWTO/NFS-HOWTO/security.html (6.3.3 => quote: 'full file
locking is supported')

So I'm pretty sure that its your configuration.

Keep us up to date about your progress, please.

Marc Weber