[Nix-dev] Should we drop 9P?

[Eelco Dolstra]

Hi,

On 03/20/2017 11:27 PM, Volth wrote:

> 9P is used by NixOS to share host's nix store with Qemu virtual
> machines. Such technique is used in the build process, in the
> test-driver, so to say in the critical places.
> Recently few bugs in 9P were found (#23957 #23020 #22695) which
> reveals that 9P code is not very mature and perhaps NixOS is the first
> team which uses 9P heavily and relies on it in production.

I wouldn't call using it in tests "production" :-)

> Shouldn't we replace 9P with something battle-tested like NFS or Samba?

We used to use Samba/CIFS for the VM tests, but it was slower than 9p [1],
required TCP/IP at an early stage in the initrd, and required babysitting a
Samba process, whereas 9p is built into QEMU. It was also not super-reliable
(e.g. CIFS mounting had a tendency to time out if the host was under high load).

A while ago I played with Ganesha NFS but couldn't get it to work. It's probably
doable but much more complicated than passing a couple of flags to QEMU.

[1] https://github.com/NixOS/nixpkgs/commit/0e386d0c1373809d3459d09532ad36be0faf6dfb

> It may also improve the performance because 9P server works in qemu
> process, in user mode and there are as many servers as virtual
> machines running.

It would probably be slower, requiring more context switches between the VM,
QEMU and the FS server. Also, 9p uses the virtio transport which is ideal for VM
<-> host communication.

Vladimir wrote:

>> In terms of performance getting rid of QEMU where possible is probably
>> the better option.  Containers are fairly mature these days, and then
>> sharing file-systems is a matter of bind-mounting.
> 
> For some tests containers certainly aren't enough, as we do want to test
> booting, the kernel, etc.

Also, running a NixOS system requires a full UID namespace, whereas currently
Nix builds only get a single UID. For tests that don't require testing the
kernel or initrd, running them as a container would be great though.

-- 
Eelco Dolstra | LogicBlox, Inc. | http://nixos.org/~eelco/