[Nix-dev] In multi-user Nix, let the daemon handle creation of GC roots

[sbaugh at catern.com]

Tuomas Tynkkynen <tuomas.tynkkynen at iki.fi> writes:
> How would deleting indirect roots work? Keep in mind that currently
> users can simply delete any result symlinks from nix-build with 'rm' or
> any other file management tool to remove the root. That won't work
> over NFS because such an indirect root could be created on a different
> filesystem (e.g. any local fs) that the machine running nix-daemon can't
> access to verify whether the symlink is still there.
>
> Additionally there's the problem of paths being in use by running processes
> on other machines being garbage collected under their feet.

Both of these could be handled client-side. Instead of having a per-user
roots directory in /nix/var/nix, we would have some truly per-user
directory (in /home/user/.var/nix/ or something) where roots are
put. We'd also have some kind of state tracking for what roots we've
told the nix-daemon about. Then when we one runs nix-store --gc, it
would:

1. Clean up broken links in .var/nix/roots
2. Search for runtime roots as the current user (this, incidentally, is
a critical feature on its own; my systems run with an unprivileged
nix-daemon and hidepid=2, so the nix-daemon (as it should be) cannot
determine what programs other users are running)
3. Tell the nix-daemon to remove any roots that were previously
registered and are no longer present, excluding roots that are currently
runtime roots.

Unfortunately, this would mean that if a user runs a program out of the
store, without first contacting the daemon and registering a (temporary)
root, the store paths for that program could be GC'd out from under
it. But I think that makes sense, anyway - the runtime root stuff
inherently requires a lot of privileges for the nix-daemon, which are
impractical/inadvisable to grant it in containerized/multi-host setups.
Much better to move runtime root detection to be as local as possible.

This would be an optional mode of operation, of course, global runtime
roots are still helpful if you can get them.