[Nix-dev] Best Practices on Modularizing Configuration.nix?
Jan Malakhovski
oxij at oxij.org
Sat Mar 4 17:30:26 CET 2017
Hi.
I seem to be doing something different from all the comments, so I
though I'd share.
Firstly, I maintain a huge number of machines from a single place and
run many builds at once (and switch between nixpkgs branches a lot), so
all the symlinking doesn't cut it.
Secondly, I don't use nixops, because my configs require more flex
(nixops `hardware/software` split doesn't cut it for me, I use
`hardware/network-env/software/temp-stuff` configs for machines and
manage most of that through environment variables).
Thirdly, I don't ever use `nixos-rebuild`, because most of the time I
produce derivations locally on a laptop and then push builds to other
machines near the machines the results belong to (which `nixos-rebuild`
can't do, AFAIK). Most of the time I don't even trust those machines
enough to send all of my nix files to them. Managing subsets of files to
rsync/branches to git-push to various hosts is painful. So I just store
everything in a central place and only push derivations around.
For instance, say, I'm Elon Musk, and I have `slow-pc` machine near
`fast-build-cluster` on Mars and I want to update `slow-pc`'s software,
but I don't trust any of the two machines not to be hijacked by aliens
with (general) quantum computers that would break my public keys used
for my machines on Earth if they got their hands on key material. That,
of course, assumes aliens haven't hijacked (or got in bed with enough
people from) NSA/FiveEyes yet and can't sniff my pubkeys from undersea
cables here on Earth (or maybe I decided to spend some more billions for
the future of Humanity and build my own
non-compromised-by-government-for-your-money Internet infrastructure).
Anyway, I trust my skills not to leak keys in unrelated derivations/not
paranoid enough to setup completely separate repo for machines on Mars.
With my build.sh script I do
$ build.sh machine.slow-pc --on fast-build-cluster.on.mars --for slow-pc.on.mars --and-then switch
and go back to worrying about Earthly matters.
Actually, most of the time I do
$ build.sh machine.slow-pc --on fast-build-cluster.on.mars --tmux
and then detach from tmux, go for a walk, attach back, and only then run
the previous command, if everything seems fine. The script can be
enhanced in that regard, I know, but I'm lazy.
Some other uses:
* Rebuild laptop's config on a trusted host near me
$ build.sh machine.me --on trusted-fast-build-cluster.near.me --for . --and-then boot
* Build manpages for the machine
$ build.sh doc.slow-pc
* Build /etc
$ build.sh etc.slow-pc
* Build package `qemu` with overrides from `slow-pc`s config
$ build.sh mpackage.slow-pc.qemu
* Just a wrapper around the usual `nix-build -A qemu`
$ build.sh package.qemu
See attachments for the scripts and patches to nixpkgs required (I hope
I haven't forgot anything, my private branch is >200 commits atm).
As to the NixOS configs I use a combination of nixos and nixops
approaches: each machine has its own `machine/<name>.nix` root which
requires all the common things, including my own `modules` that threat
`nixos/modules` as `nixos/modules` treat nixpkgs.
I.e. I have my own set of NixOS subtrees of options. E.g.
oxij.role.mailer = {
enable = true;
parent = "some-host.near.me";
};
in `machine/<name>.nix configures postfix/dovecot/spam filter/etc in a
single go.
For instance, config of my machine looks like this:
# Some special snow-flaking
oxij.hardware.me = true;
oxij.hardware.xen = true;
# This adds users used on all home machines with relevant pubkeys,
# sets up network config, configures the use of common tor/i2p
# instances on home gateway (so that traffic from different machines
# won't be distinguishable from outside and to use other machines for
# cover traffic) and so on.
# This module actually pattern matches on hostname inside to disable
# access to some users on some machines (but add this users anyway so
# that `ls` would print names instead of ids).
oxij.location.home = true;
# Special user for giving public presentations and stuff. Privacy
# matters, right?
oxij.location.public = true;
# developers! Developers! DEVELOPERS!
oxij.role.dev.all = true;
oxij.role.dev.js = mkForce false;
oxij.role.tex = true;
oxij.role.emacs = true;
oxij.role.xmonad = true;
oxij.role.desktop = true;
Also, I allow overriding much of the stuff with environment variables
(unlike nixops) instead of using separate files for every singe build
option.
For instance, my machine has several other modes of operation, e.g.
running
$ MACHINE_PROFILE=roaming build.sh machine.me
adds
# Build our own tor/i2p routers and provide no external ssh access for
# "out of home" experience
oxij.location.home = mkForce false;
oxij.location.work = mkForce false;
oxij.location.roaming = true;
and
$ MACHINE_PROFILE=work build.sh machine.me
does
# Like roaming, but configure some work stuff
oxij.location.home = mkForce false;
oxij.location.roaming = true;
oxij.location.work = true;
Some other common roles:
oxij.role.gateway
oxij.role.build-slave
oxij.role.with-ui-for-normies
oxij.role.games
oxij.role.astro
Finally, I have another set of options that declares who is expected use
the machine (me, family, someone else) and who is expected to service it
in case there's an emergency an I'm not available. This influences
initial passwords, locales and keyboard layouts in X11 and tty (people
get confused with non-standard stuff, and I can't live without it).
At the time of writing I have ~800K of such modules and configs (they
include some that are normally done with dotfiles, though, as I love to
feel at home right after new install).
Speaking of installs, I actually use my own hacky patched nixos-install,
because usually I install like this:
* write new machine's expression on laptop, start building it somewhere
near to target
* boot the installer on the target, generate hardware-config.nix and
lsmod and copy it over to my laptop
* make hardware fixes in config, cancel previous build, start new build
* partition disks and such
* wait for build to finish
* run my nixos-install that installs by copying closure from a remote
machine
I someone implements that proper instead of my normal routine:
* I build two closures: minimal one, so that it fit into memory on
target, and actual one
* Copy the minimal closure to target (onto unionfs with tmpfs)
* Use patched nixos-install the minimal closure (disables binary caches,
I'm paranoid) to install from closure (yes, it can do that if you ask
it nicely, read the source, AFAIK its not documented).
* Reboot.
* Copy actual closure to HDD. Switch, reboot to check it will boot the
next time.
Having a means to install-from-remote-closure would be nice, but I don't
install new machines often enough to bother implementing that myself.
Also, lately I'm thinking I need to start using multiple *shared git
clones* of nixpkgs, because switching branches all the time got painful
enough to bother.
Cheers,
Jan
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 487 bytes
Desc: not available
URL: <http://lists.science.uu.nl/pipermail/nix-dev/attachments/20170304/cabcb6f5/attachment-0001.sig>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0001-Revert-Remove-NIXOS_LABEL-and-NIXOS_VERSION.patch
Type: text/x-patch
Size: 2131 bytes
Desc: not available
URL: <http://lists.science.uu.nl/pipermail/nix-dev/attachments/20170304/cabcb6f5/attachment-0002.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0001-nixos-export-packages-of-the-current-configuration-i.patch
Type: text/x-patch
Size: 1177 bytes
Desc: not available
URL: <http://lists.science.uu.nl/pipermail/nix-dev/attachments/20170304/cabcb6f5/attachment-0003.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: build.sh
Type: application/x-sh
Size: 5984 bytes
Desc: not available
URL: <http://lists.science.uu.nl/pipermail/nix-dev/attachments/20170304/cabcb6f5/attachment-0001.sh>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: git-curbranch
Type: application/octet-stream
Size: 67 bytes
Desc: not available
URL: <http://lists.science.uu.nl/pipermail/nix-dev/attachments/20170304/cabcb6f5/attachment-0001.obj>
More information about the nix-dev
mailing list