[Nix-dev] ZFS root boot

[Ricardo M. Correia]

Hi Danny,

I use a ZFS root pool with a btrfs /boot partition. I may not have done
everything correctly, but it seems to work for me, although my ZFS pool
only has 1 disk drive.
This is what I did:

When installing NixOS, after booting the CD, I added this to the
configuration.nix of the install cd:

boot.supportedFilesystems = [ "btrfs" "zfs" ];

Then I ran "nixos-rebuild switch". This will automatically install the ZFS
and btrfs tools, as well as compile the ZFS kernel modules.
Obviously you don't need to add btrfs if you just use ext4 as the /boot
filesystem.

Then I ran "modprobe zfs" to load the ZFS kernel modules.

After that, I created the partitions, the btrfs filesystem and the ZFS root
pool. I also did:

$ zpool set mountpoint=/ rpool

... to make sure the root filesystem of the pool will get mounted on /.
This will generate an error as it will also try to mount the filesystem on
/ at the exact moment you run that command, but you can ignore it.

To install NixOS, you have to mount the root pool and /boot under /mnt.
This is what I did:

$ zpool export rpool
$ zpool import -o altroot=/mnt rpool
$ mount /dev/sda1 /mnt/boot    (instead of /dev/sda1, put whatever
partition you used for /boot).

Finally, to install:

$ nixos-generate-config --root /mnt

After that, I had to edit /mnt/etc/nixos/hardware-configuration.nix and
delete all autodetected ZFS filesystems except for the root filesystem
("/").
It seems that if you don't do this, NixOS will try to mount them on boot
but fail because ZFS had already automatically mounted these filesystems by
itself.

Next, I edited /mnt/etc/nixos/configuration.nix and added:

boot.supportedFilesystems = [ "btrfs" "zfs" ];

... as well as changing boot.loader.grub.device as the manual says, of
course.

That's all I had to do. I still get some warnings/errors mounting
filesystems when booting NixOS, but it seems to be working fine.

By the way, my ZFS root pool is also encrypted and this was very easy to do
as well and seems to work fine.
All I had to do was run cryptsetup to format and mount the luks device just
before creating the pool (see the Arch wiki instructions as a guide for
cryptsetup).

And finally, I just added this to /mnt/etc/nixos/configuration.nix:

boot.initrd.luks.devices = [ { device = "/dev/sda2"; name = "root"; } ];

And that was all... Although I guess if you have multiple drives, you would
have to add multiple luks devices and consequently you would have to enter
your password once for each drive/partition, which could be annoying... I
don't know if there is a better way to do it, though.

I hope that helps.

- Ricardo


On Fri, Nov 8, 2013 at 8:44 AM, deCube.net | Danny Wilson
<danny at decube.net>wrote:

> Hi list,
>
> I have a machine with 5 disks in a ZFS raidz1 zpool. It would be great if
> I could boot from this.
>
> I tried running the nixos installer by following the instructions in the
> documentation, but grub couldn’t read /boot (probably because of raidz1)
> Next I tried giving every disk a small boot partition and creating raidz1
> out of the remaining space.
>
> This boots, but doesn’t get past Stage 1, busybox can’t mount /
>
>
> Does anyone have nixos configuration examples of a ZFS root boot? :-)  I
> googled my ass off.
>
>
> Cheers
> _______________________________________________
> nix-dev mailing list
> nix-dev at lists.science.uu.nl
> http://lists.science.uu.nl/mailman/listinfo/nix-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.science.uu.nl/pipermail/nix-dev/attachments/20131108/b3daadb9/attachment.html