Deploy to MicroVM.nix

For me, isolating the breakage of one service from another is a strong motivation to run containers and virtual machines. Upgrade some, let all others remain unaffected.

MicroVM.nix ships its own tooling for updating your independent NixOS systems: the microvm command. All that it is doing consists of maintaining the source flakeref in /var/lib/microvms/*/flake so that updates can be built from somewhere, and then creating the /var/lib/microvms/*/current symlink to the config.microvm.declaredRunner. Are these internals already, or should I start documenting them?

At c3d2.de we just switched from Proxmox to a setup that consists of MicroVMs on NixOS. It is very pleasing to see my cheap but NixOS-native “virtualization solution” arriving at more deployments. I really wished for this setup to become much more convenient than what we had before.

Both Initial deployment and updates from your local working state of this infrastructure’s flake are conducted with a nix run .#microvm-update-$hostname that will run something that looks like this:

nix copy --to ssh://root@${server} ${self}
ssh root@${server} -- bash -e<<EOF
mkdir -p /var/lib/microvms/${name}
cd /var/lib/microvms/${name}
nix build -L -o current \
  ${self}#nixosConfigurations.${name}.config.microvm.declaredRunner
echo 'git+https://gitea.c3d2.de/c3d2/nix-config?ref=flake-update' > flake
systemctl restart microvm@${name}.service
EOF

First, we copy the current flake to the server so that it can build from it, which is then done via ssh. Afterwards, the flake file is written so that microvm -u later works, and finally the new or updated MicroVM is restarted. I simplified a bit. The MicroVM can also be built locally or with a remote builder. The service shouldn’t be restarted if there was no change to the current symlink. There is not only plenty of room but really an open space to design these workflows to your own preferences and work style.

To prepare all updates from stable NixOS for our infrastructure, we’ve got a systemd.timer that runs nix flake update --commit-lock-file and pushes to that flake-update branch which is then prebuilt by our local Hydra, saving us a lot of time once we run microvm -u. We can even fetch JSON from the Hydra API to obtain the resulting store path which we can then just nix copy onto target server so that /var/lib/microvms/*/current can get updated, relieving you of that extra Nix evaluation. If you made ssh a requirement for your MicroVMs and also ran with a /nix/store mounted from the host, you could even switch to the new system without rebooting the MicroVM.

The included microvm tool still works: (stderr omitted for two lines of embarassing warnings.)

Composing infrastructure with NixOS is fun!