Incus, a manager and hypervisor for system containers (LXC) and virtual machines (QEMU), is an excellent tool for managing and orchestrating your applications and services. It is a fork of LXD by the original maintainers.

I found the documentation regarding NixOS lacking and thought I should put it somewhere for future reference. If you have experience with LXD, it will mostly be similar but expect things to get different as time passes.

Installation

Incus is already present in nixpkgs and can be installed by adding

virtualisation.incus.enable = true

to your configuration.nix. Consider adding yourself to incus-admin group to avoid using sudo every time. It can be done by

users.user.USER.extraGroups = [ "incus-admin" ];

Of course, replace USER with your username.

You need IP forwarding for NAT'ing to work

boot.kernel.sysctl = {
    "net.ipv4.conf.all.forwarding" = true;
    "net.ipv4.default.forwarding" = true;
};

Enable kernel module for IP forwarding to work

boot.kernelModules = [ "nf_nat_ftp" ];

Set up a bridge

networking.bridges = { incusbr0.interfaces = []; };

This is used to provide NAT'd internet to the guest. It is manipulated directly by incus, so no need to specify any bridged interfaces here.

Add firewall rules to enable networking in the container

networking.firewall.extraCommands = ''
    iptables -A INPUT incusbr0 -j ACCEPT
    iptables -A FORWARD -o incusbr0 -j ACCEPT
    iptables -A FORWARD -i incusbr0 -j ACCEPT
    iptables -A OUTPUT -o incusbr0 -j ACCEPT
'';

Enable lxcfs to use it

virtualisation.lxc.lxcfs.enable = true;

Now switch to the new configuration with

nixos-rebuild switch

Setting up incus

Incus requires initial setup for networking and storage. It can be done interactively by running

incus admin init

List all available images

incus image list images:

Create a new image alpine based on Alpine Linux

incus launch images:alpine/3.19 alpine

Interact with the newly created image

incus exec alpine -- ash

This will drop you in an ash shell in the container.

You can copy containers by running

incus copy $CONTAINER1 $CONTAINER2

List containers

incus list

Stop container

incus stop $CONTAINER

Delete container

incus delete $CONTAINER

Configuration

Launch a new container with resource constrants

incus launch images:alpine/3.19 alp1 --config limits.cpu=1 --config limits.memory=192MiB

Check configuration

incus config show alp1

Update configuration

incus config set alp1 limits.memory=128MiB

Interaction

Run arbitrary commands

incus exec alpine -- apk update

Pull a file from container

incus file pull alpine/etc/hosts .

Push file back to the container

incus file push hosts alpine/etc/hosts

Snapshots

Create a snapshot

incus snapshot create alpine alpine_snapshot

Restore the container to the snapshot

incus snapshot restore alpine alpine_snapshot

Delete the snapshot

incus delete alpine/alpine_snapshot

References

1. Howto setup LXD on NixOS with NixOS guest using unmanaged bridge network interface

2. First steps with Incus