vmsilo

A NixOS VM compartmentalization system inspired by Qubes OS. Runs programs in isolated VMs using crosvm (Chrome OS VMM), displaying their windows natively on the host desktop.

Thanks to Thomas Leonard (@talex5), who wrote the wayland proxy and made qubes-lite, which made this project possible. https://gitlab.com/talex5/qubes-lite

This is a hacky side project. If you need a serious and secure operating system, use Qubes.

Features

• Qubes-style colored window decorations enforced by patched kwin
• Fast graphics
• Supports many wayland protocols for things like HDR, fractional scaling and smooth video playback
• Sound playback (no sound input yet)
• VMs can be configured fully disposable with no state kept between restarts
• Shared directories over virtiofs for easily sharing files between VMs
• PCI passthrough
• System tray integration (VM tray applets appear in host system tray, with VM color border)
• Auto shutdown idle VMs

Comparison to Qubes

The main benefits compared to Qubes are:

• Fast, modern graphics. Wayland calls are proxied to the host.
• Better power management. Qubes is based on Xen, and its support for modern laptop power management is significantly worse than linux.
• NixOS-based declarative VM config.

The cost for that is security. Qubes is laser-focused on security and hard compartmentalisation. This makes it by far the most secure general-purpose operating system there is.

Ways in which we are less secure than Qubes (list is not even remotely exhaustive):

• The host system is not isolated from the network at all. The user needs to use discipline to not access untrusted network resources from the host. Even if they do, handling VM network traffic makes the host attack surface much larger.
• There is no attempt to isolate the host system from hardware peripherals. Qubes segregates USB and network into VMs.
• Currently clipboard is shared between host and all VMs. This will be fixed at some point, the plan is to implement a two-level clipboard like Qubes.
• Proxying wayland calls means the attack surface from VM to host is way larger than Qubes' raw framebuffer copy approach.
• Probably a million other things.

If you are trying to defend against a determined, well-resourced attacker targeting you specifically then you should be running Qubes.

Quick Start

Add to your flake inputs:

{
  inputs.vmsilo.url = "git+https://git.dsg.is/dsg/vmsilo.git";
}

Import the module and configure VMs in your NixOS configuration:

{ config, pkgs, ... }: {
  imports = [ inputs.vmsilo.nixosModules.default ];

  # User must have explicit UID for vmsilo
  users.users.david.uid = 1000;

  programs.vmsilo = {
    enable = true;
    user = "david";
    hostNetworking.nat = {
      enable = true;
      interface = "eth0";
    };

    nixosVms = [
      {
        name = "banking";
        memory = 4096;
        cpus = 4;
        autoShutdown = { enable = true; after = 120; };
        network = {
          nameservers = [ "1.1.1.1" ];
          interfaces.wan = {
            type = "tap";
            tap.hostAddress = "10.0.0.254/24";
            addresses = [ "10.0.0.1/24" ];
            routes."0.0.0.0/0" = { via = "10.0.0.254"; };
          };
        };
        guestPrograms = with pkgs; [ firefox konsole ];
      }
      {
        name = "shopping";
        memory = 2048;
        cpus = 2;
        autoShutdown.enable = true;
        network = {
          nameservers = [ "1.1.1.1" ];
          interfaces.wan = {
            type = "tap";
            tap.hostAddress = "10.0.1.254/24";
            addresses = [ "10.0.1.1/24" ];
            routes."0.0.0.0/0" = { via = "10.0.1.254"; };
          };
        };
        guestPrograms = with pkgs; [ firefox konsole ];
      }
      {
        name = "personal";
        memory = 4096;
        cpus = 4;
        # No network.interfaces = offline VM
        guestPrograms = with pkgs; [ libreoffice ];
      }
    ];
  };
}

Configuration Options

programs.vmsilo

Option	Type	Default	Description
enable	bool	false	Enable vmsilo VM management
user	string	required	User who owns TAP interfaces and runs VMs (must have explicit UID)
hostNetworking.nat.enable	bool	false	Enable NAT for VM internet access
hostNetworking.nat.interface	string	""	External interface for NAT (required if nat.enable)
nixosVms	list of VM configs	[]	List of NixOS-based VMs to create
enableBashIntegration	bool	true	Enable bash completion for vm-* commands
crosvm.logLevel	string	"info"	Log level for crosvm (error, warn, info, debug, trace)
crosvm.extraArgs	list of strings	[]	Extra args passed to crosvm before "run" subcommand
crosvm.extraRunArgs	list of strings	[]	Extra args passed to crosvm after "run" subcommand
vmsilo-balloond.logLevel	string	"info"	Log level for vmsilo-balloond daemon (error, warn, info, debug, trace)
vmsilo-balloond.pollInterval	string	"2s"	Policy evaluation interval (e.g. "2s", "1s", "500ms")
vmsilo-balloond.criticalHostPercent	int	5	Host critical threshold as percentage of total RAM
vmsilo-balloond.guestAvailableBias	string	"300m"	Guest bias term (e.g. "300m", "500m")
vmsilo-balloond.extraArgs	list of strings	[]	Extra command line arguments for vmsilo-balloond daemon
vmsilo-tray.logLevel	string	"info"	Log level for tray proxy host and guest daemons (error, warn, info, debug, trace)
isolatedPciDevices	list of strings	[]	PCI devices to isolate with vfio-pci

VM Configuration (nixosVms items)

Option	Type	Default	Description
name	string	required	VM name for scripts
memory	int	1024	Memory allocation in MB
cpus	int	2	Number of virtual CPUs
color	string	"red"	Window decoration color (named color or hex, e.g., "#2ecc71")
network.nameservers	list of strings	[]	DNS nameservers for this VM
network.interfaces	attrset of interface configs	{}	Network interfaces (keys are guest-visible names)
autoShutdown.enable	bool	false	Auto-shutdown when idle (after autoShutdown.after seconds)
autoShutdown.after	int	60	Seconds to wait before shutdown
autoStart	bool	false	Start VM automatically at boot instead of on-demand
dependsOn	list of strings	[]	VM names to also start when this VM starts
additionalDisks	list of disk configs	[]	Additional disks to attach (see Disk Configuration)
rootDisk	disk config or null	null	Custom root disk (defaults to built rootfs)
kernel	path or null	null	Custom kernel image
initramfs	path or null	null	Custom initramfs
rootDiskReadonly	bool	true	Whether root disk is read-only
sharedHome	bool or string	true	Share host dir as /home/user via virtiofs (true=/shared/<vmname>, string=custom path, false=disabled)
kernelParams	list of strings	[]	Extra kernel command line parameters
gpu	bool or attrset	true	GPU config (false=disabled, true=default, attrset=custom)
sound	bool or attrset	true	Sound config (false=disabled, true=default PulseAudio, attrset=custom)
sharedDirectories	list of attrsets	[]	Shared directories with path, tag, and optional kv pairs
pciDevices	list of attrsets	[]	PCI devices to passthrough (path + optional kv pairs)
guestPrograms	list of packages	[]	VM-specific packages
guestConfig	NixOS module(s)	[]	VM-specific NixOS configuration (module, list of modules, or path)
vhostUser	list of attrsets	[]	Manual vhost-user devices
crosvm.logLevel	string or null	null	Per-VM log level override (uses global if null)
crosvm.extraArgs	list of strings	[]	Per-VM extra args (appended to global crosvm.extraArgs)
rootOverlay.type	"qcow2" or "tmpfs"	"qcow2"	Overlay upper layer: disk-backed (qcow2) or RAM-backed (tmpfs)
rootOverlay.size	string	"10G"	Max ephemeral disk size (qcow2 only). Parsed by qemu-img
crosvm.extraRunArgs	list of strings	[]	Per-VM extra run args (appended to global crosvm.extraRunArgs)

Network Interface Configuration (network.interfaces.<name>)

The network.interfaces option is an attrset where keys become guest-visible interface names (e.g., wan, internal).

Option	Type	Default	Description
type	"tap"	"tap"	Interface type
macAddress	string or null	null	MAC address (auto-generated from vmName-ifName hash if null)
tap.name	string or null	null	TAP interface name on host (default: <vmname>-<ifIndex>)
tap.hostAddress	string or null	null	Host-side IP with prefix (e.g., "10.0.0.254/24"). Mutually exclusive with tap.bridge.
tap.bridge	string or null	null	Bridge name to add TAP to (via networking.bridges). Mutually exclusive with tap.hostAddress.
dhcp	bool	false	Enable DHCP for this interface
addresses	list of strings	[]	Static IPv4 addresses with prefix
routes	attrs	{}	IPv4 routes (destination -> { via = gateway; })
v6Addresses	list of strings	[]	Static IPv6 addresses with prefix
v6Routes	attrs	{}	IPv6 routes

Shared directories

The attributes of the sharedDirectories option are the parameters to crosvm's --shared-dir. For a full list, see crosvm run --help.

Example to mount /shared/personal on the host, mapping all UID/GID on the vm to the same UID/GID on the host:

sharedDirectories = [ { path = "/shared/personal"; id = "home"; uidmap = "0 0 65535"; gidmap = "0 0 65535"; } ];

To allow only uid/gid 1000:

sharedDirectories = [ { path = "/shared/personal"; id = "home"; uid = 1000; gid = 1000; uidmap = "1000 1000 1"; gidmap = "1000 1000 1"; } ];

Note that both UID and GID of the vm user accessing files need to exist on the host, otherwise you will get an "Invalid argument" error.

Shared Home

By default, each VM's /home/user is shared from the host via virtiofs (sharedHome = true). The host directory is /shared/<vmname>. On first VM start, if the directory doesn't exist, it is initialized by copying /var/lib/vmsilo/home-template. You can seed that template with dotfiles, configs, etc.

• sharedHome = true — use default path /shared/<vmname> (default)
• sharedHome = "/custom/path" — use a custom host path
• sharedHome = false — disable, guest /home/user lives on the root overlay

Both /shared and /var/lib/vmsilo/home-template are owned by the configured user.

Disk Configuration (additionalDisks items)

Free-form attrsets passed directly to crosvm --block. The path attribute is required and used as a positional argument.

additionalDisks = [{
  path = "/tmp/data.qcow2";  # required, positional
  ro = false;                # read-only
  sparse = true;             # enable discard/trim
  block-size = 4096;         # reported block size
  id = "data";               # device identifier
  direct = false;            # O_DIRECT mode
}];
# Results in: --block /tmp/data.qcow2,ro=false,sparse=true,block-size=4096,id=data,direct=false

Wayland Proxy

waylandProxy = "wayland-proxy-virtwl";  # Default: wayland-proxy-virtwl by Thomas Leonard
waylandProxy = "sommelier";             # ChromeOS sommelier (experiment, does not work currently)

GPU Configuration

gpu = false;  # Disabled
gpu = true;   # Default: context-types=cross-domain:virgl2
gpu = { context-types = "cross-domain:virgl2"; width = 1920; height = 1080; };  # Custom

Sound Configuration

sound = false;  # Disabled
sound = true;   # Default PulseAudio config
sound = { backend = "pulse"; capture = true; };  # Custom

PCI Passthrough Configuration

pciDevices = [{
  path = "01:00.0";  # BDF format, auto-converted to sysfs path
  iommu = "on";      # Optional: enable IOMMU
}];
# Results in: --vfio /sys/bus/pci/devices/0000:01:00.0/,iommu=on

vhost-user Devices

vhostUser = [{
  type = "net";
  socket = "/path/to/socket";
}];
# Results in: --vhost-user type=net,socket=/path/to/socket

Each attrset is formatted as key=value pairs for crosvm --vhost-user.

Window Decoration Colors

Each VM's color option controls its KDE window decoration color, providing a visual indicator of which security domain a window belongs to:

nixosVms = [
  { name = "banking";   color = "#2ecc71"; ... }  # Green
  { name = "shopping";  color = "#3498db"; ... }  # Blue
  { name = "untrusted"; color = "red";     ... }  # Red (default)
];

The color is passed to KWin via the wayland security context. A KWin patch (included in the module) reads the color and applies it to the window's title bar and frame. Serverside decorations are forced for VM windows so the color is always visible. Text color is automatically chosen (black or white) based on the background luminance.

Supported formats: named colors ("red", "green"), hex ("#FF0000"), RGB ("rgb(255,0,0)").

Commands

After rebuilding NixOS, the following commands are available:

Run command in VM (recommended)

vm-run <name> <command>

Example: vm-run banking firefox

This is the primary way to interact with VMs. The command:

1. Connects to the VM's socket at /run/vmsilo/<name>-command.socket
2. Triggers socket activation to start the VM if not running
3. Sends the command to the guest

Start/Stop VMs

vm-start <name>    # Start VM via systemd (uses polkit, no sudo needed)
vm-stop <name>     # Stop VM via systemd (uses polkit, no sudo needed)

Start VM for debugging

vm-start-debug <name>

Starts crosvm directly in the foreground (requires sudo), bypassing socket activation. Useful for debugging VM boot issues since crosvm output is visible.

Shell access

vm-shell <name>              # Connect to serial console (default)
vm-shell --ssh <name>        # SSH into VM as user
vm-shell --ssh --root <name> # SSH into VM as root

The default serial console mode requires no configuration. Press CTRL+] to escape.

SSH mode requires SSH keys configured in per-VM guestConfig (see Advanced Configuration).

Socket activation

VMs run as system services (for PCI passthrough and sandboxing) and start automatically on first access via systemd socket activation:

# Check socket status
systemctl status vmsilo-banking.socket

# Check VM service status
systemctl status vmsilo-banking-vm.service

Sockets are enabled by default and start on boot.

Network Architecture

Interface Types

TAP interfaces (type = "tap"): For host networking and NAT internet access.

• Creates a TAP interface on the host with tap.hostAddress or adds it to a bridge with tap.bridge
• tap.hostAddress and tap.bridge are mutually exclusive
• Guest uses addresses from addresses option
• Routes configured via routes option

Interface Naming

Interface names are user-specified via network.interfaces attrset keys. Names are passed to the guest via vmsilo.ifname=<name>,<mac> kernel parameters and applied at early boot via udev rules.

NAT

When hostNetworking.nat.enable = true, the module configures:

• IP forwarding (net.ipv4.ip_forward = 1)
• NAT masquerading on hostNetworking.nat.interface
• Internal IPs derived from TAP interface addresses

PCI Passthrough

Pass PCI devices (USB controllers, network cards) directly to VMs for hardware isolation.

Configuration

programs.vmsilo = {
  # Devices to isolate from host (claimed by vfio-pci)
  isolatedPciDevices = [ "01:00.0" "02:00.0" ];

  nixosVms = [{
    name = "sys-usb";
    memory = 1024;
    pciDevices = [{ path = "01:00.0"; }];  # USB controller
  }
  {
    name = "sys-net";
    memory = 1024;
    pciDevices = [{ path = "02:00.0"; }];  # Network card
  }];
};

# Recommended: blacklist native drivers for reliability
boot.blacklistedKernelModules = [ "xhci_hcd" ];  # for USB controllers

How It Works

1. Early boot: vfio-pci claims isolated devices before other drivers load
2. Activation: If devices are already bound, they're rebound to vfio-pci
3. VM start: IOMMU groups are validated, then devices are passed via --vfio

Architecture

Each NixOS VM gets:

• A dedicated qcow2 rootfs image with packages baked in
• Overlayfs root (read-only ext4 lower + ephemeral qcow2 upper by default, tmpfs fallback)
• Wayland proxy for GPU passthrough (wayland-proxy-virtwl or sommelier)
• Session setup via vmsilo-session-setup (imports display variables into user manager, starts graphical-session.target)
• Socket-activated command listener (vsock-cmd.socket + vsock-cmd@.service, user services gated on graphical-session.target)
• Optional idle watchdog for auto-shutdown VMs (queries user service instances)
• Systemd-based init

The host provides:

• Persistent TAP interfaces via NixOS networking
• NAT for internet access (optional)
• Socket activation for commands (/run/vmsilo/<name>-command.socket)
• Console PTY for serial access (/run/vmsilo/<name>-console)
• VM services run as root for PCI passthrough and sandboxing (crosvm drops privileges)
• Polkit rules for the configured user to manage VM services without sudo
• CLI tools: vm-run, vm-start, vm-stop, vm-start-debug, vm-shell
• Desktop integration with .desktop files for guest applications

Tray Integration

VM system tray applets (StatusNotifierItems like nm-applet, bluetooth, etc.) automatically appear in the host KDE system tray. No configuration needed — tray proxying is always enabled.

• Tray items are prefixed with the VM name (e.g., [banking] NetworkManager)
• User interactions (clicks, scrolls, context menus) are forwarded back to the guest app
• Host-side whitelist sanitization ensures no untrusted data reaches the host D-Bus unchecked (pixmap dimensions, string lengths, menu depth/count limits, markup stripping)
• One host service per VM (vmsilo-<name>-tray.service), bound to VM lifecycle