vmsilo/CLAUDE.md

CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

Project Overview

vmsilo is a lightweight virtualization system inspired by Qubes OS. It runs isolated VMs using cloud-hypervisor (default) or crosvm with different security domains (banking, shopping, untrusted, etc.). VMs are configured declaratively via a NixOS module.

Environment

You are running under NixOS. If you need any tools not in the environment, use nix-shell.

Development Rules

• Update CLAUDE.md along with code. Keep this concise. Only add information that is not obvious and can not be easily looked up in code.
• Always update README.md for any user-visible changes (NixOS options, CLI flags, etc).
• There are no tests for the Nix modules.
• nix build uses the git index for source filtering. New untracked files must be git add'd before nix build will see them.

Build Commands

# Build the default rootfs image
nix build .#

Fuzzing

# Enter cargo-fuzz devShell (nightly Rust + cargo-fuzz)
nix develop .#fuzz
cd vmsilo-dbus-proxy && cargo fuzz run <target>

# Or one-liner (cargo-fuzz / libFuzzer)
nix run .#fuzz-cargo-dbus-proxy -- <target>

# Parallel cargo-fuzz (restarts automatically on crash)
nix run .#fuzz-cargo-dbus-proxy -- <target> --fork=4

# Re-test cargo-fuzz artifacts after a fix, delete those that no longer crash
nix run .#fuzz-clean-cargo-dbus-proxy -- <target>

# AFL++ with SymCC (single instance)
nix run .#fuzz-afl-dbus-proxy -- <target>

# AFL++ parallel (1 main + 3 secondaries + SymCC companion)
nix run .#fuzz-afl-dbus-proxy -- <target> --jobs=4

# Re-test AFL++ crash files after a fix
nix run .#fuzz-clean-afl-dbus-proxy -- <target>

# Generate/refresh seed corpus for both engines
nix run .#fuzz-gen-corpus

Fuzz targets live in vmsilo-dbus-proxy/fuzz/ (cargo-fuzz) and vmsilo-dbus-proxy/fuzz-afl/ (AFL++). Protocol types have Arbitrary derives behind the fuzz feature flag. Invariant assertions are in src/fuzz_helpers.rs.

Fixing fuzz findings

The threat model is an untrusted VM guest sending crafted data to the host-side dbus-proxy. The priority is preventing code execution and privilege escalation on the host — not DoS resilience (a misbehaving VM only affects its own proxy service). Therefore:

• Reject and disconnect on malformed input rather than logging and continuing. An anyhow::bail! / return Err(...) that tears down the connection is always preferable to silently discarding bad data.
• No unsafe in parsing or sanitization paths.
• Validate at the boundary (immediately after deserialization), not deep in business logic.
• Tighten the invariant assertions in src/fuzz_helpers.rs whenever you add or change a constraint — the fuzzer can only find violations it can check.

Code Style

This project uses treefmt-nix with nixfmt for formatting. Run before committing:

nix fmt

Architecture

VM Launch Flow (NixOS module)

VMs run as system services (root) for PCI passthrough and sandboxing support. crosvm drops privileges before starting the guest.

VMs start automatically when first accessed via socket activation:

1. vm-run banking firefox connects to /run/vmsilo/banking/command.socket
2. Socket activation triggers vmsilo-banking@.service (proxy template)
3. Proxy requires vmsilo-banking-vm.service, which starts crosvm
4. Proxy waits for guest vsock:5000, then forwards command

The configured user can manage VM services via polkit (no sudo required for vm-start/vm-stop).

Key Files

NixOS module (modules/): Split across many files — there is no single config.nix.

• default.nix — entry point (imports all submodules)
• options.nix — option declarations (programs.vmsilo; includes vm.<name>.hypervisor for VMM selection: cloud-hypervisor (default) or crosvm)
• scripts.nix — VM launcher scripts (crosvm/cloud-hypervisor) and user-facing CLI scripts (vm-run, vm-start, vm-stop, vm-shell)
• services.nix — systemd units (VM service, proxy, console relay, session bind, GPU device backend, wayland-seccontext)
• usb.nix — vm-usb CLI script (USB passthrough via usbip-rs)
• desktop.nix — .desktop file generation and icon copying
• networking.nix — TAP interfaces, host nftables for netvm = "host", guest network config
• netvm.nix — auto VM-to-VM and VM-to-host network links
• pci.nix — VFIO PCI passthrough setup
• overlay.nix — root overlay (raw/tmpfs) setup
• assertions.nix — validation checks
• package.nix — final package derivation
• lib/helpers.nix — crosvm argument building helpers
• css-colors.nix — CSS named color lookup table

Guest rootfs (rootfs-nixos/):

• default.nix — NixOS-based rootfs builder (outputs { erofs, kernel, initrd })
• configuration.nix — entry point (imports all guest submodules)
• guest/boot.nix — boot config, initrd, overlayfs root
• guest/users.nix — user accounts, sudo, SSH
• guest/networking.nix — network config from kernel params
• guest/wayland.nix — wayland proxy and session setup
• guest/command.nix — vsock command listener, dbus proxy, idle watchdog
• guest/system.nix — packages, graphics, fonts, misc
• guest/kernel-param-helper.nix — shared shell helper for parsing kernel cmdline
• guest/usb.nix — usbip-rs client listener service, vhci_hcd module
• Self-contained erofs image with overlayfs root; no host /nix sharing

Rust crates:

• vmsilo-balloond/ — dynamic balloon memory management daemon (equalizes host/guest free memory via virtio-balloon; run --help for CLI options)
• vmsilo-dbus-proxy/ — D-Bus proxy for system tray and notification forwarding between guest and host over vsock:5001
• vmsilo-wayland-seccontext/ — creates Wayland security context socket (wp_security_context_v1); run by per-VM systemd service before the GPU device service
• vmsilo-vsock/ — shared async vsock connection library (kernel vsock + fc-vsock CONNECT handshake, autodetection by VM name)
• vmsilo-tools/ — Rust workspace for small utilities. Contains tap-open (opens TAP device by name, execs command with inherited FD) and vsock-connect (CLI for connecting to VM vsock ports)

Other:

• patches/ — KWin/Plasma patches for VM window decoration colors and clipboard isolation
• flake.nix — exposes nixosModules.default and lib.makeRootfsNixos

Generated Scripts

• vm-run <name> <cmd> — run command in VM (starts VM on-demand via socket activation)
• vm-start <name> — start VM via systemd (polkit, no sudo)
• vm-stop [--all | <name>] — stop VM(s) via systemd (polkit, no sudo)
• vm-shell <name> — connect to VM serial console (default) or SSH with --ssh
• vm-usb [attach|detach] [<name> <vid:pid|devpath>] — list USB devices, attach/detach USB devices to VMs
• vsock-connect [--timeout <s>] [--probe] <vmname> <port> — connect to VM vsock port (autodetects hypervisor type)

See README.md for full usage details and options.

Dependencies

• Custom crosvm fork: git.dsg.is/dsg/crosvm.git
• wayland-proxy-virtwl: Wayland/X11 proxying between host and guests
• NixOS 25.11 base
• usbip-rs: git.dsg.is/dsg/usbip-rs.git — USB/IP over vsock for USB passthrough

Key Patterns

• Socket activation chain: command socket → proxy template service → VM service → crosvm
• Offline-by-default networking: VMs have no network unless network.interfaces or network.netvm is configured.
• Host netvm: network.netvm = "host" routes VM traffic through the host directly (TAP with hostAddress, host nftables NAT + input filter). VMs cannot initiate connections to host services — only established/related return traffic is allowed. No bridge or netvm VM needed.
• Root overlay: raw disk-backed (default) or tmpfs-backed overlayfs upper layer. Ephemeral — created at VM start, deleted at stop.
• Session bind: GPU-enabled VMs (default) are tied to the desktop session via per-VM systemd user services bound to graphical-session.target. For autoStart GPU VMs, the session-bind service also starts the VM on login. Non-GPU autoStart VMs start at multi-user.target (boot).
• Automatic DNS: All VMs have systemd-resolved enabled by default (guest rootfs). Netvm VMs get unbound as a recursive resolver via guestConfig injection. Downstream VMs get nameserver kernel params pointing at their netvm's IP via netvmInjections.nameservers. VMs with netvm = "host" or no netvm need manual DNS config.
• GPU device backend: vmsilo-<name>-gpu service runs the GPU device backend sandboxed; selectable via gpu.backend between vhost-device-gpu (default, vhost-device-gpu in rutabaga mode) and crosvm (crosvm device gpu). Both crosvm and cloud-hypervisor VMMs attach via vhost-user. vmsilo-<name>-wayland-seccontext must start first. GPU is enabled when any capability (wayland, opengl, vulkan) is true; wayland defaults true. Set gpu.wayland = false to disable.
• Per-VM runtime dirs: all sockets under /run/vmsilo/<vmname>/ subdirectories (not flat). virtiofs instances get per-instance dirs at /run/vmsilo/<vmname>/virtiofs-<tag>/.
• USB passthrough: usbip-over-vsock on port 5002. Guest runs usbip-rs client listen, host runs one usbip-rs host connect per device as vmsilo-<vm>-usb@<devpath>.service. Works with both crosvm and cloud-hypervisor.
• CID file: /run/vmsilo/<vmname>/cid written by prep service, read by vsock-connect and dbus-proxy for autodetection.
• Sound control socket: /run/vmsilo/<vmname>/sound/control.socket — written by vhost-device-sound when sound is enabled; used by the device tray to query and toggle playback/capture state at runtime. Initial state comes from sound.playback/sound.capture NixOS options (passed as --initial-streams).
• CH sandboxing: CH VMs use NixOS confinement (chroot), PrivateUsers=identity, PrivateNetwork, PrivatePIDs, PrivateIPC, empty CapabilityBoundingSet. TAP FDs passed via vmsilo-tap-open + ch-remote add-net. All privileged operations in ExecStartPre=+/ExecStartPost=+/ExecStopPost=+. Gated by cloud-hypervisor.disableSandbox.
• virtiofsd sandboxing: virtiofsd has built-in sandboxing (--sandbox=namespace): creates mount/PID/network namespaces, does pivot_root, drops capabilities, and applies its own seccomp filter. The systemd unit adds non-overlapping hardening: IPC/UTS namespace isolation, seccomp-based protections (clock/modules/logs/personality), capability bounding set (as defense-in-depth), and LimitNOFILE=1048576. Per-instance runtime dirs at /run/vmsilo/<vmname>/virtiofs-<tag>/. Gated by virtiofs.disableSandbox; seccomp controlled independently by virtiofs.seccompPolicy.

Gotchas

• nix build git index filtering: new files must be git add'd before nix build sees them (applies to all src = ./<path> references in flake.nix).
• Explicit UID required: the configured user must have users.users.<name>.uid set explicitly.
• O_DIRECT on root disk: intentional — bypasses host page cache to avoid double-caching (erofs is read-only, guest caches internally).
• Module split: there is no modules/config.nix. Implementation is split across scripts.nix, services.nix, desktop.nix, networking.nix, pci.nix, overlay.nix, assertions.nix, package.nix.
• cloud-hypervisor package: sourced from a custom flake input (git.dsg.is/dsg/cloud-hypervisor.git), not nixpkgs.

NixOS Module Usage

The configured user must have an explicit UID set (users.users.<name>.uid = <number>).

See README.md for the full configuration reference and examples.