LUNAROPS · OPERATIONAL UPLINK 100% UPTIME 1,247d POSTS 893 JEFF.MOON@LUNAROPS.DEV UTC --:--:--

Firecracker and the microVM

firecrackermicrovmvirtualizationcontainerssecurityserverless

A Linux container is not a security boundary you should bet a multi-tenant platform on. It is a bundle of namespaces and cgroups sharing one host kernel, and when that kernel has a bug — runc alone shipped three fresh container escapes in late 2025 — the namespace wall is exactly as strong as the kernel underneath it, which is to say: not a wall, a curtain. The sandboxed-runtimes post opened on this problem. This post is about the answer that won: give every untrusted workload its own kernel, inside a real hardware-virtualized VM, but make that VM so small and so fast that it feels like a container.

That is the microVM, and the canonical implementation is Firecracker — the open-source Virtual Machine Monitor (VMM) Amazon wrote in Rust to run AWS Lambda and Fargate. Every Lambda invocation you have ever triggered ran inside a Firecracker microVM. The genius of it is subtraction: take a hypervisor, then delete everything a transient cloud workload does not need — the BIOS, the PCI bus, USB, most device emulation, the boot loader — until what remains boots a guest Linux kernel in around 125 milliseconds with a few megabytes of memory overhead, isolated by the same KVM hardware boundary that protects EC2 instances from each other.


What Firecracker actually is

Strip away the marketing and Firecracker is one Linux process that asks KVM to create a virtual machine, hands it a minimal set of emulated devices, boots a kernel inside it, and exposes a REST API to control the whole thing. One process, one microVM. You run a thousand of them, you have a thousand processes and a thousand isolated guest kernels.

   Container (runc)                      microVM (Firecracker)
   ────────────────                      ─────────────────────
   your process                          your process
        │ syscalls                            │ syscalls
        ▼                                      ▼
   ┌──────────────┐                      ┌──────────────┐
   │ namespaces / │                      │ GUEST kernel │  <- its own kernel
   │   cgroups    │                      └──────┬───────┘
   └──────┬───────┘                      virtio │ (minimal devices)
          ▼                                      ▼
   ┌──────────────┐                      ┌──────────────┐
   │ HOST kernel  │ <- shared by all     │  Firecracker │ (Rust VMM)
   └──────────────┘                      ├──────────────┤
                                          │ KVM / HOST   │ <- hardware boundary
                                          └──────────────┘

The control surface is a tidy HTTP API served over a Unix socket — you configure the boot source, drives, network, and machine shape, then issue InstanceStart:

firecracker --api-sock /run/fc.sock &

# point it at a kernel and a root filesystem
curl --unix-socket /run/fc.sock -X PUT 'http://localhost/boot-source' \
  -d '{"kernel_image_path":"vmlinux","boot_args":"console=ttyS0 reboot=k panic=1 pci=off"}'

curl --unix-socket /run/fc.sock -X PUT 'http://localhost/drives/rootfs' \
  -d '{"drive_id":"rootfs","path_on_host":"rootfs.ext4","is_root_device":true,"is_read_only":false}'

curl --unix-socket /run/fc.sock -X PUT 'http://localhost/machine-config' \
  -d '{"vcpu_count":2,"mem_size_mib":1024}'

# boot
curl --unix-socket /run/fc.sock -X PUT 'http://localhost/actions' \
  -d '{"action_type":"InstanceStart"}'

Notice pci=off in the boot args. That is not an optimization detail — it is the whole philosophy in three characters, and it leads directly to the next section.


The minimal device model is the entire security thesis

A normal hypervisor (QEMU is the archetype) emulates an entire PC: a BIOS, a PCI bus, IDE and SATA controllers, USB, VGA, sound, a floppy drive nobody has used since 2003. Every one of those emulated devices is C code parsing untrusted input from the guest, and every one is attack surface — historically the richest source of VM-escape CVEs.

Firecracker’s answer is to emulate almost nothing. The device model is four things: virtio-net (network), virtio-block (disk), virtio-vsock (host/guest socket), and a serial console plus a one-button keyboard controller for clean shutdown. No BIOS — it boots the Linux kernel directly. No PCI, no USB, no GPU, no legacy anything. Written in Rust, the VMM is memory-safe by construction, and its tiny surface is auditable in a way QEMU’s millions of lines are not.

This is the trade at the heart of Firecracker: you give up generality to shrink the attack surface and the boot time at once. The same subtraction that makes it secure makes it fast (less to initialize) and dense (less per-VM memory). Minimalism is not a side effect here; it is the product.

The headline numbers that fall out of that design:

Property Firecracker microVM Typical QEMU VM runc container
Boot to userspace ~125 ms seconds milliseconds
Memory overhead per instance < 5 MB tens–hundreds MB negligible
Isolation boundary KVM (hardware) KVM (hardware) shared kernel
Device attack surface ~4 virtio devices full emulated PC N/A (no VMM)
Density per host thousands dozens–hundreds thousands

That combination — hardware isolation at near-container density and speed — is exactly why Lambda can afford to give every function invocation its own VM.


The jailer: defense-in-depth around the hypervisor

Firecracker’s design assumes the guest is hostile and might find a bug in the VMM itself. So in production you never run firecracker directly — you run it wrapped in the jailer, a companion that builds a hostile-resistant cell around the hypervisor process before it ever touches a guest. The reasoning is sober: if an attacker breaks out of the guest kernel and then exploits the Rust VMM, the jailer ensures they land in a box with almost nothing in it.

The jailer applies, in order:

  • a chroot into a dedicated, near-empty directory, so an escaped process sees no host filesystem;
  • a seccomp-bpf filter allowing only the roughly two-dozen syscalls Firecracker legitimately needs — everything else is killed;
  • a dedicated cgroup to cap CPU and memory and prevent a noisy or malicious VM from starving the host;
  • namespaces (mount, pid, network) and a drop of privileges to an unprivileged uid/gid;
  • pinning into a specific network namespace you prepared.
jailer --id myvm --exec-file /usr/bin/firecracker \
  --uid 30000 --gid 30000 \
  --chroot-base-dir /srv/jail \
  --netns /var/run/netns/myvm \
  -- --api-sock /run/fc.sock

So a tenant’s code has to escape two boundaries to reach the host — first the guest Linux kernel, then the KVM/jailer-confined VMM — and even a full VMM compromise drops them into a chrooted, seccomp-throttled, unprivileged dead end. That layered posture is the difference between “we run a hypervisor” and “we run untrusted code from strangers at scale and sleep at night.”


Snapshotting: clone a running machine in microseconds

The second Firecracker superpower is snapshotting. You can pause a running microVM and serialize its full state — guest memory, vCPU registers, device state — to a pair of files (a small state file plus the memory image). Later you load that snapshot into a fresh Firecracker process and resume, and the guest continues as if it had never stopped.

# pause and capture
curl --unix-socket /run/fc.sock -X PATCH 'http://localhost/vm' -d '{"state":"Paused"}'
curl --unix-socket /run/fc.sock -X PUT  'http://localhost/snapshot/create' \
  -d '{"snapshot_type":"Full","snapshot_path":"vm.snap","mem_file_path":"vm.mem"}'

# elsewhere, restore into a brand-new microVM and resume
curl --unix-socket /run/fc2.sock -X PUT 'http://localhost/snapshot/load' \
  -d '{"snapshot_path":"vm.snap","mem_file_path":"vm.mem","resume_vm":true}'

The payoff is enormous for serverless and sandboxing. Instead of cold-booting a kernel and userspace per request (~125 ms, plus your runtime’s own init), you boot once, snapshot the warmed-up state — kernel up, language runtime loaded, libraries imported — and then restore that snapshot per invocation in a few milliseconds or less, often serving guest memory lazily on page-fault so the restore is nearly instantaneous. A snapshot becomes a golden image you stamp out clones from on demand. This is the mechanism behind the “pre-warmed pool” pattern every serious Firecracker platform uses to kill cold starts.

The honest caveat, and it is a real one: Firecracker does not do live migration. Snapshots move state between hosts at rest, but there is no transparent migrating-a-running-VM-under-load story. Workloads are expected to be stateless or to handle durability at the application layer. If you need live migration, that is precisely where Cloud Hypervisor (below) comes in. There is also a subtle correctness footgun: naively cloning one snapshot into many running VMs duplicates entropy and any baked-in secrets, so multi-clone setups must re-seed randomness and rotate identity after restore.


Building a multi-tenant runtime: the pattern

Firecracker is a primitive, not a platform. It boots one VM and exposes an API; you build the orchestration that turns that into a function service or a code sandbox. The proven shape:

   request ──▶ scheduler ──▶ grab a warm microVM from the pool
                               (snapshot-restored, jailer-wrapped,
                                its own netns + tap device)
                                       │
                                       ▼
                               run tenant code to completion
                                       │
                                       ▼
                               destroy the microVM (never reuse across
                               tenants) and replenish the pool

The load-bearing rules: one microVM per tenant request, never reused across trust boundaries; each wrapped by its own jailer; networking via a per-VM tap device in a dedicated network namespace; a background process keeping a pool of snapshot-restored, pre-warmed VMs ready so the request path never pays a cold boot. This is, in essence, a from-scratch description of how Lambda works. You do not have to build it raw, though — firecracker-containerd plugs microVMs into the containerd/OCI world, and Kata Containers (next section) wires them into Kubernetes via RuntimeClass, so a pod can transparently run inside a microVM instead of a shared-kernel container.


Firecracker vs gVisor vs Kata vs Cloud Hypervisor

These four names come up together constantly and they are not alternatives at the same layer — two are VMMs, one is a userspace kernel, and one is an orchestration framework that uses the VMMs. Sorting that out is most of understanding the space.

Tool What it is Isolation mechanism Notable trade-off
Firecracker Minimal KVM VMM (a building block) Own guest kernel in a hardware VM; jailer for defense-in-depth Tiny device model, no live migration, no PCI passthrough/GPU; you build the orchestration
Cloud Hypervisor Fuller KVM VMM, also in Rust Own guest kernel in a hardware VM Larger device model and live migration, at a bigger surface than Firecracker
gVisor A user-space kernel (Sentry) Intercepts guest syscalls in userspace; no dedicated guest kernel No second kernel to escape into, but syscall-compat gaps and overhead on syscall-heavy work; weaker boundary than a real VM
Kata Containers Orchestration framework Runs OCI containers inside microVMs, using Firecracker / Cloud Hypervisor / QEMU under the hood Container-native UX (CRI/RuntimeClass) over real VM isolation; operational weight of running VMs under Kubernetes

The mental model that makes it click:

  • Firecracker and Cloud Hypervisor are the engines. Both are Rust KVM VMMs; Firecracker is the minimalist (smallest surface, fastest, no migration), Cloud Hypervisor is the pragmatist (more devices, more features, live migration, slightly larger surface). Choose Firecracker when the workload is ephemeral and stateless; choose Cloud Hypervisor when you need migration, more device richness, or longer-lived VMs.
  • Kata is the chassis. It does not isolate anything itself — it takes one of those VMM engines and makes a microVM look and behave like a normal Kubernetes container, so you get hardware isolation with a kubectl-native workflow. Most teams who want microVMs in Kubernetes adopt Kata rather than wiring Firecracker by hand.
  • gVisor is the odd one out, deliberately. It refuses the second kernel entirely: its Sentry process emulates the Linux syscall ABI in userspace, so there is no guest kernel to attack. That is a genuinely different and clever boundary, lighter than a VM, but it is not hardware isolation and it pays a tax on syscall-heavy workloads where the interception cost dominates. It shines for workloads that are CPU-bound and syscall-light.

There is no universal winner. The 2026 consensus across the platforms that do this for a living is to mix them: a sandbox provider might default to Cloud Hypervisor (for migration), reach for Firecracker on the most ephemeral functions, and offer gVisor where the workload profile suits it.


Where Firecracker fits — and where it does not

Reach for Firecracker when:

  • you run untrusted or multi-tenant code — functions, CI jobs, code-execution sandboxes, AI-agent tool runners — and a shared kernel is an unacceptable boundary;
  • workloads are short-lived and stateless, so no-live-migration is a non-issue and snapshots give you instant warm starts;
  • you want maximum density and minimum cold-start at hardware-isolation strength, and you are willing to build (or adopt via Kata/firecracker-containerd) the orchestration around it.

Look elsewhere when:

  • you need PCI passthrough, GPUs, or rich devices — Firecracker’s minimal model deliberately excludes them; use full KVM/libvirt or Cloud Hypervisor;
  • you need live migration of running VMs — that is Cloud Hypervisor’s or QEMU’s job;
  • your workloads are trusted and long-lived general-purpose servers — a normal hypervisor or just containers is a better fit, and the microVM ceremony buys you nothing;
  • you are not on Linux with KVM — Firecracker is KVM-only by design (x86_64 and arm64).

And the standing caveat: Firecracker is a component, not a product. Out of the box you get a fast, secure VM and an API. Everything that makes it a platform — image building, networking, the warm pool, scheduling, lifecycle — is yours to assemble or to inherit from a higher-level project. That is a feature for builders and a wall for everyone who wanted a turnkey answer.


The verdict

Firecracker is the clearest proof that the container-versus-VM dichotomy was always false. By subtracting a hypervisor down to four virtio devices and a direct kernel boot, Amazon built something with a VM’s hardware isolation, a container’s speed and density, and an attack surface small enough to put in front of the entire internet’s untrusted code. If you are running other people’s code — functions, CI, sandboxes, agent tools — the question is no longer whether you need stronger-than-namespace isolation but which microVM stack delivers it: Firecracker for ephemeral and minimal, Cloud Hypervisor when you need migration and devices, Kata to make either feel native in Kubernetes, and gVisor when a userspace kernel’s lighter, different boundary fits the workload. The shared-kernel container remains the right tool for trusted workloads — but it was never a sandbox, and the microVM is what we reach for when “trusted” is not on the table.


Sources

Comments