Firecracker

Firecracker is a virtual machine monitor that is capable of quickly starting Linux guest VMs using KVM. It's main selling feature is the speed to start up VMs. It is developed by AWS and is used by AWS in their serverless Lambda and Fargate offerings.

Setup

You need to have a Linux host with KVM already installed. The host must have a CPU should have virtualization flags which you can verify by running cat /proc/cpuinfo | grep -iE 'vmx|svm'. Intel uses the vmx flag while AMD uses the svm flag. If you intend to run Firecracker as a normal user, ensure that /dev/kvm is readable by that user.

Build or download the binaries from their Github repo: https://github.com/firecracker-microvm/firecracker

Building

# docker run --rm -ti -v `pwd`/firecracker:/firecracker rust:1.49 sh
## Inside the container, run
# git clone https://github.com/firecracker-microvm/firecracker.git

## Build, as per the README
# cd firecracker/
# tools/devtool build
# toolchain="$(uname -m)-unknown-linux-musl"

Running VMs

You can interface with Firecracker via a HTTP API or with config files. The next few examples will cover usage in both.

Basic Setup

With all the software prerequisites set up, you can now start VMs with Firecracker. The Firecracker README has some instructions on how to get your first VM running, which I will summarize below:

# Obtain the kernel and initramfs
$ wget https://s3.amazonaws.com/spec.ccfc.min/img/hello/kernel/hello-vmlinux.bin
$ wget https://s3.amazonaws.com/spec.ccfc.min/img/hello/fsfiles/hello-rootfs.ext4

# Create a config file outlining the VM
$ cat <<EOF > config.json
{
  "boot-source": {
    "kernel_image_path": "hello-vmlinux.bin",
    "boot_args": "console=ttyS0 reboot=k panic=1 pci=off"
  },
  "drives": [
    {
      "drive_id": "rootfs",
      "path_on_host": "hello-rootfs.ext4",
      "is_root_device": true,
      "is_read_only": false
    }
  ],
  "machine-config": {
    "vcpu_count": 2,
    "mem_size_mib": 1024,
    "ht_enabled": false
  }
}
EOF

# Start the VM
$ firecracker --no-api --config-file config.json

The VM should boot into a functional Linux system in a few seconds. The guest OS serial console is mapped to Firecracker's standard input/output. To stop the VM, I had to kill Firecracker from another terminal.

With Networking

Networking can be set up using tun/tap interfaces on the host.

On the host:

# ip tuntap add dev tap0 mode tap
# sysctl -w net.ipv4.conf.tap0.proxy_arp=1
# sysctl -w net.ipv6.conf.tap0.disable_ipv6=1
# ip addr add 172.17.0.1/16 dev tap0
# ip link set tap0 up

Configure the VM to have this network interface with the network-interfaces section. The MAC address can be specified with guest_mac but can be left undefined for random MAC address.

{
  "boot-source": {
    "kernel_image_path": "hello-vmlinux.bin",
    "boot_args": "console=ttyS0 reboot=k panic=1 pci=off"
  },
  "drives": [
    {
      "drive_id": "rootfs",
      "path_on_host": "hello-rootfs.ext4",
      "is_root_device": true,
      "is_read_only": false
    }
  ],
  "network-interfaces": [
      {
          "iface_id": "eth0",
          "host_dev_name": "tap0"
      }
  ],
  "machine-config": {
    "vcpu_count": 2,
    "mem_size_mib": 1024,
    "ht_enabled": false
  }
}

After starting up the VM again, you should see that eth0 is up but without an address. You will need to bring the interface up manually:

# ip addr add 172.17.100.1/16 dev eth0
# ip link set eth0 up

Or alternatively, pass the interface config as a kernel parameter. See: https://mjmwired.net/kernel/Documentation/filesystems/nfs/nfsroot.txt#81. In general, you pass: ip=IP_ADDRESS::GATEWAY:NETMASK:HOSTNAME:DEVICE:AUTOCONFIG, where AUTOCONFIG should be set to off for a static network configuration. Change the kernel arguments in the config.json file:

...
  "boot-source": {
    "kernel_image_path": "hello-vmlinux.bin",
    "boot_args": "console=ttyS0 reboot=k panic=1 pci=off ip=172.17.100.1::172.17.0.1:255.255.0.0::eth0:off"
  },
...

You should be able to ping the gateway 172.17.0.1, which will be the tap0 device on the host system.

How about internet access?

You need to make your host route traffic and act as a NAT. You then need to specify the nameservers for the guest system.