Packer
Packer automates the creation of VM images. To that end, you need to tell Packer where to build the VM, how to boot the VM, and how to finish setting up the VM. Each of these steps are defined in the builder and one or more provisioner components.
Setup
Install Packer
See: https://www.packer.io/downloads
Ubuntu
# curl -fsSL https://apt.releases.hashicorp.com/gpg | apt-key add -
# apt-add-repository "deb [arch=amd64] https://apt.releases.hashicorp.com $(lsb_release -cs) main"
# apt-get update && sudo apt-get install packer
Red Hat / Fedora / Rocky Linux
# dnf install -y dnf-plugins-core
# dnf config-manager --add-repo https://rpm.releases.hashicorp.com/fedora/hashicorp.repo
# dnf -y install packer
Quick introduction
To help you get a sense of what Packer can do, this page will go over how to create a simple Alpine Linux template on Proxmox using Packer.
Concepts
The Packer configuration file
You can either write a Packer configuration in native HCL (HashCorp's own language) or JSON. Native HCL configs should end with a .pkr.hcl file extension, otherwise it will be read as JSON. I'll be using the older json format for now.
We will define two things in our configuration: a builder, and any number of provisioners.
Packer builder
A builder is what Packer uses to interface with the hypervisor and can be something local (like VirtualBox or VMware Workstation), something remote (like a Proxmox server), or something in the cloud (like Azure, AWS EC2). You tell the builder what type of VM you want to build, what ISOs to attach, and what size of disk to use. In the example below, we use the Proxmox builder that's included with Packer. Review the builder's documentation to see what parameters are required. At the minimum, we need to specify our Proxmox server and authentication information, the network the VM should attach to, and a ISO boot media. The VM configuration can also be defined here.
Packer automates the OS installation step by sending keystrokes defined in the "boot_command" list. Everything defined in this list will be typed into the VM in sequence character by character. Some special keystrokes like the enter key or arrow keys are defined as special strings like <enter> and <up>. Short delays can be inserted with <wait> or <wait1>, and even longer delays are possible with <wait10m> for 10 minutes or <wait10h> for 10 hours. Review the builder's documentation for a detailed list of supported boot commands.
Packer provisioner
Packer will try to complete the OS setup by connecting to the VM and running the provisioner. Packer can start a provisioner on the guest VM with either SSH or WinRM. A provisioner can be as simple as a set of shell scripts to execute, to something more complex such as triggering an Ansible playbook run.
Be aware that even if you do not have a provisioner, Packer still will need to connect to the VM once before the build process is considered successful.
Examples
Alpine Linux on Proxmox
In the example below, I will set up Alpine Linux 3.15 on Proxmox from an ISO. I've added the appropriate boot_commands so that setup-alpine works properly (the answers file doesn't seem to work?). Once the setup is complete, I add the QEMU guest agent so that Packer is able to determine the guest's IP address and connect to it via SSH using password based authentication. Once Packer connects for the first time, it will run a series of scripts in the provisioning step to finalize the VM.
{
"variables": {
"username": "terraform@pam!terraform_token_id",
"token": "***"
},
"builders": [
{
"type": "proxmox",
"proxmox_url": "https://server.home.steamr.com:8006/api2/json",
"insecure_skip_tls_verify": true,
"username": "{{user `username`}}",
"token": "{{user `token`}}",
"node": "server",
"network_adapters": [{"bridge": "vmbr0"}],
"scsi_controller": "virtio-scsi-single",
"disks": [
{
"type": "scsi",
"disk_size": "5G",
"storage_pool": "data",
"storage_pool_type": "zfspool"
}
],
"iso_url": "http://private-mirror/alpine-virt-3.15.0-x86_64.iso",
"iso_checksum": "e97eaedb3bff39a081d1d7e67629d5c0e8fb39677d6a9dd1eaf2752e39061e02",
"iso_storage_pool": "local",
"http_directory": "http",
"boot_wait": "12s",
"boot_command": [
"root<enter><wait>",
"ifconfig eth0 up \u0026\u0026 udhcpc -i eth0<enter><wait10>",
"wget http://{{ .HTTPIP }}:{{ .HTTPPort }}/answers<enter><wait>",
"BOOT_SIZE=50 setup-alpine -f answers<enter><wait2>",
"us us<enter><wait1>",
"<enter><wait1>",
"<enter><wait1>",
"<enter><wait1>",
"<enter><wait1>",
"<enter><wait1>",
"<enter><wait1>",
"<enter><wait1>",
"<enter><wait1>",
"{{user `root_pw`}}<enter><wait>",
"{{user `root_pw`}}<enter><wait3>",
"America/Edmonton<enter><wait1>",
"<enter><wait1>",
"<enter><wait1>",
"openssh<enter><wait1>",
"sda<enter><wait1>",
"lvm<enter><wait1>",
"sys<enter><wait1>",
"y<enter><wait20>",
"rc-service sshd stop<enter>",
"mount /dev/vg0/lv_root /mnt<enter>",
"mount --bind /dev /mnt/dev<enter>",
"chroot /mnt<enter>",
"echo 'PermitRootLogin yes' >> /etc/ssh/sshd_config<enter>",
"echo 'PermitRootLogin yes' >> /etc/ssh/sshd_config<enter><wait1>",
"sed -i -e 's/#//g' /etc/apk/repositories<enter>",
"apk update<enter><wait5>",
"apk add --no-cache qemu-guest-agent<enter><wait5>",
"rc-update add local default<enter>",
"echo -e '#!/bin/sh\\nqemu-ga -d -p /dev/vport1p1' > /etc/local.d/Qemu.start<enter>",
"chmod 755 /etc/local.d/Qemu.start<enter>",
"exit<enter>",
"umount /mnt/dev<enter><wait1>",
"umount /mnt<enter><wait1>",
"reboot<enter>"
],
"ssh_username": "root",
"ssh_password": "{{user `root_pw`}}",
"ssh_timeout": "15m",
"unmount_iso": true,
"template_name": "alpine",
"template_description": "Alpine Linux, generated on {{ isotime \"2006-01-02T15:04:05Z\" }}"
}
],
"provisioners": [
{
"scripts": [
"scripts/00_base.sh",
"scripts/01_cloudinit.sh",
"scripts/02_docker.sh",
"scripts/05_misc.sh",
"scripts/99_cleanup.sh"
],
"type": "shell"
}
]
}
CloudStack
The CloudStack Packer plugin isn't the best but it does function. However, there are some limitations to be aware of:
- The plugin doesn't support
boot_commandwhich means you can't automate the OS install using Packer. Rather, you'll need to make the install media boot and start the install process automatically using some other way (eg. baking a Kickstart file into the ISO). - As of February 2022, the latest release of the plugin is broken against CloudStack 1.16 and results in an error while deploying an instance. You have to compile the most recent one yourself. Instructions to do this are given below.
Setup the CloudStack Packer plugin
You can compile the latest CloudStack plugin using Docker:
# git clone https://github.com/hashicorp/packer-plugin-cloudstack.git
# docker run --rm -ti -v `pwd`:/packer golang:latest bash
## In the container, run:
# go build
I then copied the compiled plugin and overwrote the existing one in ~/.config/packer/plugins/github.com/hashicorp/cloudstack/packer-plugin-cloudstack_v1.0.0_x5.0_linux_amd64.
Example config
Here's an example CloudStack Packer config in HCL.
packer {
required_plugins {
cloudstack = {
version = ">= 1.0.0"
source = "github.com/hashicorp/cloudstack"
}
}
}
source "cloudstack" "rocky" {
api_url = var.cloudstack_api_url
api_key = var.cloudstack_api_key
secret_key = var.cloudstack_secret_key
service_offering = "rcs.c2"
disk_offering = "Medium"
hypervisor = "KVM"
network = "leosnet"
source_iso = "Rocky-8.5-x86_64-minimal.iso"
template_display_text = "RockyLinux 8.5 x86_64, generated on [[:Template:Isotime \"2006-01-02T15:04:05Z\"]]"
template_name = "RockyLinux 8.5"
template_os = "CentOS 8"
template_featured = true
template_password_enabled = true
template_scalable = true
zone = "zone1"
ssh_username = "root"
ssh_password = "packer"
ssh_timeout = "30m"
}
build {
sources = [
"source.cloudstack.rocky"
]
provisioner "shell" {
scripts = [
"scripts/00_base.sh",
"scripts/01_cloudinit.sh",
"scripts/05_misc.sh",
"scripts/99_cleanup.sh"
]
}
}