Proxmox

Proxmox Virtual Environment (VE) is an open source virtualization management software that allows deployment and management of KVM-based virtual machines and LXC containers. The platform itself run on a Debian based distribution and supports filesystems such as ZFS and Ceph.

In terms of functionality, Proxmox VE is similar to VMware ESXi with some overlap with vSphere. You can have a cluster of Proxmox servers controlled and managed through a singular web console, command line tools, or via the provided REST API.

Installation

Installation is as simple as installing any other Linux distribution. Download the Proxmox iso image from https://www.proxmox.com/en/downloads. After installation, you should be able to access the web console via HTTPS on port 8006.

Tasks

Remove the subscription nag

If your Proxmox server has no active subscription, you will be nagged every time you log in to the web interface. This can be disabled by running the following as root in the server's console:

sed -Ezi.bak "s/(Ext.Msg.show\(\{\s+title: gettext\('No valid sub)/void\(\{ \/\/\1/g" /usr/share/javascript/proxmox-widget-toolkit/proxmoxlib.js

See also, the ansible role at https://github.com/FuzzyMistborn/ansible-role-proxmox-nag-removal/blob/master/tasks/remove-nag.yml

Deployment using Hashicorp Terraform

You can use Hashicorp's Terraform deployment tool to automatically provision virtual machines on Proxmox. After installing Terraform, follow the following steps to get your deployments working.

Setup an access account on Proxmox

Create a new user account on Proxmox. Then, create a new API key for this user. Assign this user permissions to create VMs as well as volumes on the local storage.

Setup Terraform

Terraform is configured by defining plans using Hashicorp's HCL language. We will dump everything within main.tf and any variables that go with it in vars.tf. Additionally, we will rely on the Tellmate Terraform provider plugin (https://github.com/Telmate/terraform-provider-proxmox).

# cat <<EOF > main.tf
terraform {
  required_providers {
    proxmox = {
      source = "telmate/proxmox"
      version = "2.7.4"
    }
  }
}



provider "proxmox" {
  pm_api_url = "https://proxox-server:8006/api2/json"
  pm_api_token_id = "terraform@pam!terraform_token_id"
  pm_api_token_secret = "e458e7bc-d8e6-4028-885b-d0896f4becfa"
  pm_tls_insecure = true
}

resource "proxmox_vm_qemu" "test-vm" {
  count = 1   # 0 will destroy the VM
  name = "test-vm-${count.index + 1}"  # count.index starts at 0. We want the VM to be named test-vm-1
  target_node = var.proxmox_host   # target proxmox host
  clone = var.template_name   # VM template to clone from

  # basic VM settings here. agent refers to guest agent
  agent = 1
  os_type = "cloud-init"
  cores = 2
  sockets = 1
  cpu = "host"
  memory = 2048
  scsihw = "virtio-scsi-pci"
  bootdisk = "scsi0"

  disk {
    slot = 0
    size = "10G"
    type = "scsi"
    storage = "data"
    iothread = 1
  }
  
  network {
    model = "virtio"
    bridge = "vmbr0"
  }
  
  lifecycle {
    ignore_changes = [
      network,
    ]
  }
  
  ipconfig0 = "ip=10.1.1.10${count.index + 1}/22,gw=10.1.1.1"
  
  # sshkeys set using variables. the variable contains the text of the key.
  sshkeys = <<EOF
  ${var.ssh_key}
  EOF
}

EOF

# cat <<EOF > vars.tf
variable "ssh_key" {
  default = "ssh-rsa  ... "
}

variable "proxmox_host" {
	default = "proxmox-server"
}

variable "template_name" {
	default = "ubuntu-2004-cloudinit-template"
}
EOF

With these two files defined, initialize the Terraform provider with terraform init. Plan the execution with terraform plan. If everything looks good, execute the plan with terraform apply.

Troubleshooting

LXC containers does not start networking

I have a Fedora based LXC container which takes a long time to start up. After the tty does start, the only network adapter isn't up. systemd-networkd is also in a a failed state with a bad exit code and status 226/namespace.

This was resolved by enabling nesting on this container with the following commands:

# pct set $CTID -features nesting=1 
# pct reboot $CTID