Posted inVirtualization

Automate Virtual Machine Deployment on Proxmox/KVM with Cloud-Init: Create Templates and Customize on First Boot

Virtualization tutorial - IT technology blog
Virtualization tutorial - IT technology blog

Why Automate Virtual Machine Deployment?

In today’s IT landscape, managing and deploying server infrastructure is crucial. For those running a homelab or a small system, constantly creating or configuring virtual machines (VMs) can be very time-consuming and prone to errors. For instance, I currently manage 12 VMs and containers on Proxmox VE for my homelab – where I test everything before official deployment. With this volume, I am always looking for ways to automate all processes.

There are several methods for deploying virtual machines. Below are the common approaches:

1. Manual Deployment

This is the most basic method. You create the VM step-by-step via the Proxmox web interface (or virt-manager for KVM), then manually install the operating system, configure networking, necessary software packages, create users, and so on. For just one or two VMs, this is not an issue. However, if you need 5-10 or more VMs with similar configurations, this approach quickly becomes a nightmare.

Advantages:

  • Easy to understand, suitable for beginners.
  • Complete control over each step.

Disadvantages:

  • Very time-consuming.
  • Prone to human error, leading to inconsistency among VMs.
  • Not scalable when the number of VMs increases.

2. Scripting & VM Cloning

A more advanced method is to create a base VM. You install the operating system and common tools, then clone it into multiple other VMs. After cloning, you can use scripts (e.g., shell script, Ansible) to SSH into each VM and run specific configuration commands, such as changing the hostname, IP, or installing particular services.

Advantages:

  • Much faster than the manual method.
  • Ensures cloned VMs have a similar basic configuration set.

Disadvantages:

  • Still requires a configuration step after cloning, usually via SSH.
  • Scripts can become complex if they need to handle many different conditions.
  • Not as “clean” as having each VM initialized with the correct desired configuration from the start.

3. Cloud-Init: The Optimal Solution for Automation

Cloud-Init is an open-source software package, pre-installed on most Linux Cloud Images, such as Ubuntu or CentOS Cloud Images. Its core function is to automatically configure the VM from the first boot.

Cloud-Init reads configuration data from a special source (such as a virtual CD-ROM drive or metadata service in a cloud environment). It then performs tasks such as:

  • Setting up the hostname.
  • Configuring networking (static IP, DHCP).
  • Creating users and setting passwords.
  • Adding public SSH keys for access.
  • Executing custom scripts.

Advantages:

  • Full Automation: Simply provide the configuration, and the VM will “self-setup” automatically on first boot.
  • Consistency: Ensures every deployed VM has an identical configuration as defined.
  • Flexibility: Provides extensive customization capabilities for individual VMs or groups of VMs.
  • Efficiency: Significantly saves time, reduces manual configuration errors by up to 90%, and speeds up VM deployment.
  • No Initial SSH Needed: SSH keys can be configured from the start, allowing access to the VM without a password.

Disadvantages:

  • Initial setup to create a Cloud-Init template can be a bit complex for beginners.
  • Requires OS images (Cloud Image) with Cloud-Init support.

For my homelab, Cloud-Init is an ideal tool for deploying new services quickly and consistently. I can create VMs for Kubernetes, Docker Swarm, or simply a web server in just a few minutes, with all basic configurations ready.

Guide to Deploying Cloud-Init on Proxmox VE

To start using Cloud-Init, just follow the steps below:

Step 1: Prepare the Cloud-Init OS Image (Cloud Image)

First, you need to download an operating system image optimized for cloud environments and with Cloud-Init pre-integrated. Ubuntu Cloud Image is a popular and easy-to-use choice.

Access the Proxmox VE shell (via SSH or Console in the web interface) and download the image:


# Change to temporary directory
cd /var/lib/vz/template/iso

# Download Ubuntu Server 22.04 LTS Cloud-Init image
wget https://cloud-images.ubuntu.com/releases/22.04/release/ubuntu-22.04-server-cloudimg-amd64.img

# Or the latest image (check the link on the Ubuntu Cloud Images homepage)
# wget https://cloud-images.ubuntu.com/jammy/current/jammy-server-cloudimg-amd64.img

After downloading, you need to import this image into a new volume on your Proxmox storage. I usually use local-lvm for VM disks.


# Replace 'local-lvm' with the name of the storage where you want to save the image
# Replace '100' with the next available VM ID. Example: 100
# Replace 'ubuntu-22.04-server-cloudimg-amd64.img' with the name of the file you just downloaded

qm importdisk 100 ubuntu-22.04-server-cloudimg-amd64.img local-lvm

Step 2: Create a Basic Cloud-Init VM Template

Now, you will create a new VM to use as a template. This VM’s ID needs to match the ID you used in the disk import step (e.g., 100).

Create a new VM in Proxmox GUI:

  1. Click Create VM.
  2. General: Enter VM ID (e.g., 100), Name (e.g., ubuntu-2204-cloudinit-template).
  3. OS: Select Do not use any media.
  4. System: Leave as default, or select Qemu Agent to install later.
  5. Hard Disk: Delete the default hard disk. We will use the imported disk from above.
  6. CPU: Customize as desired (e.g., 2 Cores).
  7. Memory: Customize (e.g., 2048 MB).
  8. Network: Select VirtIO (paravirtualized), configure the bridge suitable for your network.
  9. Confirm: Review and uncheck Start after creation.

Attach the imported disk and configure the Cloud-Init Drive:

After the VM is created, select VM 100 (ubuntu-2204-cloudinit-template), go to the Hardware tab:

  1. Unused Disk 0: Select the imported disk (local-lvm:vm-100-disk-0), click Edit, choose Bus/Device: VirtIO Block, click Add.
  2. Add -> Cloud-Init Drive: Select local-lvm (or your preferred storage), click Add.
  3. (Optional) Add -> CD/DVD Drive: Leave empty (if you want to manually install more packages before converting to a template).
  4. Display: Choose SPICE or VNC as desired.

Configure Cloud-Init in VM Options:

Go to the Cloud-Init tab of VM 100:

  • User: Set the default user name (e.g., ubuntu). Cloud-Init will create this user and grant sudo privileges.
  • Password: Set a password for this user.
  • SSH Public Key: Paste your public key here. This allows you to SSH into the VM without a password.
  • IP Configuration: Leave as default DHCP or configure Static if desired.

Adjust disk size and convert to template:

By default, Cloud-Init images are quite small. You need to increase the size of the main disk for the template before converting it:

  1. Select VM 100, go to the Hardware tab.
  2. Select Hard Disk (scsi0) (the VirtIO disk you just attached), click Disk Action -> Resize.
  3. Enter the new size (e.g., 30G).

Finally, convert this VM into a template:


# Replace '100' with the VM ID of the template
qm template 100

Or right-click on VM 100 in the Proxmox GUI, select Convert to Template.

Step 3: Deploy Virtual Machines from the Cloud-Init Template

Once you have a template, deploying new VMs is very simple. You will clone the template and configure Cloud-Init options for the new VM.

Create a new VM from the template in Proxmox GUI:

  1. Right-click on template 100 (ubuntu-2204-cloudinit-template), select Clone.
  2. Mode: Select Full Clone (recommended, as it creates an independent disk for the new VM).
  3. Target Storage: Choose the storage where you want to save the VM.
  4. VM ID: Enter the new ID (e.g., 101).
  5. Name: Name the new VM (e.g., my-webserver).
  6. Click Clone.

Configure Cloud-Init for the new VM:

After cloning, select VM 101 (my-webserver), go to the Cloud-Init tab:

  • Hostname: Enter the desired hostname (e.g., webserver01).
  • IP Configuration: Configure static IP or DHCP according to this VM’s needs. Example static IP:
    • 
# IPv4
ipconfig0: ip=192.168.1.101/24,gw=192.168.1.1

# You can add DNS servers
dns: 1.1.1.1 8.8.8.8
dns-search: mydomain.local
  • User, Password, SSH Public Key: You can leave these blank to inherit from the template, or re-enter them if you want to change them specifically for this VM.
  • User data: This is the most powerful part. You can add bash scripts or YAML configurations for Cloud-Init to execute when the VM boots. For example, installing Nginx:
    • 
#cloud-config
package_update: true
package_upgrade: true
packages:
  - nginx
runcmd:
  - systemctl enable nginx
  - systemctl start nginx
  - [ sh, -c, "echo 'Hello from Cloud-Init on webserver01!' > /var/www/html/index.nginx-debian.html" ]

After configuring, simply Start VM 101. Cloud-Init will do the rest.

Step 4: Check and Confirm

After the VM finishes booting, you can check if Cloud-Init has worked correctly:

  1. Access via SSH: Use the user and SSH key you configured.
    2. 
ssh [email protected]
  2. Check hostname:
    3. 
hostname
# Result: webserver01 (or the name you set)
  3. Check Nginx service (if installed):
    4. 
systemctl status nginx
curl localhost
  4. Check Cloud-Init log:
    5. 
cat /var/log/cloud-init-output.log

With these simple steps, you can fully automate the virtual machine deployment process on Proxmox VE. This will save you significant time, ensure consistency, and easily scale your homelab. If you’re struggling with manual VM configuration, try Cloud-Init – you’ll be surprised at the change in your workflow!

Share:
Tags: