In this article, Packer images will created for Azure using azure-arm build type. The images will use an Ubuntu image available on Azure, as base image.

Let’s consider the following head of a Packer template file:

{  
  "builders": [{
      "type": "azure-arm",

      "use_azure_cli_auth": true,
      "subscription_id": "{{ user `subscription_id` }}",

      "managed_image_resource_group_name": "{{user `resource_group`}}",
      "managed_image_name": "{{user `managed_image_name`}}",

      "os_type": "{{user `basevm_os_type`}}",
      "image_publisher": "{{user `basevm_publisher`}}",
      "image_offer": "{{user `basevm_offer`}}",
      "image_sku": "{{user `basevm_sku`}}",

      "virtual_network_name": "{{user `virtual_network_name`}}",
      "virtual_network_resource_group_name": "{{user `virtual_network_resource_group_name`}}",
      "virtual_network_subnet_name": "{{user `virtual_network_subnet_name`}}",

      "azure_tags": {
        "dept": "DevOps",
        "task": "My custom base image"
      },

      "location": "{{user `location`}}",
      "vm_size": "{{user `vm_size`}}",

      "os_disk_size_gb": "{{user `os_disk_size_gb`}}",

      "shared_image_gallery_destination": {
        "subscription": "{{ user `subscription_id` }}",
        "resource_group": "{{user `resource_group`}}",
        "gallery_name": "{{user `gallery_name`}}",
        "image_name": "{{user `base_image`}}",
        "image_version": "{{user `base_image_version`}}",
        "replication_regions": "{{user `replication_regions`}}"
      },
      "shared_image_gallery_timeout": "2h1m1s"
    }],

All the variables surrounded by {{user `` }} are template variables, and must be configured in a separate file, example my_base_image_vars.json, which will be included later-on when initiating the Packer build command. The final image will be stored in Azure, in a Shared Image Gallery.

Packer however allows to configure extra provisioners, which will be executed once the initial virtual machine has been created and before the final image will be created.

One of those provisioner types is the ansible-local type. This provisioner allows to execute Ansible playbooks (and roles) once the virtual machine is booted, directly on the guest machine.

    "provisioners": [
      {
        "type": "shell",
        "inline_shebang": "/bin/sh -x",
        "execute_command": "chmod +x {{ .Path }}; {{ .Vars }} sudo -E sh '{{ .Path }}'",
        "script": "presetup.sh"
      },
      {
        "type": "ansible-local",
        "playbook_file": "./ansible/default_setup.yml",
        "playbook_dir": "./ansible",
        "role_paths": ["./ansible/defaults"],
        "clean_staging_directory": true,
        "staging_directory": "/tmp/packer-provisioner-ansible-local",
        "extra_arguments" : [ "--extra-vars", "ansible_python_interpreter=/usr/bin/python3" ]
      },

The first provisioner called, is the shell provisioner. This provisioner will upload and execute the script presetup.sh. This script could for instance ensure that the Ansible package is already installed before the next provisioner is called.

The second provisioner is the ansible-local type. The provisioner requires some parameters that must be configured:

• playbook_file: the Ansible playbook which will be executed
• playbook_dir: the base directory of the Ansible playbooks, roles, static_files, …
• role_paths: the path of the role which will be called in playbook_file (when required)

The above example will upload all files in the ./ansible directory (parameter playbook_dir) to the new guest virtual machine, to the directory configured in staging_directory, and will call the Ansible playbook default_setup.yml.

---
- name: Default post installation
  hosts: all
  connection: local
  become: yes
  tasks:
    - name: Import the Microsoft signing key into apt
      apt_key:
        url: "https://packages.microsoft.com/keys/microsoft.asc"
        state: present

    - name: Add the Azure CLI software repository
      apt_repository:
        repo: "deb [arch=amd64] https://packages.microsoft.com/repos/azure-cli/ {{ansible_distribution_release}} main"
        filename: azure-cli
        state: present

  roles:
    - defaults

The above Ansible playbook use the parameter connection: local. This is very important and must be included in each playbook executed by the ansible-local provisioner.

At first the playbook will execute two tasks which will ensure that the Microsoft Azure-CLI APT repository is installed and properly set up.

Afterwards, it will call the defaults Ansbile role. This role is set up as a typical Ansible role:

ansible/defaults:
tasks  templates  vars

ansible/defaults/tasks:
main.yml

ansible/defaults/templates:
modprobe_blacklist.j2  policy-rc.d.conf.j2  policy-rc.d.j2  securetty.j2  sysctl_base.j2

ansible/defaults/vars:
main.yml

The main.yml in the tasks sub-directory is used to configure the required post installation steps.

Example:

---
- name: Ensure default packages
  apt:
    name: "{{ default_packages }}"
    state: latest

- name: Ensure latest version of all packages
  apt:
    upgrade: dist
    force_apt_get: yes
    dpkg_options: 'force-confold,force-confdef'
    autoremove: yes
    autoclean: yes

- name: Update python alternatives (for Log Analytics)
  shell: |
    update-alternatives --remove-all python
    update-alternatives --install /usr/bin/python python /usr/bin/python2 1

- name: Forward Syslog-NG logs to Log Analytics
  ansible.builtin.copy:
    src: static_files/syslog-ng-lad.conf
    dest: /etc/syslog-ng/conf.d/syslog-ng-lad.conf
    owner: root
    group: root
    mode: '0644'

...

After the default installation of Ubuntu 20.4-lts, the following packages are required to get started as a hypervisor:

apt install qemu-kvm libvirt-daemon bridge-utils virtinst libvirt-daemon-system virt-top libguestfs-tools libosinfo-bin qemu-system virt-manager qemu pm-utils

Once these are installed, the vnet_hosts module needs to be pre-loaded in /etc/modules:

echo vhost_net | tee -a /etc/modules
modprobe vhost_net

The hypervisor is now ready to start creating and deploying virtual machines.

In this article, Terraform will be used to manage the virtual machines in libvirtd. All example code snippets are available on Github, under https://github.com/insani4c/terraform-libvirt

Terraform has an excellent provider (dmacvicar/libvirtd) to manage the libvirtd nodes, which needs to be loaded and initialized:

terraform {
  required_providers {
    libvirt = {
      source = "dmacvicar/libvirt"
    }
  }
}

provider "libvirt" {
  uri = "qemu+ssh://root@192.168.1.1/system"
}

In the above code snippet, Terraform will ensure the libvirt provider is loaded and it is configured to connect to the host with IP address 192.168.1.1 as the root user (this requires that a SSH public key is installed on the remote server, of the user who will be executing Terraform).

Next, the network will defined, where the virtual nodes will be deployed in.

resource "libvirt_network" "my_network" {
  name = "my_net"

  mode = "nat"

  addresses = ["10.1.2.0/24"]
  domain    = var.dns_domain

  autostart = true

  dhcp {
    enabled = false
  }

  dns {
    enabled = true

    local_only = false
    forwarders { address = "127.0.0.53" }

    hosts  {
        hostname = "host01"
        ip = "10.1.2.10"
      }
    hosts {
        hostname = "host02"
        ip = "10.1.2.20"
      }
  }  
}

The above code will ensure that a network of type NAT (to allow internal IPs, reachable from the hypervisor only) with network mask 10.1.2.0/24, will be created. It will ensure that DHCP is disabled and it will enable a DNS setup (the package dnsmasq must be installed) with two predefined hosts, host01 and host02.

Up next is the definition of the storage pools and volumes required for the virtual machines.

resource "libvirt_pool" "default" {
  name = "default"
  type = "dir"
  path = "/data/vms/cluster_storage"
}

resource "libvirt_volume" "local_install_image" {
  name   = var.local_install_image
  pool   = libvirt_pool.default.name
  source = var.os_img_url
  format = "qcow2"
}

The above defines the libvirt_pool, which basically configures the path on-disk for storing all sorts of volumes. Next it defines a volume called “local_install_image“, which will be used to set up the virtual machine as the volume will contain the “cloud image” for the installation. This volume requires two variables:

variable "os_img_url" {
  description = "URL to the OS image"
  default     = "https://cloud-images.ubuntu.com/focal/current/focal-server-cloudimg-amd64.img"
}

variable "local_install_image" {
    description = "The name of the local install image"
    default     = "base-os-ubuntu-focal.qcow2"
}