In this article we will have a closer look how to ensure that

• a default database has been created
• a set of configured extensions have been installed
• an initial database schema has been deployed

The user that will log on to the remote host using Ansible (and SSH), must be able to become the postgres user using sudo without a password (in this example, can be changed of course).

The tasks below are split in two big tasks, both containing a block. Blocks allow to group certain tasks which might need the same variables (for instance the same tags, become variables, …).

The first block will ensure that the default database has been created, based on a YAML variable called {{ db_name }}. This variable must be set in either a group_vars file, host_vars file or supplied at the command line.

In the same block, Ansible will also ensure that the pg_stat_statements extension is enabled and installed in the above database. pg_stat_statements is a very useful extension to debug or display statistics regarding the SQL statements executed on that specific database.

And the end of the first block, a task was added to loop over a YAML variable called {{ shared_preload_extensions }}. The variable is a supposed to be a list and should contain all extra extensions required to enabled to the above database. Shared preload extensions also need to be configured in the postgresql.conf file, which is not covered in this article.

- name: Default Database
  tags: default_database
  vars:
    ansible_become_user: postgres
    ansible_become_method: sudo
    ansible_become_pass: null
  block:
    - name: Ensure required database
      postgresql_db:
        name: "{{ db_name }}"
        encoding: UTF-8

    - name: Ensure pg_stat_statements extension
      postgresql_ext:
        name: "pg_stat_statements"
        db: "{{ db_name }}"
        schema: public
        state: present

    - name: Ensure shared_preload extensions
      postgresql_ext:
        name: "{{ item }}"
        db: "{{ db_name }}"
        state: present
      loop: "{{ shared_preload_libraries.split(',') }}"
      loop_control:
        label: " {{ item }}"

- name: Ensure initial database schema
  tags: default_database
  block:
    - name: Schema definition file
      template:
        src: "{{ db_name }}/schema_definition.sql.j2"
        dest: /var/lib/postgresql/initial_schema_definition.sql
        owner: postgres
        group: postgres
        mode: 0600
      when: ( role_path + "/templates/" + db_name + "/schema_definition.sql.j2" ) is file
      register: initial_schema

    - name: Apply the schema definition
      ignore_errors: True
      vars:
        ansible_become_user: postgres
        ansible_become_method: sudo
        ansible_become_pass: null
      community.postgresql.postgresql_query:
        db: "{{ db_name }}"
        path_to_script: /var/lib/postgresql/initial_schema_definition.sql
        encoding: UTF-8
        as_single_query: yes
      when: initial_schema.changed

  rescue:
    - debug:
        msg: "No schema definition found for {{db_name}}, skipping..."

The second block of tasks will ensure that an initial database schema is deployed. This block consists of two tasks only.

The first task will upload the schema definition SQL-based file to a specific path on the remote host.

The second task will try to execute that SQL file, if it was changed by the first task. This means of course that if the file was changed manually on the remote host (or even removed), Ansible will update that file again in the first task and it will deploy the full file again in the second task! This might overwrite or break your database.

In this article, Packer images will created for Azure using azure-arm build type. The images will use an Ubuntu image available on Azure, as base image.

Let’s consider the following head of a Packer template file:

{  
  "builders": [{
      "type": "azure-arm",

      "use_azure_cli_auth": true,
      "subscription_id": "{{ user `subscription_id` }}",

      "managed_image_resource_group_name": "{{user `resource_group`}}",
      "managed_image_name": "{{user `managed_image_name`}}",

      "os_type": "{{user `basevm_os_type`}}",
      "image_publisher": "{{user `basevm_publisher`}}",
      "image_offer": "{{user `basevm_offer`}}",
      "image_sku": "{{user `basevm_sku`}}",

      "virtual_network_name": "{{user `virtual_network_name`}}",
      "virtual_network_resource_group_name": "{{user `virtual_network_resource_group_name`}}",
      "virtual_network_subnet_name": "{{user `virtual_network_subnet_name`}}",

      "azure_tags": {
        "dept": "DevOps",
        "task": "My custom base image"
      },

      "location": "{{user `location`}}",
      "vm_size": "{{user `vm_size`}}",

      "os_disk_size_gb": "{{user `os_disk_size_gb`}}",

      "shared_image_gallery_destination": {
        "subscription": "{{ user `subscription_id` }}",
        "resource_group": "{{user `resource_group`}}",
        "gallery_name": "{{user `gallery_name`}}",
        "image_name": "{{user `base_image`}}",
        "image_version": "{{user `base_image_version`}}",
        "replication_regions": "{{user `replication_regions`}}"
      },
      "shared_image_gallery_timeout": "2h1m1s"
    }],

All the variables surrounded by {{user `` }} are template variables, and must be configured in a separate file, example my_base_image_vars.json, which will be included later-on when initiating the Packer build command. The final image will be stored in Azure, in a Shared Image Gallery.

Packer however allows to configure extra provisioners, which will be executed once the initial virtual machine has been created and before the final image will be created.

One of those provisioner types is the ansible-local type. This provisioner allows to execute Ansible playbooks (and roles) once the virtual machine is booted, directly on the guest machine.

    "provisioners": [
      {
        "type": "shell",
        "inline_shebang": "/bin/sh -x",
        "execute_command": "chmod +x {{ .Path }}; {{ .Vars }} sudo -E sh '{{ .Path }}'",
        "script": "presetup.sh"
      },
      {
        "type": "ansible-local",
        "playbook_file": "./ansible/default_setup.yml",
        "playbook_dir": "./ansible",
        "role_paths": ["./ansible/defaults"],
        "clean_staging_directory": true,
        "staging_directory": "/tmp/packer-provisioner-ansible-local",
        "extra_arguments" : [ "--extra-vars", "ansible_python_interpreter=/usr/bin/python3" ]
      },

The first provisioner called, is the shell provisioner. This provisioner will upload and execute the script presetup.sh. This script could for instance ensure that the Ansible package is already installed before the next provisioner is called.

The second provisioner is the ansible-local type. The provisioner requires some parameters that must be configured:

• playbook_file: the Ansible playbook which will be executed
• playbook_dir: the base directory of the Ansible playbooks, roles, static_files, …
• role_paths: the path of the role which will be called in playbook_file (when required)

The above example will upload all files in the ./ansible directory (parameter playbook_dir) to the new guest virtual machine, to the directory configured in staging_directory, and will call the Ansible playbook default_setup.yml.

---
- name: Default post installation
  hosts: all
  connection: local
  become: yes
  tasks:
    - name: Import the Microsoft signing key into apt
      apt_key:
        url: "https://packages.microsoft.com/keys/microsoft.asc"
        state: present

    - name: Add the Azure CLI software repository
      apt_repository:
        repo: "deb [arch=amd64] https://packages.microsoft.com/repos/azure-cli/ {{ansible_distribution_release}} main"
        filename: azure-cli
        state: present

  roles:
    - defaults

The above Ansible playbook use the parameter connection: local. This is very important and must be included in each playbook executed by the ansible-local provisioner.

At first the playbook will execute two tasks which will ensure that the Microsoft Azure-CLI APT repository is installed and properly set up.

Afterwards, it will call the defaults Ansbile role. This role is set up as a typical Ansible role:

ansible/defaults:
tasks  templates  vars

ansible/defaults/tasks:
main.yml

ansible/defaults/templates:
modprobe_blacklist.j2  policy-rc.d.conf.j2  policy-rc.d.j2  securetty.j2  sysctl_base.j2

ansible/defaults/vars:
main.yml

The main.yml in the tasks sub-directory is used to configure the required post installation steps.

Example:

---
- name: Ensure default packages
  apt:
    name: "{{ default_packages }}"
    state: latest

- name: Ensure latest version of all packages
  apt:
    upgrade: dist
    force_apt_get: yes
    dpkg_options: 'force-confold,force-confdef'
    autoremove: yes
    autoclean: yes

- name: Update python alternatives (for Log Analytics)
  shell: |
    update-alternatives --remove-all python
    update-alternatives --install /usr/bin/python python /usr/bin/python2 1

- name: Forward Syslog-NG logs to Log Analytics
  ansible.builtin.copy:
    src: static_files/syslog-ng-lad.conf
    dest: /etc/syslog-ng/conf.d/syslog-ng-lad.conf
    owner: root
    group: root
    mode: '0644'

...