LUNAROPS · OPERATIONAL UPLINK 100% UPTIME 1,247d POSTS 893 JEFF.MOON@LUNAROPS.DEV UTC --:--:--

Ansible Playbooks: Idempotent Configuration Management at Any Scale

ansibleautomationdevopsconfiguration-managementhomelabinfrastructure-as-codelinux
Contents

At some point, every sysadmin and homelabber reaches the same inflection point. You’ve got a handful of servers running various services, and every time you spin up a new one, you’re re-running the same steps from memory: create a user, copy your SSH key, install a bunch of packages, configure the firewall, tweak sshd_config. You either have a checklist in a notes app or you just wing it and hope you don’t miss anything.

Ansible is the solution that makes this pain go away permanently. Write a playbook once, run it against any number of machines, and every server comes out identical. Break something? Run the playbook again. Add ten new nodes? Run the playbook. It handles all of it in minutes, and the playbooks are readable YAML that serves as living documentation of how your infrastructure is configured.

This guide covers everything from first installation through production-grade role organization, vault-encrypted secrets, and practical playbooks for real-world tasks.


What is Ansible and Why Should You Use It?

Ansible is an open-source automation platform that manages configuration, deploys applications, and orchestrates multi-machine workflows. It was acquired by Red Hat in 2015 and is now one of the most widely used automation tools in the industry.

Agentless: No Daemon Required

The critical architectural difference between Ansible and older tools like Puppet or Chef is that Ansible is agentless. It connects to managed nodes over SSH (or WinRM for Windows), runs tasks, and disconnects. There is no persistent daemon on your servers, no agent to install, no agent to keep updated, and no agent to troubleshoot when something breaks.

The only requirements on a managed node are: SSH access and Python 3. Both are present by default on essentially every modern Linux distribution.

Idempotency: Safe to Run Twice (or Twenty Times)

The most important property of good automation is idempotency — running a task multiple times produces the same result as running it once. If you’ve already installed nginx, running the playbook again doesn’t install it again; it verifies it’s present and moves on. If a config file already matches your template, it doesn’t get overwritten.

Ansible’s built-in modules are designed to be idempotent. The apt module doesn’t run apt-get install nginx blindly — it checks whether nginx is already installed and at the desired version first. Same for user, file, service, and virtually every other module. When a task makes no change, Ansible reports ok instead of changed. A healthy playbook run against an already-configured system should show all ok and zero changed.

This matters because it makes playbooks safe to use as remediation tools. If someone manually changes a config file on a server, you can just re-run the playbook to bring it back to the desired state.

Ansible vs Terraform: Different Jobs, Complementary Tools

These tools are frequently conflated because both manage infrastructure. They solve different problems.

Terraform is a provisioner. It creates and destroys infrastructure — virtual machines, DNS records, firewall rules, cloud storage buckets. It answers the question: “Does this resource exist?”

Ansible is a configuration manager. It installs software, writes config files, manages users, and controls services on machines that already exist. It answers the question: “Is this machine in the desired state?”

The typical workflow: Terraform provisions the VM, Ansible configures it. You can trigger Ansible from Terraform’s local-exec provisioner for post-creation setup, but they’re most powerful when used together intentionally.

Ansible vs Puppet and Chef

Puppet and Chef are pull-based systems. Each node runs an agent that periodically checks in with a central server and pulls down its configuration. This requires running and maintaining that central server, installing and managing agents on every node, and dealing with agent synchronization delays.

Ansible is push-based. You run a command on the control node and Ansible immediately pushes the tasks to the targets over SSH. No central server to maintain, no agents, no waiting for the next pull cycle. For homelabs and smaller environments, this simplicity is a significant advantage.

Chef uses Ruby DSL for its recipes, which has a learning curve. Puppet uses its own declarative language. Ansible uses YAML, which is readable by essentially everyone with basic technical background. This makes it far easier to onboard team members and to read playbooks that someone else wrote.

When to Reach for Ansible

  • Server bootstrapping: First-time setup of new machines (users, SSH keys, firewall, base packages)
  • Application deployment: Install dependencies, configure services, deploy code
  • Configuration drift remediation: Enforce known-good state across a fleet
  • Bulk OS updates: Run apt upgrade across 50 servers with a pre/post check
  • Homelab automation: Manage your entire homelab config as code in git
  • Security hardening: Enforce hardening standards (SSH config, fail2ban, firewall rules) consistently

Installation and Initial Setup

Installing Ansible on the Control Node

Ansible runs on a control node — typically your workstation, a bastion host, or a CI/CD runner. It does not need to be installed on managed nodes.

Ubuntu/Debian:

1
2
3
4
sudo apt update
sudo apt install -y software-properties-common
sudo add-apt-repository --yes --update ppa:ansible/ansible
sudo apt install -y ansible

Via pip (recommended for latest version and virtual environments):

1
2
3
4
5
python3 -m pip install --user ansible
# Or in a virtual environment:
python3 -m venv ansible-env
source ansible-env/bin/activate
pip install ansible

macOS:

1
brew install ansible

Ansible requires Python 3.9 or newer on the control node. Check your version: python3 --version. Managed nodes need Python 3 as well, which is available on any modern distribution.

The ansible.cfg Configuration File

Ansible looks for configuration in this order: ANSIBLE_CONFIG environment variable, ./ansible.cfg in the current directory, ~/.ansible.cfg in your home directory, and /etc/ansible/ansible.cfg globally. Put ansible.cfg in your project directory so it travels with your playbooks.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
# ansible.cfg
[defaults]
inventory          = inventory/hosts.yml
remote_user        = ansible
private_key_file   = ~/.ssh/id_ed25519
host_key_checking  = false          # convenient for homelab; enable for production
forks              = 10             # parallel connections
retry_files_enabled = false
stdout_callback    = yaml           # cleaner output than default
interpreter_python = auto_silent    # avoids python discovery warnings

[privilege_escalation]
become       = true
become_method = sudo
become_ask_pass = false

[ssh_connection]
ssh_args     = -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no
pipelining   = true                 # significant performance improvement

host_key_checking = false is fine for a homelab where you control all the machines. In production environments handling sensitive data, leave it enabled.

Testing Connectivity

1
ansible all -m ping

A healthy response looks like:

homelab-web01 | SUCCESS => {
    "changed": false,
    "ping": "pong"
}
homelab-db01 | SUCCESS => {
    "changed": false,
    "ping": "pong"
}

Inventory Management

The inventory tells Ansible which hosts exist and how to reach them. It also provides a natural grouping structure that drives your playbook targeting.

Static Inventory: YAML Format

The YAML format is more verbose than INI but easier to read and extend, especially with nested groups and per-host variables.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
# inventory/hosts.yml
all:
  vars:
    ansible_user: ansible
    ansible_ssh_private_key_file: ~/.ssh/id_ed25519

  children:
    webservers:
      hosts:
        web01:
          ansible_host: 192.168.1.10
        web02:
          ansible_host: 192.168.1.11
        web03:
          ansible_host: 192.168.1.12
      vars:
        nginx_worker_processes: 4
        nginx_worker_connections: 1024

    databases:
      hosts:
        db01:
          ansible_host: 192.168.1.20
          ansible_port: 22
        db02:
          ansible_host: 192.168.1.21
      vars:
        postgres_max_connections: 200
        postgres_shared_buffers: "2GB"

    monitoring:
      hosts:
        prometheus01:
          ansible_host: 192.168.1.30
        grafana01:
          ansible_host: 192.168.1.31

    kubernetes:
      children:
        k8s_control_plane:
          hosts:
            k8s-cp01:
              ansible_host: 192.168.1.40
            k8s-cp02:
              ansible_host: 192.168.1.41
        k8s_workers:
          hosts:
            k8s-w01:
              ansible_host: 192.168.1.50
            k8s-w02:
              ansible_host: 192.168.1.51
            k8s-w03:
              ansible_host: 192.168.1.52
      vars:
        k8s_version: "1.30"

    # Cross-cutting group: every host that needs base hardening
    hardened:
      children:
        webservers:
        databases:
        monitoring:
        kubernetes:

Group Variables and Host Variables

Rather than putting all variables in the inventory file, use dedicated directories:

inventory/
  hosts.yml
  group_vars/
    all.yml          # applies to every host
    webservers.yml   # applies to hosts in webservers group
    databases.yml    # applies to hosts in databases group
  host_vars/
    db01.yml         # applies only to db01
    web01.yml        # applies only to web01
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
# inventory/group_vars/all.yml
timezone: "America/Los_Angeles"
ntp_servers:
  - "0.pool.ntp.org"
  - "1.pool.ntp.org"
base_packages:
  - vim
  - curl
  - wget
  - htop
  - net-tools
  - git
  - unzip
  - jq

sshd_port: 22
sshd_permit_root_login: "no"
sshd_password_authentication: "no"
1
2
3
4
# inventory/group_vars/webservers.yml
nginx_version: "latest"
ssl_certificate_path: "/etc/ssl/certs/server.crt"
ssl_key_path: "/etc/ssl/private/server.key"
1
2
3
4
5
# inventory/host_vars/db01.yml
postgres_version: "16"
postgres_data_dir: "/data/postgres"
backup_enabled: true
backup_schedule: "0 2 * * *"

Viewing Your Inventory

1
2
3
4
5
6
7
8
# Show all hosts in a tree view
ansible-inventory -i inventory/hosts.yml --graph

# Output full inventory as JSON (useful for debugging)
ansible-inventory -i inventory/hosts.yml --list

# Show variables for a specific host
ansible-inventory -i inventory/hosts.yml --host web01

Dynamic Inventory (Brief Overview)

For larger environments where hosts come and go — cloud instances, Proxmox VMs, Kubernetes nodes — static inventory files become impractical. Dynamic inventory plugins query an external source and return the current host list.

1
2
3
4
5
# Install the Proxmox collection
ansible-galaxy collection install community.general

# Install the AWS collection
ansible-galaxy collection install amazon.aws

Dynamic inventory is configured in your ansible.cfg or as a plugin config file in the inventory directory. The AWS EC2 plugin, for example, will query the EC2 API and automatically group instances by region, VPC, tags, and instance type. Your playbooks can then target tag_role_webserver without ever touching an inventory file.


Ad-Hoc Commands

Ad-hoc commands let you run a single Ansible module against a host or group without writing a playbook. They’re useful for quick checks, one-time operations, and testing module behavior before writing it into a playbook.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
# Check connectivity
ansible all -m ping

# Install a package (requires become/sudo)
ansible webservers -m apt -a "name=nginx state=present" -b

# Check uptime on all hosts
ansible all -m shell -a "uptime"

# Copy a file to all hosts
ansible all -m copy -a "src=./config.conf dest=/etc/myapp/config.conf mode=0644" -b

# Restart a service
ansible webservers -m service -a "name=nginx state=restarted" -b

# Gather facts about a specific host
ansible web01 -m setup

# Gather a specific fact
ansible all -m setup -a "filter=ansible_distribution*"

# Execute a command on a subset of hosts
ansible webservers[0:1] -m command -a "df -h"

When to use ad-hoc commands vs playbooks: Use ad-hoc commands for quick checks, emergency fixes, one-time operations, and module exploration. Once you find yourself running the same ad-hoc commands repeatedly, or if the operation is part of a defined workflow, write a playbook. Playbooks are idempotent, version-controlled, self-documenting, and repeatable — ad-hoc commands are not.


Playbook Anatomy

Basic Structure

A playbook is a YAML file containing one or more plays. Each play targets a set of hosts and runs a list of tasks.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
---
# site.yml — top-level playbook that orchestrates everything
- name: Configure web servers
  hosts: webservers
  become: true            # use sudo for all tasks
  gather_facts: true      # collect host information (distro, IP, memory, etc.)

  vars:
    nginx_port: 80
    nginx_user: "www-data"

  pre_tasks:
    - name: Update apt cache
      ansible.builtin.apt:
        update_cache: true
        cache_valid_time: 3600  # only update if cache is older than 1 hour

  tasks:
    - name: Install nginx
      ansible.builtin.apt:
        name: nginx
        state: present

    - name: Ensure nginx is running and enabled
      ansible.builtin.service:
        name: nginx
        state: started
        enabled: true

  post_tasks:
    - name: Verify nginx is listening
      ansible.builtin.command:
        cmd: "ss -tlnp | grep :{{ nginx_port }}"
      changed_when: false   # this task never makes changes, just checks

A Complete Working Playbook: Install and Configure Nginx

This is a full, runnable playbook that installs nginx, deploys a custom config from a template, and manages the service lifecycle with handlers.

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
---
# playbooks/nginx.yml
- name: Install and configure nginx
  hosts: webservers
  become: true
  gather_facts: true

  vars:
    nginx_server_name: "{{ inventory_hostname }}"
    nginx_root: "/var/www/html"
    nginx_http_port: 80
    nginx_worker_processes: "auto"
    nginx_worker_connections: 1024
    nginx_keepalive_timeout: 65
    nginx_gzip_enabled: true
    app_name: "myapp"

  vars_files:
    - ../vars/common.yml    # shared variables across playbooks

  handlers:
    - name: reload nginx
      ansible.builtin.service:
        name: nginx
        state: reloaded

    - name: restart nginx
      ansible.builtin.service:
        name: nginx
        state: restarted

  tasks:
    - name: Install nginx
      ansible.builtin.apt:
        name: nginx
        state: present
        update_cache: true
      notify: restart nginx

    - name: Create web root directory
      ansible.builtin.file:
        path: "{{ nginx_root }}/{{ app_name }}"
        state: directory
        owner: "www-data"
        group: "www-data"
        mode: "0755"

    - name: Deploy nginx main configuration
      ansible.builtin.template:
        src: templates/nginx.conf.j2
        dest: /etc/nginx/nginx.conf
        owner: root
        group: root
        mode: "0644"
        validate: "/usr/sbin/nginx -t -c %s"
      notify: reload nginx

    - name: Deploy site virtual host configuration
      ansible.builtin.template:
        src: templates/vhost.conf.j2
        dest: "/etc/nginx/sites-available/{{ app_name }}"
        owner: root
        group: root
        mode: "0644"
      notify: reload nginx

    - name: Enable site
      ansible.builtin.file:
        src: "/etc/nginx/sites-available/{{ app_name }}"
        dest: "/etc/nginx/sites-enabled/{{ app_name }}"
        state: link
      notify: reload nginx

    - name: Remove default nginx site
      ansible.builtin.file:
        path: /etc/nginx/sites-enabled/default
        state: absent
      notify: reload nginx

    - name: Deploy index page
      ansible.builtin.template:
        src: templates/index.html.j2
        dest: "{{ nginx_root }}/{{ app_name }}/index.html"
        owner: "www-data"
        group: "www-data"
        mode: "0644"

    - name: Ensure nginx is started and enabled on boot
      ansible.builtin.service:
        name: nginx
        state: started
        enabled: true

    - name: Check nginx status and capture output
      ansible.builtin.command:
        cmd: systemctl status nginx
      register: nginx_status
      changed_when: false
      failed_when: "'active (running)' not in nginx_status.stdout"

    - name: Print nginx status
      ansible.builtin.debug:
        msg: "nginx is {{ 'running' if 'active (running)' in nginx_status.stdout else 'NOT running' }}"

Variables

Variables can come from many places. Here’s a quick tour of the most common ones:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
# Inline in the play
vars:
  app_version: "2.1.0"
  debug_mode: false

# From a file
vars_files:
  - vars/app.yml
  - vars/secrets.yml  # encrypted with ansible-vault

# From the command line (highest precedence)
# ansible-playbook site.yml -e "app_version=2.2.0"
# ansible-playbook site.yml -e @overrides.yml

Register and Debug

Capturing task output with register is essential for conditional logic and troubleshooting:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
- name: Check if config file exists
  ansible.builtin.stat:
    path: /etc/myapp/config.yml
  register: config_stat

- name: Show config file info
  ansible.builtin.debug:
    var: config_stat

- name: Create config if it doesn't exist
  ansible.builtin.template:
    src: templates/config.yml.j2
    dest: /etc/myapp/config.yml
  when: not config_stat.stat.exists

- name: Get current app version
  ansible.builtin.command:
    cmd: myapp --version
  register: app_version_output
  changed_when: false

- name: Show version
  ansible.builtin.debug:
    msg: "Current version: {{ app_version_output.stdout }}"

Loops

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
- name: Install required packages
  ansible.builtin.apt:
    name: "{{ item }}"
    state: present
  loop:
    - nginx
    - curl
    - git
    - htop
    - vim

# Loop over a list variable
- name: Install packages from variable
  ansible.builtin.apt:
    name: "{{ item }}"
    state: present
  loop: "{{ base_packages }}"

# Loop over a dict
- name: Create application users
  ansible.builtin.user:
    name: "{{ item.key }}"
    uid: "{{ item.value.uid }}"
    groups: "{{ item.value.groups }}"
    shell: "{{ item.value.shell | default('/bin/bash') }}"
  loop: "{{ app_users | dict2items }}"

# Loop with index
- name: Create numbered directories
  ansible.builtin.file:
    path: "/data/partition{{ idx }}"
    state: directory
    mode: "0755"
  loop: "{{ partitions }}"
  loop_control:
    index_var: idx
    label: "{{ item }}"

Conditionals

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
- name: Install Apache on RedHat family
  ansible.builtin.yum:
    name: httpd
    state: present
  when: ansible_os_family == "RedHat"

- name: Install nginx on Debian family
  ansible.builtin.apt:
    name: nginx
    state: present
  when: ansible_os_family == "Debian"

- name: Skip on hosts without enough memory
  ansible.builtin.debug:
    msg: "This host has {{ ansible_memtotal_mb }}MB RAM — skipping memory-intensive task"
  when: ansible_memtotal_mb < 2048

- name: Run only on primary database
  ansible.builtin.command:
    cmd: /usr/local/bin/run-migration.sh
  when:
    - inventory_hostname == groups['databases'][0]
    - run_migrations | default(false) | bool

Tags

Tags let you run a subset of tasks in a playbook without modifying the file:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
tasks:
  - name: Install packages
    ansible.builtin.apt:
      name: "{{ item }}"
      state: present
    loop: "{{ base_packages }}"
    tags:
      - packages
      - bootstrap

  - name: Configure SSH
    ansible.builtin.template:
      src: templates/sshd_config.j2
      dest: /etc/ssh/sshd_config
    tags:
      - ssh
      - security
      - hardening
1
2
3
4
5
# Run only tasks tagged 'packages'
ansible-playbook site.yml --tags packages

# Skip security-related tasks
ansible-playbook site.yml --skip-tags security

Jinja2 Templating

The template module renders Jinja2 templates on the control node and copies the result to the managed host. This is how you create config files that adapt to each target machine.

A Realistic nginx.conf.j2 Template

{# templates/nginx.conf.j2 #}
# Managed by Ansible — do not edit manually
# Generated: {{ ansible_date_time.date }} for {{ inventory_hostname }}

user {{ nginx_user | default('www-data') }};
worker_processes {{ nginx_worker_processes | default('auto') }};
pid /run/nginx.pid;

events {
    worker_connections {{ nginx_worker_connections | default(1024) }};
    multi_accept on;
    use epoll;
}

http {
    sendfile on;
    tcp_nopush on;
    tcp_nodelay on;
    keepalive_timeout {{ nginx_keepalive_timeout | default(65) }};
    types_hash_max_size 2048;
    server_tokens off;

    include /etc/nginx/mime.types;
    default_type application/octet-stream;

    # Logging
    log_format main '$remote_addr - $remote_user [$time_local] "$request" '
                    '$status $body_bytes_sent "$http_referer" '
                    '"$http_user_agent" "$http_x_forwarded_for"';

    access_log /var/log/nginx/access.log main;
    error_log  /var/log/nginx/error.log warn;

    {% if nginx_gzip_enabled | default(true) %}
    # Gzip compression
    gzip on;
    gzip_vary on;
    gzip_min_length 1024;
    gzip_proxied expired no-cache no-store private auth;
    gzip_types text/plain text/css application/json application/javascript
               text/xml application/xml application/xml+rss text/javascript;
    {% endif %}

    # Rate limiting zones
    {% for zone in nginx_rate_limit_zones | default([]) %}
    limit_req_zone {{ zone.key }} zone={{ zone.name }}:{{ zone.size | default('10m') }} rate={{ zone.rate }};
    {% endfor %}

    # Upstream blocks
    {% for upstream in nginx_upstreams | default([]) %}
    upstream {{ upstream.name }} {
        {% for server in upstream.servers %}
        server {{ server }};
        {% endfor %}
    }
    {% endfor %}

    # Include virtual hosts
    include /etc/nginx/conf.d/*.conf;
    include /etc/nginx/sites-enabled/*;
}

Useful Jinja2 Filters

{# Default value if variable is undefined or empty #}
{{ nginx_port | default(80) }}

{# String manipulation #}
{{ app_name | lower }}
{{ app_name | upper }}
{{ hostname | replace('.', '_') }}

{# List operations #}
{{ allowed_ips | join(', ') }}
{{ packages | unique | sort | join(' ') }}

{# Type conversion #}
{{ max_connections | int }}
{{ debug_mode | bool }}
{{ config_dict | to_json }}
{{ config_dict | to_nice_yaml }}

{# Testing #}
{% if ssl_cert_path is defined and ssl_cert_path != '' %}
ssl_certificate {{ ssl_cert_path }};
{% endif %}

{# Ternary operator #}
worker_processes {{ ansible_processor_vcpus if nginx_worker_processes == 'auto' else nginx_worker_processes }};

Roles: Organizing at Scale

When playbooks grow beyond a few tasks, or when you need to reuse the same configuration logic across multiple playbooks, roles are the answer. A role is a self-contained unit with a standardized directory structure.

Role Directory Structure

roles/
  common/
    tasks/
      main.yml          # entry point — include other task files from here
      packages.yml
      users.yml
      ssh.yml
      firewall.yml
    handlers/
      main.yml          # handlers referenced by tasks
    templates/
      sshd_config.j2
      fail2ban.local.j2
      ufw_rules.j2
    files/
      motd              # static files (no templating needed)
      authorized_keys
    vars/
      main.yml          # role-specific variables (high precedence)
    defaults/
      main.yml          # default values (lowest precedence — easy to override)
    meta/
      main.yml          # role metadata, dependencies
    README.md

Creating a Role

1
ansible-galaxy init roles/common

This creates the full directory structure with placeholder files.

Role Defaults vs Vars

Understanding the distinction is important:

  • defaults/main.yml — default values that should be overridden. Use these for everything configurable. They have the lowest precedence of any variable source, meaning they’ll be overridden by inventory variables, play vars, and command-line extra-vars.
  • vars/main.yml — values that are internal to the role and should NOT be overridden by the user. These have higher precedence and are for things like internal paths or constants the role depends on.

As a general rule: put everything in defaults/ and move something to vars/ only if you have a specific reason why it must not be overridden.

A Complete common Role: Hardening a Fresh Server

This role performs the essential hardening steps you’d want on every new server.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
# roles/common/defaults/main.yml
---
common_timezone: "UTC"
common_ntp_servers:
  - "0.pool.ntp.org"
  - "1.pool.ntp.org"

common_packages:
  - vim
  - curl
  - wget
  - git
  - htop
  - net-tools
  - unzip
  - jq
  - fail2ban
  - ufw
  - acl

common_admin_user: "ansible"
common_admin_groups:
  - sudo
  - adm

# SSH hardening
sshd_port: 22
sshd_permit_root_login: "no"
sshd_password_authentication: "no"
sshd_pubkey_authentication: "yes"
sshd_max_auth_tries: 3
sshd_login_grace_time: 30
sshd_client_alive_interval: 300
sshd_client_alive_count_max: 2
sshd_allowed_users: []           # empty = allow all users

# UFW firewall
ufw_default_incoming: deny
ufw_default_outgoing: allow
ufw_rules:
  - { rule: allow, port: "{{ sshd_port }}", proto: tcp, comment: "SSH" }

# Fail2ban
fail2ban_maxretry: 5
fail2ban_bantime: 3600
fail2ban_findtime: 600
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
# roles/common/tasks/main.yml
---
- name: Include package installation tasks
  ansible.builtin.include_tasks: packages.yml
  tags: [packages, bootstrap]

- name: Include user management tasks
  ansible.builtin.include_tasks: users.yml
  tags: [users, bootstrap]

- name: Include SSH hardening tasks
  ansible.builtin.include_tasks: ssh.yml
  tags: [ssh, security, hardening]

- name: Include firewall tasks
  ansible.builtin.include_tasks: firewall.yml
  tags: [firewall, security, hardening]

- name: Include fail2ban tasks
  ansible.builtin.include_tasks: fail2ban.yml
  tags: [fail2ban, security, hardening]

- name: Set timezone
  community.general.timezone:
    name: "{{ common_timezone }}"
  tags: [timezone]

- name: Configure NTP with timedatectl
  ansible.builtin.command:
    cmd: "timedatectl set-ntp true"
  changed_when: false
  tags: [ntp]
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
# roles/common/tasks/packages.yml
---
- name: Update apt cache
  ansible.builtin.apt:
    update_cache: true
    cache_valid_time: 3600

- name: Upgrade all packages to latest
  ansible.builtin.apt:
    upgrade: dist
    autoremove: true
    autoclean: true

- name: Install common packages
  ansible.builtin.apt:
    name: "{{ common_packages }}"
    state: present
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
# roles/common/tasks/ssh.yml
---
- name: Deploy hardened sshd_config
  ansible.builtin.template:
    src: sshd_config.j2
    dest: /etc/ssh/sshd_config
    owner: root
    group: root
    mode: "0600"
    validate: "/usr/sbin/sshd -t -f %s"
    backup: true
  notify: restart sshd

- name: Ensure SSH service is running and enabled
  ansible.builtin.service:
    name: ssh
    state: started
    enabled: true
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
# roles/common/tasks/firewall.yml
---
- name: Install ufw
  ansible.builtin.apt:
    name: ufw
    state: present

- name: Set default incoming policy to deny
  community.general.ufw:
    default: "{{ ufw_default_incoming }}"
    direction: incoming

- name: Set default outgoing policy to allow
  community.general.ufw:
    default: "{{ ufw_default_outgoing }}"
    direction: outgoing

- name: Apply UFW rules
  community.general.ufw:
    rule: "{{ item.rule }}"
    port: "{{ item.port | string }}"
    proto: "{{ item.proto | default('tcp') }}"
    comment: "{{ item.comment | default('') }}"
  loop: "{{ ufw_rules }}"

- name: Enable UFW
  community.general.ufw:
    state: enabled
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
# roles/common/handlers/main.yml
---
- name: restart sshd
  ansible.builtin.service:
    name: ssh
    state: restarted

- name: reload sshd
  ansible.builtin.service:
    name: ssh
    state: reloaded

- name: restart fail2ban
  ansible.builtin.service:
    name: fail2ban
    state: restarted

Using Roles in Playbooks

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
# site.yml
---
- name: Apply common configuration to all hosts
  hosts: all
  become: true
  roles:
    - role: common
      vars:
        common_timezone: "America/Los_Angeles"
        sshd_port: 2222
        ufw_rules:
          - { rule: allow, port: "2222", proto: tcp, comment: "SSH custom port" }
          - { rule: allow, port: "80", proto: tcp, comment: "HTTP" }
          - { rule: allow, port: "443", proto: tcp, comment: "HTTPS" }

- name: Configure web servers
  hosts: webservers
  become: true
  roles:
    - role: common           # apply common first
    - role: nginx            # then nginx-specific config
    - role: certbot          # then TLS cert management

Ansible Galaxy and requirements.yml

Ansible Galaxy is the community hub for sharing roles and collections. Avoid reinventing wheels — there are battle-tested roles for nearly everything.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
# requirements.yml
---
roles:
  - name: geerlingguy.docker
    version: "7.1.0"
  - name: geerlingguy.nodejs
    version: "6.2.0"
  - src: https://github.com/dev-sec/ansible-collection-hardening
    name: dev-sec.os-hardening

collections:
  - name: community.general
    version: ">=8.0.0"
  - name: ansible.posix
    version: ">=1.5.4"
  - name: community.docker
    version: ">=3.4.0"
1
2
3
4
5
6
# Install all requirements
ansible-galaxy install -r requirements.yml
ansible-galaxy collection install -r requirements.yml

# Or both at once with newer Ansible
ansible-galaxy install -r requirements.yml --roles-path roles/

Variable Precedence and Ansible Vault

Variable Precedence

Ansible has 22 levels of variable precedence. The simplified version, from lowest to highest:

  1. Role defaults (roles/rolename/defaults/main.yml)
  2. Inventory group_vars/all
  3. Inventory group_vars/groupname (more specific groups win)
  4. Inventory host_vars/hostname
  5. Playbook group_vars/all
  6. Playbook group_vars/groupname
  7. Playbook host_vars/hostname
  8. Inventory host variables (inline in hosts.yml)
  9. Play vars (vars: in the play)
  10. Play vars_files
  11. Role vars (roles/rolename/vars/main.yml)
  12. Task vars (set_fact, register)
  13. Extra vars (-e on command line) — always wins

The practical takeaway: put defaults in role defaults/, put environment-specific values in group_vars/ or host_vars/, and use -e for one-time overrides. Never fight the precedence system — work with it.

Encrypting Secrets with Ansible Vault

Never store passwords, API keys, or private keys in plaintext in your playbooks or variable files. Ansible Vault encrypts values and files at rest.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
# Encrypt an entire file
ansible-vault encrypt inventory/group_vars/all/vault.yml

# Edit an encrypted file
ansible-vault edit inventory/group_vars/all/vault.yml

# Encrypt a single string (paste the output into a vars file)
ansible-vault encrypt_string 'super_secret_password' --name 'db_password'

# Decrypt for inspection
ansible-vault decrypt inventory/group_vars/all/vault.yml

# Re-encrypt with a new password
ansible-vault rekey inventory/group_vars/all/vault.yml

A common pattern is to have two files per group: one for plaintext vars and one for vault-encrypted vars:

inventory/
  group_vars/
    all/
      vars.yml        # plaintext variables
      vault.yml       # encrypted with ansible-vault encrypt
    databases/
      vars.yml
      vault.yml
1
2
3
4
5
6
# inventory/group_vars/databases/vars.yml
db_host: "192.168.1.20"
db_port: 5432
db_name: "appdb"
db_user: "appuser"
db_password: "{{ vault_db_password }}"   # reference the vault variable
1
2
# inventory/group_vars/databases/vault.yml (encrypted)
vault_db_password: "actual_secret_password_here"
1
2
3
4
5
6
7
8
# Run with vault password prompt
ansible-playbook site.yml --ask-vault-pass

# Run with password file (for automation/CI)
ansible-playbook site.yml --vault-password-file ~/.vault_pass

# Run with vault ID (for multiple vault passwords)
ansible-playbook site.yml --vault-id prod@~/.vault_pass_prod

Practical Homelab Playbooks

Example 1: Bootstrap a New Server

This playbook handles everything you’d do after a fresh OS install.

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
---
# playbooks/bootstrap.yml
# Usage: ansible-playbook playbooks/bootstrap.yml -i inventory/hosts.yml --limit newserver
- name: Bootstrap new server
  hosts: "{{ target | default('all') }}"
  become: true
  gather_facts: true

  vars:
    admin_username: "ops"
    admin_ssh_public_key: "{{ lookup('file', '~/.ssh/id_ed25519.pub') }}"
    sudo_passwordless: true
    ssh_port: 22
    allowed_ssh_networks:
      - "192.168.1.0/24"

  tasks:
    - name: Update apt cache and upgrade all packages
      ansible.builtin.apt:
        update_cache: true
        upgrade: dist
        autoremove: true
        autoclean: true

    - name: Install essential packages
      ansible.builtin.apt:
        name:
          - vim
          - curl
          - wget
          - git
          - htop
          - net-tools
          - unzip
          - ufw
          - fail2ban
          - chrony
          - acl
          - python3-pip
        state: present

    - name: Create admin user
      ansible.builtin.user:
        name: "{{ admin_username }}"
        groups:
          - sudo
          - adm
        shell: /bin/bash
        create_home: true
        state: present

    - name: Create .ssh directory for admin user
      ansible.builtin.file:
        path: "/home/{{ admin_username }}/.ssh"
        state: directory
        owner: "{{ admin_username }}"
        group: "{{ admin_username }}"
        mode: "0700"

    - name: Add authorized SSH key for admin user
      ansible.builtin.authorized_key:
        user: "{{ admin_username }}"
        key: "{{ admin_ssh_public_key }}"
        state: present

    - name: Configure passwordless sudo for admin user
      ansible.builtin.lineinfile:
        path: /etc/sudoers.d/{{ admin_username }}
        line: "{{ admin_username }} ALL=(ALL) NOPASSWD:ALL"
        create: true
        mode: "0440"
        validate: "/usr/sbin/visudo -cf %s"
      when: sudo_passwordless | bool

    - name: Harden SSH configuration
      ansible.builtin.blockinfile:
        path: /etc/ssh/sshd_config
        block: |
          PermitRootLogin no
          PasswordAuthentication no
          PubkeyAuthentication yes
          MaxAuthTries 3
          LoginGraceTime 30
          ClientAliveInterval 300
          ClientAliveCountMax 2
          X11Forwarding no
          AllowAgentForwarding no
          Protocol 2
        marker: "# {mark} ANSIBLE MANAGED BLOCK"
        backup: true
      notify: restart sshd

    - name: Configure UFW defaults
      community.general.ufw:
        default: deny
        direction: incoming

    - name: Allow SSH through firewall
      community.general.ufw:
        rule: allow
        port: "{{ ssh_port | string }}"
        proto: tcp
        src: "{{ item }}"
        comment: "SSH from trusted network"
      loop: "{{ allowed_ssh_networks }}"

    - name: Enable UFW
      community.general.ufw:
        state: enabled

    - name: Configure fail2ban for SSH
      ansible.builtin.copy:
        dest: /etc/fail2ban/jail.local
        content: |
          [DEFAULT]
          bantime  = 3600
          findtime = 600
          maxretry = 5
          backend  = systemd

          [sshd]
          enabled  = true
          port     = {{ ssh_port }}
          logpath  = %(sshd_log)s
        mode: "0644"
      notify: restart fail2ban

    - name: Set message of the day
      ansible.builtin.copy:
        dest: /etc/motd
        content: |
          =========================================
          Managed by Ansible — do not edit manually
          Host: {{ inventory_hostname }}
          Last provisioned: {{ ansible_date_time.date }}
          =========================================
        mode: "0644"

  handlers:
    - name: restart sshd
      ansible.builtin.service:
        name: ssh
        state: restarted

    - name: restart fail2ban
      ansible.builtin.service:
        name: fail2ban
        state: restarted

Example 2: Deploy a Docker Application

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
---
# playbooks/deploy_docker_app.yml
- name: Deploy Docker application
  hosts: "{{ target_hosts | default('webservers') }}"
  become: true
  gather_facts: true

  vars:
    app_name: "myapp"
    app_user: "deploy"
    app_base_dir: "/opt/apps"
    app_dir: "/opt/apps/{{ app_name }}"
    app_image: "registry.example.com/myapp"
    app_version: "{{ version | default('latest') }}"
    app_port: 8080
    app_env: "production"

    postgres_host: "{{ hostvars[groups['databases'][0]]['ansible_host'] }}"
    postgres_port: 5432
    postgres_db: "{{ app_name }}"
    postgres_user: "{{ vault_postgres_user }}"
    postgres_password: "{{ vault_postgres_password }}"

    redis_host: "localhost"
    redis_port: 6379

  vars_files:
    - ../inventory/group_vars/all/vault.yml

  tasks:
    - name: Install Docker dependencies
      ansible.builtin.apt:
        name:
          - ca-certificates
          - curl
          - gnupg
          - lsb-release
        state: present

    - name: Add Docker GPG key
      ansible.builtin.apt_key:
        url: https://download.docker.com/linux/ubuntu/gpg
        state: present

    - name: Add Docker repository
      ansible.builtin.apt_repository:
        repo: "deb [arch=amd64] https://download.docker.com/linux/ubuntu {{ ansible_distribution_release }} stable"
        state: present

    - name: Install Docker Engine
      ansible.builtin.apt:
        name:
          - docker-ce
          - docker-ce-cli
          - containerd.io
          - docker-compose-plugin
        state: present
        update_cache: true

    - name: Create deploy user
      ansible.builtin.user:
        name: "{{ app_user }}"
        groups:
          - docker
        shell: /bin/bash
        create_home: true
        state: present

    - name: Create application directories
      ansible.builtin.file:
        path: "{{ item }}"
        state: directory
        owner: "{{ app_user }}"
        group: "{{ app_user }}"
        mode: "0755"
      loop:
        - "{{ app_base_dir }}"
        - "{{ app_dir }}"
        - "{{ app_dir }}/data"
        - "{{ app_dir }}/logs"
        - "{{ app_dir }}/config"

    - name: Deploy docker-compose.yml
      ansible.builtin.template:
        src: templates/docker-compose.yml.j2
        dest: "{{ app_dir }}/docker-compose.yml"
        owner: "{{ app_user }}"
        group: "{{ app_user }}"
        mode: "0640"

    - name: Deploy application environment file
      ansible.builtin.template:
        src: templates/app.env.j2
        dest: "{{ app_dir }}/.env"
        owner: "{{ app_user }}"
        group: "{{ app_user }}"
        mode: "0600"

    - name: Pull latest Docker image
      community.docker.docker_image:
        name: "{{ app_image }}:{{ app_version }}"
        source: pull
        force_source: true

    - name: Run docker compose up
      community.docker.docker_compose_v2:
        project_src: "{{ app_dir }}"
        state: present
        pull: always
        recreate: auto

    - name: Wait for application to be healthy
      ansible.builtin.uri:
        url: "http://localhost:{{ app_port }}/health"
        method: GET
        status_code: 200
      register: health_check
      until: health_check.status == 200
      retries: 12
      delay: 10

    - name: Open application port in UFW
      community.general.ufw:
        rule: allow
        port: "{{ app_port | string }}"
        proto: tcp
        comment: "{{ app_name }} application port"

Example 3: Bulk OS Updates with Pre/Post Checks

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
---
# playbooks/update_all.yml
# Usage: ansible-playbook playbooks/update_all.yml --check  (dry run first!)
#        ansible-playbook playbooks/update_all.yml --limit webservers
- name: Bulk OS update with pre and post checks
  hosts: all
  become: true
  gather_facts: true
  serial: "30%"    # Update 30% of hosts at a time to avoid downtime

  tasks:
    # --- PRE-UPDATE CHECKS ---
    - name: Check disk space before update
      ansible.builtin.command:
        cmd: df -h /
      register: disk_before
      changed_when: false

    - name: Fail if disk space is critically low
      ansible.builtin.fail:
        msg: "Insufficient disk space. Available: {{ disk_before.stdout_lines[-1] }}"
      when: >
        disk_before.stdout_lines[-1].split()[4] | replace('%', '') | int > 90

    - name: Record uptime before update
      ansible.builtin.command:
        cmd: uptime -p
      register: uptime_before
      changed_when: false

    - name: Check running services before update
      ansible.builtin.service_facts:

    - name: Record package count before update
      ansible.builtin.command:
        cmd: dpkg -l | grep -c '^ii'
      register: pkg_count_before
      changed_when: false

    # --- PERFORM UPDATE ---
    - name: Update apt cache
      ansible.builtin.apt:
        update_cache: true

    - name: Show packages that will be upgraded
      ansible.builtin.command:
        cmd: apt list --upgradable
      register: upgradable_packages
      changed_when: false

    - name: Log upgradable packages
      ansible.builtin.debug:
        msg: "Packages to update on {{ inventory_hostname }}:\n{{ upgradable_packages.stdout }}"

    - name: Perform full dist-upgrade
      ansible.builtin.apt:
        upgrade: dist
        autoremove: true
        autoclean: true
      register: apt_upgrade_result

    # --- POST-UPDATE CHECKS ---
    - name: Check if reboot is required
      ansible.builtin.stat:
        path: /var/run/reboot-required
      register: reboot_required

    - name: Log reboot status
      ansible.builtin.debug:
        msg: "{{ inventory_hostname }} {{ 'REQUIRES REBOOT' if reboot_required.stat.exists else 'does not require reboot' }}"

    - name: Reboot if required (with wait)
      ansible.builtin.reboot:
        reboot_timeout: 300
        pre_reboot_delay: 5
        post_reboot_delay: 30
        msg: "Rebooting for kernel/package updates (Ansible managed)"
      when: reboot_required.stat.exists

    - name: Verify critical services are running after update
      ansible.builtin.service:
        name: "{{ item }}"
        state: started
      loop:
        - ssh
        - ufw
      failed_when: false
      register: service_check

    - name: Check disk space after update
      ansible.builtin.command:
        cmd: df -h /
      register: disk_after
      changed_when: false

    - name: Print update summary
      ansible.builtin.debug:
        msg: |
          Update summary for {{ inventory_hostname }}:
          - Packages upgraded: {{ apt_upgrade_result.stdout_lines | select('match', '^[0-9]+ upgraded') | list | first | default('unknown') }}
          - Disk before: {{ disk_before.stdout_lines[-1] }}
          - Disk after:  {{ disk_after.stdout_lines[-1] }}
          - Reboot was:  {{ 'required and performed' if reboot_required.stat.exists else 'not required' }}

Running and Managing Playbooks

Essential ansible-playbook Flags

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
# Basic run
ansible-playbook site.yml -i inventory/hosts.yml

# Dry run (no changes made)
ansible-playbook site.yml --check

# Show file diffs (what would change in config files)
ansible-playbook site.yml --check --diff

# Target a subset of hosts
ansible-playbook site.yml --limit webservers
ansible-playbook site.yml --limit "web01,web02"
ansible-playbook site.yml --limit "webservers:!web01"  # all webservers except web01

# Run only tasks with specific tags
ansible-playbook site.yml --tags "packages,ssh"
ansible-playbook site.yml --skip-tags "hardening"

# Start at a specific task (useful for resuming after failure)
ansible-playbook site.yml --start-at-task "Deploy nginx configuration"

# Override variables on the command line
ansible-playbook site.yml -e "nginx_port=8080 debug_mode=true"
ansible-playbook site.yml -e @overrides.yml

# Verbose output (add more v's for more detail)
ansible-playbook site.yml -v      # task results
ansible-playbook site.yml -vv     # file paths and task arguments
ansible-playbook site.yml -vvv    # SSH connection details
ansible-playbook site.yml -vvvv   # raw SSH debug output

ansible-lint: Catch Mistakes Before Running

1
2
3
pip install ansible-lint
ansible-lint site.yml
ansible-lint roles/

ansible-lint checks for common mistakes: using shell when a module exists, missing name fields on tasks, deprecated syntax, and YAML formatting issues. Run it in CI to prevent bad playbooks from reaching production.


Error Handling and Debugging

Ignore Errors (Use Sparingly)

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
- name: Check if legacy config exists
  ansible.builtin.stat:
    path: /etc/legacy/app.conf
  register: legacy_config
  ignore_errors: true

# Better alternative: use failed_when
- name: Check if service is running
  ansible.builtin.command:
    cmd: systemctl is-active myservice
  register: service_status
  failed_when: false   # never fail, but capture rc
  changed_when: false

Custom Failure and Change Conditions

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
- name: Check disk space
  ansible.builtin.command:
    cmd: df -BG / --output=avail
  register: disk_space
  changed_when: false
  failed_when: disk_space.stdout_lines[-1] | replace('G', '') | int < 5

- name: Run idempotent script
  ansible.builtin.command:
    cmd: /usr/local/bin/configure-app.sh
  register: configure_result
  changed_when: "'Configuration updated' in configure_result.stdout"
  failed_when:
    - configure_result.rc != 0
    - "'already configured' not in configure_result.stdout"

Block, Rescue, Always (Try/Catch/Finally)

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
- name: Deploy application with rollback on failure
  block:
    - name: Stop current application
      ansible.builtin.service:
        name: myapp
        state: stopped

    - name: Deploy new version
      ansible.builtin.unarchive:
        src: "myapp-{{ new_version }}.tar.gz"
        dest: /opt/myapp
        remote_src: false

    - name: Start application
      ansible.builtin.service:
        name: myapp
        state: started

    - name: Verify application started
      ansible.builtin.uri:
        url: "http://localhost:8080/health"
        status_code: 200
      retries: 5
      delay: 10

  rescue:
    - name: Log failure
      ansible.builtin.debug:
        msg: "Deployment failed, rolling back to version {{ current_version }}"

    - name: Rollback to previous version
      ansible.builtin.unarchive:
        src: "myapp-{{ current_version }}.tar.gz"
        dest: /opt/myapp
        remote_src: false

    - name: Start application on rollback version
      ansible.builtin.service:
        name: myapp
        state: started

    - name: Fail the play after rollback
      ansible.builtin.fail:
        msg: "Deployment failed and rollback was performed. Manual intervention required."

  always:
    - name: Send deployment notification
      ansible.builtin.debug:
        msg: "Deployment attempt completed for {{ inventory_hostname }}"

Execution Strategies

Ansible’s default linear strategy runs each task on all hosts before moving to the next task. The free strategy lets each host run through tasks as fast as it can, independent of other hosts.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
- name: Fast deployment (free strategy)
  hosts: webservers
  strategy: free    # each host runs independently
  tasks:
    - ...

- name: Debug failing tasks interactively
  hosts: problematic-host
  strategy: debug   # drops into interactive debugger on failure
  tasks:
    - ...

Tips and Best Practices

Use modules over shell/command. Every time you reach for shell or command, ask whether an Ansible module handles the task. Modules handle idempotency, error conditions, and return values correctly. shell gives you a raw bash session that’s almost never idempotent.

Always run --check --diff before a real run. Make this a habit. --check shows you what would change without making changes; --diff shows you the exact content of file modifications. This prevents surprises and builds confidence.

Pin your collection and role versions. In requirements.yml, always specify a version or version constraint. Community modules change; unpinned dependencies can break your playbooks on a Tuesday with no code changes on your end.

Keep playbooks small and focused. A playbook that does 50 things is harder to debug and harder to reuse than five playbooks that each do 10 things well. Use roles for reusable configuration units and a top-level site.yml that orchestrates the roles.

Use tags aggressively. Tags let you run subsets of tasks without modifying playbooks. Tag by category (packages, config, security, service) and by sensitivity (reboot, destructive). You’ll thank yourself when you need to re-run only the config template tasks.

Never store secrets in plaintext. Every password, API key, and private key in your inventory or vars files should be encrypted with Ansible Vault. Store the vault password in a password manager, not in the repo.

Test with Molecule. Molecule is the standard testing framework for Ansible roles. It spins up containers or VMs, runs your role, and verifies the results with built-in verifiers. Even a simple smoke test that confirms the role doesn’t crash on a fresh Ubuntu image is better than no testing. Install it with pip install molecule molecule-plugins[docker] and run molecule init scenario in your role directory to get started.

Commit your playbooks to git. Your Ansible code is infrastructure documentation. Every change should be a commit with a message explaining why. Pull requests, code review, and git history are free benefits that make your homelab — and your team’s production environment — significantly more resilient.


Putting It All Together

Here’s the recommended project layout for a real Ansible setup, whether it’s a homelab or a production environment:

ansible/
  ansible.cfg
  site.yml                    # top-level: imports all play files
  requirements.yml            # role and collection dependencies

  inventory/
    hosts.yml                 # or hosts.ini
    group_vars/
      all/
        vars.yml
        vault.yml             # ansible-vault encrypted
      webservers/
        vars.yml
      databases/
        vars.yml
        vault.yml
    host_vars/
      db01.yml

  playbooks/
    bootstrap.yml
    update_all.yml
    deploy_app.yml

  roles/
    common/
    nginx/
    docker/
    postgres/

  templates/                  # shared templates not in a role
  files/                      # shared static files not in a role
  vars/                       # shared variable files not in a role

Start with the bootstrap.yml and common role. Get those working and commit them. Then build out roles for each service you run. Within a few weeks you’ll have a complete, repeatable description of your entire infrastructure in version control — and spinning up a new server will take minutes instead of an hour of careful, error-prone manual work.

Ansible rewards the time you invest in it. The first playbook is slow to write; the tenth is fast. By the hundredth task you’ve automated, you’ll find yourself reaching for Ansible even for one-time changes, because you know the result will be documented, repeatable, and right.

Comments