How to Integrate Ansible with AWS EC2 and Manage Instances Dynamically

-

If you're working with AWS EC2 and want to automate your infrastructure using Ansible, this guide shows how to set up a dynamic inventory system. This means Ansible will automatically detect your EC2 instances based on tags—no need to maintain static inventory files!

Step 1: Install Required Dependencies

On your Ansible control node, make sure you have Python and pip installed. Then install the AWS SDK for Python (boto3), which is required for Ansible to communicate with AWS APIs.

sudo yum install python3 -y
pip3 install boto3

Also, configure AWS CLI with appropriate credentials:

aws configure

Why this matters: Boto3 and AWS CLI let Ansible fetch details about your EC2 infrastructure dynamically, based on filters like tags and regions.

Step 2: Create the Dynamic Inventory Configuration

mkdir -p /home/ec2-user
cd /home/ec2-user

Create a file called aws_ec2.yml with the following content:

plugin: amazon.aws.aws_ec2
regions:
  - ap-south-1
filters:
  tag:Env: Test
hostnames:
  - tag:Name
keyed_groups:
  - key: tags['Env']
    prefix: Env
compose:
  ansible_host: public_ip_address
ansible_user: ec2-user
ansible_ssh_private_key_file: /home/ec2-user/New-key.pem

Key Points:

  • plugin: Enables the AWS EC2 dynamic inventory plugin.
  • regions: Specify which AWS region to look into.
  • filters: Restrict to EC2 instances tagged with Env=Test.
  • hostnames: Uses the Name tag for each host's name in Ansible.
  • keyed_groups: Automatically groups hosts under Env_Test.
  • compose: Uses the instance’s public IP address as the target for SSH.

Make sure the SSH key file matches the key associated with your EC2 instances.

⚙️ Step 3: Install or Update the Ansible AWS Collection

ansible-galaxy collection install amazon.aws --upgrade

This collection includes the dynamic inventory plugin and AWS modules that Ansible needs.

Step 4: Enable Plugin in Ansible Configuration

Edit or create /home/ec2-user/ansible.cfg and add:

[inventory]
enable_plugins = aws_ec2

[defaults]
ansible_user = ec2-user

Why this matters: This tells Ansible to use the dynamic inventory plugin when you specify -i aws_ec2.yml.

Step 5: Test Your Dynamic Inventory

ansible-inventory -i /home/ec2-user/aws_ec2.yml --list

This command outputs a JSON structure of all EC2 instances that match your filter. It's useful to confirm that your instances are being discovered correctly.

Step 6: Create Your Ansible Playbook

sudo mkdir -p /var/playbook
sudo touch /var/playbook/patch.yml
sudo chown ec2-user:ec2-user /var/playbook/patch.yml

Now add the following content to /var/playbook/patch.yml:

- name: Retrieve OS version
  hosts: Env_Test
  become: yes
  gather_facts: yes
  tasks:
    - name: Display OS distribution and version
      debug:
        msg: "The operating system is {{ ansible_distribution }} version {{ ansible_distribution_major_version }}"

    - name: Retrieve system uptime
      command: uptime -p
      register: uptime_result

    - name: Display system uptime
      debug:
        msg: "System uptime: {{ uptime_result.stdout }}"

    - name: Display private IP address
      debug:
        msg: "Private IP address: {{ ansible_default_ipv4.address }}"

Explanation: This playbook collects system facts, uptime, and private IP from all EC2 instances tagged as Env=Test.

Step 7: Upload SSH Key and Set Environment Variable

If your key is in .ppk format, convert it to .pem using PuTTYgen and upload to the Ansible master node:

scp New-key.pem ec2-user@your-ansible-host:/home/ec2-user/
chmod 400 /home/ec2-user/New-key.pem

Disable SSH host key checking (optional):

echo "export ANSIBLE_HOST_KEY_CHECKING=False" >> ~/.bashrc
source ~/.bashrc

Why: This prevents Ansible from halting if the host key is not already known.

Step 8: Run the Playbook

ansible-playbook -i /home/ec2-user/aws_ec2.yml /var/playbook/patch.yml \
--private-key=/home/ec2-user/New-key.pem --check

This runs the playbook in check mode, showing what would happen without making changes.

Sample Output:

PLAY [Retrieve OS version] ****************************************************

TASK [Gathering Facts]
ok: [Terraform-EC2-0]

TASK [Display OS distribution and version]
ok: [Terraform-EC2-0] => {
    "msg": "The operating system is Amazon version 2023"
}

TASK [Display private IP address]
ok: [Terraform-EC2-0] => {
    "msg": "Private IP address: 172.31.2.184"
}

Conclusion

By using Ansible's dynamic inventory with the AWS EC2 plugin, you eliminate the need to manually track and update your hosts. It’s a scalable and efficient way to manage cloud infrastructure based on tags, environment, or region.

Best Practices:

  • Use consistent tagging in AWS (e.g., Env, Project, Owner).
  • Secure your SSH keys and restrict access to the Ansible control node.
  • Test playbooks in --check mode before actual execution.

Feel free to share this post if it helped you simplify your EC2 management with Ansible!

Comments

Post a Comment

Popular Posts

Why SELinux Is Blocking Your Service (and How to Fix It)

-
Troubleshooting SELinux-Related Issues in Linux Running services with SELinux enabled can sometimes lead to unexpected denials or blocked actions—especially if SELinux was previously disabled or the service is configured in a non-standard way. In most cases, SELinux issues are the result of misconfigured contexts, policies, or port bindings. Here's a step-by-step approach to diagnose and fix SELinux-related problems while keeping your system secure. Check SELinux Status and Mode Start by verifying that SELinux is active on your system: # sestatus Look for: SELinux status: enabled Current mode: enforcing or permissive To test if SELinux is causing a problem, temporarily switch to permissive mode: # setenforce 0 # Logs denials but doesn’t block actions # setenforce 1 # Reverts to enforcing mode If the issue disappears in permissive mode, SELinux is likely the root cause. Look for SELinux Denials i...
Read more

Puppet Code Deploy Troubleshooting & Resolution Guide

-
Initial Problem I was setting up Puppet Enterprise and trying to push code using the trusted puppet code deploy command. But instead of a clean deployment, I got this surprise: puppet code deploy --dry-run And boom — this lovely error shows up: [POST /deploys][500] Errors while collecting a list of environments to deploy (exit code: 1). ERROR -> failed to stat '/var/opt/lib/pe-puppet/.gitconfig' So... what gives? Turns out this issue is part permissions, part SSH trust, and part “who ran this command and as which user.” But don’t worry — I’ll walk through all root causes and their real-world fixes step by step. Root Cause #1: Directory Permission What Happened: The directory /var/opt/lib was locked down with drwx------ (700) and owned by root . User pe-puppet couldn’t even step inside it. Since r10k runs as pe-puppet , it failed trying to read .gitconfig . The Fix: chmod 711 /var/opt/lib Why: This lets pe-puppet enter the folder ...
Read more

Fix: SSH Permission Denied Issue | Real Solution

-
1. Verify SSH Command and Key Usage Use the correct private key with -i : ssh -i ~/.ssh/your_private_key user@host Ensure the key is not passphrase-protected (or enter the passphrase correctly). 2. Check Key Generation and Deployment Regenerate Keys (if unsure): ssh-keygen -t ed25519 -C "your_email@example.com" # Prefer ed25519 Ed25519 uses elliptic curve cryptography (Edwards-curve Digital Signature Algorithm) and provides 128-bit security (equivalent to RSA-3072 or RSA-4096) with a 256-bit key. Copy the Public Key to the Server: ssh-copy-id -i ~/.ssh/your_public_key user@host If ssh-copy-id isn’t available, manually append the public key to ~/.ssh/authorized_keys on the server. 3. Fix File/Directory Permissions On the Server: chmod 700 ~/.ssh chmod 600 ~/.ssh/authorized_keys chmod go-w ~ # Home directory should not be world-writable On the Client (Local Machine): chmod 600 ~/.ssh/your_private_key 4. Check SSH Server Configuration Edit /etc/ssh/s...
Read more