Context and Why Automate Server Hardening?
In the IT field, security is always paramount. However, early on, I was quite complacent. Until one fateful night, around 2 AM, my phone suddenly rang non-stop. Checking the logs, I was horrified to see the production server under a relentless SSH brute-force attack. With my heart pounding, I frantically typed commands to block IPs, change ports, and reset configurations. I stayed up all night just to deal with an incident that could have been prevented much earlier.
After that memorable experience, I realized that manually configuring security on each server carries significant risks. A small error, whether it’s forgetting a step, mistyping a command, or simply a minor difference between servers, can lead to serious consequences. With systems reaching dozens, or even hundreds of servers, maintaining consistency and ensuring all of them are properly hardened is almost impossible.
That’s when I started looking into Ansible. This tool not only helped me deploy configurations quickly but also ensured that every server – from brand new to operational – had the same security configuration. This process is automated and idempotent, giving me more peace of mind with a system robustly protected against basic vulnerabilities, instead of having to stay up all night as before.
Ansible Installation: Beginning the Automation Journey
To begin, you need a machine to act as the Ansible Controller and Managed Nodes (the servers that need hardening). A key feature of Ansible is that it doesn’t require an agent to be installed on the Managed Nodes; instead, it uses SSH to connect and execute commands, which is very convenient and secure.
1. Install Ansible Controller
On the controller machine, installing Ansible is quite simple. You just need to install Python and pip, then use pip to install Ansible. For example, on Ubuntu, you execute the following commands:
sudo apt update
sudo apt install python3 python3-pip -y
pip3 install ansible
Confirm Ansible was installed successfully:
ansible --version
2. Prepare Managed Nodes
The servers requiring hardening only need Python (usually pre-installed) and a running SSH server. Most importantly, you need to set up key-based SSH authentication from the Controller to the Managed Nodes. This allows Ansible to connect without manual password entry.
ssh-keygen -t rsa -b 4096
ssh-copy-id user@your_server_ip
3. Create Inventory File
The inventory (or hosts) file is where you define the servers Ansible will manage. It’s recommended to organize servers into groups for easier management and configuration application.
[web_servers]
web1.example.com
web2.example.com
[db_servers]
db1.example.com
[all:vars]
ansible_python_interpreter=/usr/bin/python3
ansible_user=your_ssh_username
Here, your_ssh_username is the username you will use to SSH into the servers.
Detailed Configuration: Step-by-Step Hardening Playbook
This is the most crucial part: writing playbooks to automate the hardening process. Typically, I will create a hardening.yml file and call specific roles or tasks within it.
1. SSH Security: The First Gateway
After that night’s incident, SSH security became my top priority. The playbook below will configure basic SSH settings:
- Disable
rootaccount login. - Disable password authentication, only allowing SSH key usage.
- Change the default SSH port (recommended but optional).
# roles/ssh_hardening/tasks/main.yml
- name: Configure SSH Server
ansible.builtin.lineinfile:
path: /etc/ssh/sshd_config
regexp: '^(PermitRootLogin|PasswordAuthentication|Port)'
line: '{{ item.line }}'
state: present
validate: '/usr/sbin/sshd -t'
loop:
- { regexp: '^PermitRootLogin', line: 'PermitRootLogin no' } # Disable root login
- { regexp: '^PasswordAuthentication', line: 'PasswordAuthentication no' } # Disable password authentication
- { regexp: '^Port', line: 'Port 2222' } # Replace 2222 with your desired port
notify: Restart sshd
- name: Ensure only allowed users can SSH
ansible.builtin.lineinfile:
path: /etc/ssh/sshd_config
regexp: '^AllowUsers'
line: 'AllowUsers your_ssh_username another_user'
state: present
validate: '/usr/sbin/sshd -t'
notify: Restart sshd
# handlers/main.yml (for ssh_hardening role)
- name: Restart sshd
ansible.builtin.systemd:
name: sshd
state: restarted
enabled: yes
Important Note: Always ensure you have a valid SSH key and user to log in to the new port before executing this playbook. Otherwise, you might lock yourself out of the server!
2. Configure Firewall with UFW
The firewall is the first line of defense. On Ubuntu, I typically use UFW (Uncomplicated Firewall) due to its ease of use. This playbook will activate UFW, allow necessary ports, and block all unwanted traffic.
# roles/firewall/tasks/main.yml
- name: Install UFW
ansible.builtin.apt:
name: ufw
state: present
- name: Allow SSH on custom port (e.g. 2222)
community.general.ufw:
rule: allow
port: '2222'
proto: tcp
- name: Allow HTTP (port 80)
community.general.ufw:
rule: allow
port: '80'
proto: tcp
- name: Allow HTTPS (port 443)
community.general.ufw:
rule: allow
port: '443'
proto: tcp
- name: Set default policy to deny all incoming traffic
community.general.ufw:
state: enabled
policy: deny
direction: incoming
- name: Enable UFW
community.general.ufw:
state: enabled
3. Manage Updates and Software Packages
Keeping the operating system and software packages updated is a basic but extremely critical hardening step. This helps prevent patched security vulnerabilities, which if overlooked, could become attack targets.
# roles/system_updates/tasks/main.yml
- name: Update package list
ansible.builtin.apt:
update_cache: yes
- name: Upgrade all packages to the latest version
ansible.builtin.apt:
upgrade: dist
autoclean: yes
autoremove: yes
- name: Install and configure automatic updates (unattended-upgrades)
ansible.builtin.apt:
name: unattended-upgrades
state: present
- name: Enable automatic updates
ansible.builtin.lineinfile:
path: /etc/apt/apt.conf.d/20auto-upgrades
regexp: 'APT::Periodic::Unattended-Upgrade "1"'
line: 'APT::Periodic::Unattended-Upgrade "1";'
state: present
4. User and Permissions Management
The Principle of Least Privilege is the guiding principle here. I only create necessary accounts and grant sudo privileges cautiously.
# roles/user_management/tasks/main.yml
- name: Create new user if not exists
ansible.builtin.user:
name: mynewadmin
comment: 'Admin User'
group: sudo
shell: /bin/bash
generate_ssh_key: yes
ssh_key_type: rsa
password: '{{ "my_strong_password" | password_hash("sha512") }}'
- name: Add SSH public key for new user
ansible.builtin.authorized_key:
user: mynewadmin
state: present
key: "{{ lookup('file', '~/.ssh/id_rsa.pub') }}"
- name: Ensure unnecessary old user is removed (replace the_old_user_to_remove)
ansible.builtin.user:
name: the_old_user_to_remove
state: absent
remove: yes
ignore_errors: yes # Ignore error if user does not exist
- name: Configure sudoers for sudo group (no password needed when using sudo)
ansible.builtin.lineinfile:
path: /etc/sudoers.d/90-nopasswd-sudo
line: '%sudo ALL=(ALL) NOPASSWD:ALL'
state: present
create: yes
validate: '/usr/sbin/visudo -cf %s'
Note: Replace mynewadmin, my_strong_password, and the_old_user_to_remove with your specific information. Using password_hash helps store passwords more securely in the playbook.
5. Main Playbook Integration (main.yml)
Finally, you will create a main hardening.yml file to call these roles:
# hardening.yml
---
- name: Execute Linux server hardening
hosts: web_servers, db_servers # Or hosts: all
become: yes # Run commands with sudo privileges
roles:
- ssh_hardening
- firewall
- system_updates
- user_management
Testing and Monitoring: Always in Control
Deployment completion doesn’t mean the work is finished. You must always test and monitor to ensure everything is functioning as expected.
1. Pre-Deployment Check (Dry Run)
Before running the actual playbook, I always do a dry run with the --check flag to see what changes Ansible would make without applying them to the server. This step has saved me from many unintended incidents.
ansible-playbook -i inventory hardening.yml --check
After reviewing the output and confirming everything looks good, I then run the actual playbook:
ansible-playbook -i inventory hardening.yml
2. Post-Deployment Verification
After the playbook finishes running, I will SSH into the server to confirm the changes. For example:
- Check SSH configuration:
sudo cat /etc/ssh/sshd_config | grep -E 'PermitRootLogin|PasswordAuthentication|Port|AllowUsers' - Check UFW status:
sudo ufw status verbose - Check user:
id mynewadmin
Once the server is operational, you can try SSHing in with the new user and port. Make sure the old user (or root) can no longer log in. I once made the mistake of forgetting to open the new SSH port on the firewall before changing the SSH daemon port, which resulted in having to use the console to fix the issue.
3. Regular Monitoring and Auditing
Hardening is not a one-time task. The security landscape is constantly evolving, and you need to ensure configurations remain up to standard. I typically schedule the hardening playbook to run periodically (e.g., monthly) to maintain consistency. Ansible is idempotent, so re-running it will not cause errors but only apply changes if there are any.
Furthermore, I also integrate log monitoring tools (like ELK stack or Prometheus with Grafana) to track security events. These events include failed logins, unusual access, or sudo actions. This helps me detect early signs of attacks or suspicious activity. I also combine this with using Lynis for periodic system auditing, and sometimes I even automate running Lynis via Ansible to easily monitor reports.
# Example task to run Lynis (add to system_auditing role if available)
- name: Install Lynis
ansible.builtin.apt:
name: lynis
state: present
- name: Run Lynis audit and save results
ansible.builtin.command: lynis audit system --quick --report-file /var/log/lynis_report.txt
args:
creates: /var/log/lynis_report.txt # Only run if report file doesn't exist or if you want to rerun
Automating Linux server hardening with Ansible has changed the way I work and brought immense peace of mind. From sleepless nights due to server attacks, I can now be confident that my systems are consistently and effectively protected. You should also start this automation journey as soon as possible!
