Mastering Linux PAM: Hardening Passwords and Stopping Brute-force Attacks for Your Server

Why do you need to dive deep into Linux PAM?

My server used to be hit by constant SSH brute-force attacks. As a result, I had to stay up all night to handle the incident. Since then, I always prioritize setting up PAM as soon as a project is initialized. If you just install the OS and leave it as is, botnets only take a few minutes to crack weak passwords using common dictionaries.

PAM (Pluggable Authentication Modules) is like a flexible “gatekeeper.” It allows us to change authentication rules without touching the application code. Instead of hoping users set strong passwords themselves, we force the system to reject weak ones. Meanwhile, auto-lock mechanisms and MFA create a solid layer of protection.

To test the system, you’ll need high-complexity sample passwords. I often use ToolCraft’s Password Generator. This tool runs entirely on the client-side (browser). Therefore, passwords are never sent to a server, ensuring absolute safety for techies.

1. Enforcing password complexity with pam_pwquality

By default, Linux is quite “lenient” with user passwords. To tighten it up, we need the pam_pwquality module. It checks complexity the moment you type the passwd command.

Installing the library

On Ubuntu or Debian, run the following command:

sudo apt update && sudo apt install libpam-pwquality -y

Detailed configuration

All main settings are located in /etc/security/pwquality.conf. However, to apply them system-wide, open this file:

sudo nano /etc/pam.d/common-password

Find the line pam_pwquality.so and replace it with the following strict configuration:

password requisite pam_pwquality.so retry=3 minlen=12 difok=3 ucredit=-1 lcredit=-1 dcredit=-1 ocredit=-1 enforce_for_root

Here is the meaning of the parameters I usually apply:

• retry=3: Rejected after 3 failed attempts during a password change session.
• minlen=12: Password must be at least 12 characters long.
• difok=3: Must differ from the old password by at least 3 characters.
• ucredit, lcredit, dcredit, ocredit=-1: Mandatory inclusion of uppercase, lowercase, numbers, and special characters.
• enforce_for_root: Applies to the root account as well to prevent complacency.

2. Stopping Brute-force with pam_faillock

Once you have strong passwords, the next step is to block automated password-guessing attacks. pam_faillock is a modern solution, a perfect replacement for the outdated pam_tally2.

Open the authentication configuration file:

sudo nano /etc/pam.d/common-auth

Insert the following lines at the very beginning of the auth configuration section:

auth required pam_faillock.so preauth silent audit deny=5 unlock_time=900
auth [default=die] pam_faillock.so authfail audit deny=5 unlock_time=900

And add this line to the end of /etc/pam.d/common-account (or the end of common-auth depending on the distro):

account required pam_faillock.so

This configuration is extremely effective. If a user enters the wrong password 5 times (deny=5), the account is locked immediately for 15 minutes. The bot’s guessing speed will be significantly slowed down, causing them to get discouraged and give up.

You can check the list of locked accounts with the command:

sudo faillock --user <username>

If a colleague accidentally locks themselves out, use the reset command to rescue them:

sudo faillock --user <username> --reset

3. Adding an MFA Layer for SSH

No matter how strong a password is, there’s always a risk of exposure through phishing or keyloggers. Adding Multi-Factor Authentication (MFA) is the final layer of protection. Google Authenticator is the quickest choice.

Installing the module is very simple:

sudo apt install libpam-google-authenticator -y

Each user needs to run the google-authenticator command to scan the QR code. Then, activate it in PAM. Open /etc/pam.d/sshd and add the following line to the end:

auth required pam_google_authenticator.so

Don’t forget to edit /etc/ssh/sshd_config. You need to change KbdInteractiveAuthentication to yes so that SSH prompts for an OTP code.

4. Monitoring and Support Tools

A vital note: Never close your current SSH session right after modifying PAM. Open a new tab to test the login. If there’s an error, you still have the old session to fix it, avoiding being locked out of the server.

Regularly monitoring logs is a good habit for sysadmins:

sudo tail -f /var/log/auth.log

When you need to configure a firewall to block attacking IP ranges, I often use ToolCraft’s Subnet Calculator. It helps calculate CIDR extremely fast, preventing mistakes when subnetting for complex security rules.

Additionally, to ensure PAM configuration files aren’t unauthorizedly modified, you should create checksums. The Hash Generator tool will help you create hashes (SHA-256) for files in /etc/pam.d/ for periodic comparison.

Conclusion

Configuring PAM requires meticulousness because a small mistake can isolate the system. However, once mastered, your server will become a true fortress. Combine strong passwords, auto-locking mechanisms, and MFA for a peaceful night’s sleep.