Why Might Your Server Be Under Secret Surveillance?
Imagine this: your server’s CPU suddenly spikes to 80-90%, and network bandwidth is consumed at tens of Mbps despite having few users. You quickly type the top or htop command to check, but everything looks “clean.” If you encounter this situation and find nothing unusual in your Linux security audit logs, there is a high chance your server is infected with a Rootkit.
Rootkits don’t cause “loud” damage like viruses. Their ultimate goal is invisibility. They can manipulate basic system commands like ls (list files), ps (view processes), or netstat to hide themselves. When this happens, the tools you trust most are actually “lying” to you about the server’s true state.
To deal with these hidden threats, I often use the “dynamic duo” of Chkrootkit and Rkhunter. These are lightweight, open-source tools that are essential for hardening Linux server and unmasking malware lurking deep within the operating system kernel.
What do Rootkits actually do on a system?
As soon as root access is gained, a hacker will install a Rootkit to maintain a backdoor and use log cleaners to wipe their tracks. Rootkits directly interfere with the Kernel layer or system libraries (LKM – Loadable Kernel Modules). Detecting them with the naked eye is nearly impossible. You need specialized scanning tools, such as those used for a Linux server security audit, to cross-reference system data with thousands of known malware signatures.
Chkrootkit: A Quick Check in a Heartbeat
Chkrootkit (Check Rootkit) is essentially a lightweight set of shell scripts. It performs quick scans of executable files to find signs of over 70 common Rootkits. The biggest advantage is speed; you can get preliminary results in just seconds.
Installing Chkrootkit
On Debian or Ubuntu, you can install it very quickly via apt:
sudo apt update
sudo apt install chkrootkit -yFor CentOS/RHEL distributions, enable the EPEL repository before installing:
sudo yum install epel-release -y
sudo yum install chkrootkit -yReal-world Usage
To start a full system scan, run the command:
sudo chkrootkitThe screen will display a series of tests. If everything is fine, you will see the line not infected. Conversely, if the word INFECTED appears in red, it’s a red alert for your server.
Pro tip: To avoid straining your eyes looking at hundreds of lines of text, filter for critical warnings directly:
sudo chkrootkit | grep INFECTEDRkhunter: The “Detective” Inspecting Every System Fingerprint
While Chkrootkit is a quick test, Rkhunter (Rootkit Hunter) is like an in-depth forensic examination. Rkhunter doesn’t just look for malware signatures; it performs system file integrity monitoring by hashing system files using the SHA-1 algorithm to compare them against a trusted database.
Installing Rkhunter
sudo apt install rkhunter -yEstablishing an Initial Security Baseline
Before running it for the first time, you must update the data so Rkhunter has a standard baseline:
- Update malware database:
sudo rkhunter --update - Capture current system file state:
sudo rkhunter --propupd
The --propupd step is extremely important. It creates a snapshot of clean system files. Later, if a hacker silently replaces /bin/ls with a modified version, Rkhunter will detect it immediately because the hash has changed.
Security Tip: When setting up a new server, I always use the password generator at toolcraft.app/en/tools/security/password-generator. This tool runs entirely in the browser, making it very secure and helping you create passwords strong enough to block brute-force attacks right from the start, complementing other measures like enhancing SSH security.
Performing a Deep Scan
Use the following command to start the scan:
sudo rkhunter --check --skThe --sk (skip-keypress) parameter allows the scan to run automatically without requiring you to press Enter repeatedly. You can review the detailed report anytime at the file /var/log/rkhunter.log.
Identifying False Positives
Rkhunter is very sensitive; sometimes it will report a Warning even if you just ran apt upgrade. Updating software versions changes the file hashes, causing Rkhunter to become suspicious.
If you are certain the change is safe, update the comparison baseline again:
sudo rkhunter --propupdAutomating Security with Cronjobs
Instead of remembering to type commands every week, let the server do it at 3 AM while everyone is asleep. You only need to receive an email if there’s an issue.
Create a script file at /etc/cron.daily/rkhunter-scan:
#!/bin/bash
/usr/bin/rkhunter --versioncheck --nocolor --quiet
/usr/bin/rkhunter --update --nocolor --quiet
/usr/bin/rkhunter --check --cronjob --report-warnings-only | mail -s "Daily Rootkit Warning - $(hostname)" [email protected]Remember to grant execution permissions to the script: sudo chmod +x /etc/cron.daily/rkhunter-scan.
Concluding Thoughts from Real-world Experience
No matter how powerful they are, Rkhunter and Chkrootkit are just supporting tools. Security is an endless race. If a Rootkit has infected the deepest levels of the Kernel, it might still bypass these scanners.
My advice for you:
- Proactive Defense: Always install security patches as soon as possible.
- Regular Scans: Set up daily automated scans to detect unusual changes as early as possible.
- Know When to Give Up: If the server has too many serious warnings, don’t try to “cleanse” it. The safest way is to back up clean data and reinstall the operating system completely.
Hopefully, this article provides you with more weapons to protect your “brainchild” from underworld attacks.
