Python and Linux: Useful Sysadmin Scripts for Task Automation
Hello there! You probably know that a sysadmin’s job requires not only system knowledge but also the skill to solve repetitive daily problems. From checking service status, managing logs, to backing up data, everything can become a burden if done manually.
The Real Problem: Repetitive Tasks and Manual Risks
When I first started my career, I spent a lot of time on repetitive daily tasks. These included checking disk usage, looking for errors in web server logs, or even creating new user accounts. Initially, it was fine, but as the system grew and the number of servers increased to dozens, these seemingly simple tasks would consume my entire morning.
Not to mention, manual work always carries risks. Once, I accidentally typed the wrong command to delete files while cleaning up logs, almost losing critical data. The feeling at that moment was truly “heart-pounding, legs shaking.” I realized that no matter how careful one is, humans can still make mistakes. And in a production environment, those mistakes can have very serious consequences.
Analyzing the Cause: Limitations of Traditional Tools
I’ve tried using Bash scripts quite a bit. For simple tasks, Bash works very well. For example, a few lines of script to restart a service or check uptime is quick and efficient. But when it comes to handling more complex logic, interacting with APIs, or managing data structures, Bash starts to reveal its limitations.
Writing a Bash script to analyze logs and send alerts via Telegram? That’s tricky. Or generating periodic system performance reports and exporting them in JSON format? That would certainly be very convoluted and difficult to maintain. I often spent a lot of time debugging long Bash scripts, because their syntax can sometimes be quite “unique” and hard to read if you’re not used to it.
Solutions: From Manual to Automation
There are many ways to solve this problem.
-
Manual: Still the traditional way, typing commands directly in the terminal. The advantage is that it’s quick for one-off tasks, but as I said, it’s unsustainable and risky when repeated.
-
Bash script: This is a big step forward. It helps automate basic command sequences, reducing errors from mistyping. However, it has limitations regarding data structures, complex error handling capabilities, and scalability.
-
Configuration Management Tools: Such as Ansible, Puppet, Chef. These tools are very powerful for managing hundreds, even thousands of servers. But sometimes, to solve a specific problem on a few machines, installing and configuring them becomes overly complex. We need a “lighter” solution, yet powerful enough to tackle complex problems that Bash scripts struggle with.
-
Python script: This is the star of this article. Python offers a perfect balance between the flexibility of scripting and the power of a general-purpose programming language.
The Best Way: Python – The “Swiss Army Knife” for Sysadmins
I see Python as a “Swiss Army knife.” It can do many things, from simple to complex tasks, and is especially friendly for those new to programming. With Python, I can write scripts to:
- Read, analyze, filter, and aggregate data from log files.
- Check service status, system resources (CPU, RAM, Disk).
- Automate data backup and recovery.
- Manage users, groups, and permissions.
- Interact with cloud service APIs, monitoring tools.
Why is Python so powerful?
Python boasts an incredibly rich library ecosystem. Want to handle files? There’s os, shutil. Want to work with JSON, YAML? There’s json, pyyaml. Want to send HTTP requests? requests is an excellent choice. Want to interact with the operating system? subprocess allows you to run shell commands easily.
Python’s syntax is also very clear and easy to read, helping me make fewer mistakes and easily maintain scripts later on.
Practical Examples: Useful Python Scripts
1. Disk Usage Check Script
One of the most basic sysadmin tasks is checking disk usage to prevent services from crashing due to running out of space. The script below will check the free space of a given path and warn if it falls below a certain threshold.
import shutil
def check_disk_usage(path='/', threshold_percent=80):
total, used, free = shutil.disk_usage(path)
used_percent = (used / total) * 100
free_gb = free / (1024**3) # Convert bytes to GB
if used_percent > threshold_percent:
print(f"WARNING: Disk usage on '{path}' is {used_percent:.2f}% ({free_gb:.2f} GB free). Exceeded threshold {threshold_percent}%.")
else:
print(f"Disk on '{path}' is {used_percent:.2f}% ({free_gb:.2f} GB free). Everything is fine.")
if __name__ == "__main__":
# Check root directory
check_disk_usage('/')
# You can check other paths
# check_disk_usage('/var/log', 90)
To run this script on Linux, you need to have Python installed (usually already available) and run it with the command:
python your_script_name.py
2. Basic Log Analysis Script
Logs are a treasure trove of system information, but sometimes too many logs make finding errors difficult. This script will help you filter out log lines containing the keyword “ERROR” in a given log file.
import re
def analyze_log_errors(log_file_path, keyword="ERROR"):
errors_found = []
try:
with open(log_file_path, 'r', encoding='utf-8') as f:
for line_num, line in enumerate(f, 1):
if keyword in line.upper(): # Case-insensitive search
errors_found.append(f"Line {line_num}: {line.strip()}")
if errors_found:
print(f"Found errors ({keyword}) in file '{log_file_path}':")
for error in errors_found:
print(error)
else:
print(f"No errors ({keyword}) found in file '{log_file_path}'.")
except FileNotFoundError:
print(f"Error: File '{log_file_path}' not found.")
except Exception as e:
print(f"An error occurred while reading the file: {e}")
if __name__ == "__main__":
# Replace 'path/to/your/logfile.log' with your log file path
analyze_log_errors('/var/log/syslog')
# You can also search for other keywords
# analyze_log_errors('/var/log/apache2/error.log', 'WARNING')
A small note for you: when working with regex (regular expressions) in Python for more complex log filtering, I often use the regex tester at toolcraft.app/en/tools/developer/regex-tester. It runs directly in the browser, no installation needed, helping me quickly test patterns before putting them into Python code. Very convenient!
3. Automatic Directory Backup Script
Backup is an indispensable part of system administration. This simple script will compress a directory and save it to a destination directory, along with a timestamp.
import shutil
import datetime
import os
def backup_directory(source_dir, destination_dir):
try:
if not os.path.exists(destination_dir):
os.makedirs(destination_dir)
timestamp = datetime.datetime.now().strftime("%Y%m%d_%H%M%S")
backup_file_name = f"{os.path.basename(source_dir)}_backup_{timestamp}"
archive_path = shutil.make_archive(
os.path.join(destination_dir, backup_file_name),
'zip', # You can change to 'tar' or 'gztar'
source_dir
)
print(f"Successfully backed up '{source_dir}' to '{archive_path}'")
except Exception as e:
print(f"Error backing up directory '{source_dir}': {e}")
if __name__ == "main__":
# Replace with the directory you want to back up and the destination directory
source_to_backup = '/home/ubuntu/important_data'
destination_for_backup = '/var/backups'
backup_directory(source_to_backup, destination_for_backup)
# Ensure source and destination directories exist for testing
# Example of creating a dummy directory for testing:
# os.makedirs(source_to_backup, exist_ok=True)
# with open(os.path.join(source_to_backup, "test.txt"), "w") as f:
# f.write("This is a test file.")
For this script to work, you need to create the /home/ubuntu/important_data directory (or any directory you want to back up) and /var/backups (the backup destination directory) beforehand.
4. Service Status Check Script (Linux)
Checking whether a service (e.g., Nginx, Apache, MySQL) is running is a frequent task. We can use Python’s subprocess module to call the systemctl command.
import subprocess
def check_service_status(service_name):
try:
# Run systemctl is-active command to check service status
result = subprocess.run(
['systemctl', 'is-active', service_name],
capture_output=True,
text=True,
check=True # Will raise CalledProcessError if the command returns a non-zero exit code
)
status = result.stdout.strip()
if status == 'active':
print(f"Service '{service_name}' is running (active).")
else:
print(f"Service '{service_name}' is not running (status: {status}).")
return status == 'active'
except subprocess.CalledProcessError as e:
print(f"Error checking service '{service_name}': {e.stderr.strip()}")
return False
except FileNotFoundError:
print("Error: 'systemctl' command not found. Is this a systemd-based system?")
return False
except Exception as e:
print(f"An unexpected error occurred: {e}")
return False
if __name__ == "__main__":
print("Checking Nginx status:")
check_service_status('nginx')
print("\nChecking Apache2 status:")
check_service_status('apache2')
print("\nChecking status of a nonexistent service:")
check_service_status('nonexistent_service')
This script uses the systemctl is-active command, which is standard on modern Linux systems using systemd.
Conclusion
Through these examples, I believe you have seen the convenience and power of Python in automating Linux system administration tasks. From simple tasks like checking disk usage, to analyzing logs or backing up data, Python can help you solve them efficiently and with less risk.
Investing time in learning Python will not only help you become a better sysadmin but also open up many development opportunities in the fields of DevOps and automation. Start writing your own scripts today!
