Managing Processes with systemd on Linux: Practical Tips from the Trenches

Up and Running in 5 Minutes — The systemd Commands You’ll Use Every Day

Just getting started with Linux servers? Here are 5 commands you’ll type so often they’ll become muscle memory:

# Check the status of a service
systemctl status nginx

# Start / stop / restart
systemctl start nginx
systemctl stop nginx
systemctl restart nginx

# Enable / disable auto-start on reboot
systemctl enable nginx
systemctl disable nginx

# Reload configuration without restarting (if the service supports it)
systemctl reload nginx

Try it right now with nginx or any service running on your machine. Green active (running) means you’re good, red failed means something’s wrong — it’s intuitive enough that you won’t even need to read the docs.

Understanding How systemd Works

Born to replace SysVinit, systemd is now the default init system on most Linux distros: Ubuntu, Debian, CentOS/AlmaLinux, Fedora, and more. The biggest differences from the old generation: parallel startup, dependency management between services, and centralized journal logging instead of log files scattered all over /var/log/.

Everything systemd manages is called a unit. You’ll encounter these 4 types most often:

• .service — background processes (nginx, mysql, sshd…)
• .timer — similar to cron, runs tasks on a schedule
• .socket — socket activation (only starts the service when a connection arrives)
• .mount — manages mount points

Unit files live in 2 main locations:

• /lib/systemd/system/ — installed by the package manager, don’t edit directly
• /etc/systemd/system/ — for overrides or new files, this is where you work

Creating a Custom systemd Service for Your Application

This is the part I use most. Whenever I deploy a Python or Node.js script to a server, I always create a service file — I never use nohup or screen anymore.

Example: deploying a Python application running on port 8000:

sudo nano /etc/systemd/system/myapp.service

[Unit]
Description=My Python App
After=network.target

[Service]
Type=simple
User=ubuntu
WorkingDirectory=/home/ubuntu/myapp
ExecStart=/home/ubuntu/myapp/venv/bin/python app.py
Restart=on-failure
RestartSec=5
StandardOutput=journal
StandardError=journal

[Install]
WantedBy=multi-user.target

# Reload so systemd picks up the new file
sudo systemctl daemon-reload

# Enable and start
sudo systemctl enable --now myapp

Notice Restart=on-failure — the service automatically restarts on crash. Last month my service crashed at 2 AM, restarted itself within 5 seconds, and I only found out when I checked the logs the next morning. No need to stay up and SSH in to fix it.

Common Restart Values

• no — never restart (default)
• on-failure — restart when exit code is non-zero
• always — always restart, even after a manual systemctl stop
• on-abnormal — restart when killed by a signal

Use on-failure for 99% of cases. always sounds safer but creates an infinite restart loop if the app crashes immediately on startup — systemd will keep retrying until it hits the limit, then mark the service as failed anyway.

Debugging a Failed Service

Service showing failed? Don’t panic. Here’s my debugging workflow:

# Show detailed status + last few log lines
systemctl status myapp -l

# View all logs for the service
journalctl -u myapp

# Follow logs in real time
journalctl -u myapp -f

# Show logs from this boot only
journalctl -u myapp -b

# Last 50 lines
journalctl -u myapp -n 50

journalctl consolidates all logs in one place, with timestamps, filterable by unit and time range. Since I got comfortable with it, I’ve completely dropped the habit of tail -f /var/log/....

The most classic mistake: editing a .service file and forgetting to run daemon-reload. systemd is still using the old cached version, so none of your changes take effect. Every time you edit a file, you must reload — no exceptions.

Advanced — Overriding a Service Without Touching the Original File

A real-world scenario: you want to raise the file descriptor limit for nginx to 65536 (the default is only 1024), but you don’t want to edit /lib/systemd/system/nginx.service because the next apt upgrade will overwrite it.

# Use the edit command — it auto-creates the override directory
sudo systemctl edit nginx

This opens an editor and creates a file at /etc/systemd/system/nginx.service.d/override.conf. Just write the parts you want to change:

[Service]
LimitNOFILE=65536
Environment="EXTRA_OPTS=-g 'worker_processes 4;'"

To inspect the final merged result after all overrides are applied:

systemctl cat nginx

Resource Limits with cgroups

cgroups is a feature most people overlook but it’s incredibly useful — systemd has it built in, letting you cap CPU and RAM usage per service:

[Service]
# Cap at 512MB RAM
MemoryMax=512M
# Limit to 50% CPU
CPUQuota=50%
# Lower CPU priority (default is 100)
CPUWeight=50

On a 2GB VPS running 5–6 services simultaneously, this is what lets me sleep at night — no single service can hog all the resources and drag the whole system down.

systemd Timers — A Better Alternative to Cron

Most of my cronjobs have migrated to systemd timers. The reason is simple: debugging is so much easier. Logs go to the journal, you can see the last run time and the next scheduled run — no more hunting through /var/log/syslog like with cron.

Example: run a backup script every day at 2 AM:

sudo nano /etc/systemd/system/backup.service

[Unit]
Description=Daily backup script

[Service]
Type=oneshot
ExecStart=/usr/local/bin/backup.sh

sudo nano /etc/systemd/system/backup.timer

[Unit]
Description=Run backup daily at 2am

[Timer]
OnCalendar=*-*-* 02:00:00
Persistent=true

[Install]
WantedBy=timers.target

sudo systemctl enable --now backup.timer

# View all timers and their status
systemctl list-timers

Persistent=true handles the case where the server was off at 2 AM and missed the scheduled run — the timer will fire immediately when the server boots back up, instead of waiting until the following night.

Practical Tips from Real Server Operations

1. Use --now to Enable and Start Simultaneously

systemctl enable --now myapp
# Equivalent to:
systemctl enable myapp && systemctl start myapp

2. View All Failed Services at Once

systemctl --failed

This is the first command I run after a server reboot. If anything failed, I know immediately — no need to check each service one by one.

3. Analyze Boot Time

systemd-analyze blame | head -20

Lists the services consuming the most boot time. On my dev machine, snapd was eating over 30 seconds — disabled it and boot time dropped from 45s to 12s.

4. Run Services as a Dedicated User (Security)

Never run a service as root unless absolutely necessary. Create a dedicated user:

sudo useradd -r -s /bin/false myappuser

[Service]
User=myappuser
Group=myappuser
# Add sandboxing if desired
NoNewPrivileges=true
ProtectSystem=strict
ProtectHome=true

5. ExecStartPre / ExecStartPost

Run a command before the service starts — for example, wait until the database is actually ready to accept connections, not just starting up:

[Service]
ExecStartPre=/bin/sh -c 'until pg_isready -h localhost; do sleep 1; done'
ExecStart=/home/ubuntu/myapp/venv/bin/python app.py

This is better than After=postgresql.service because After only guarantees start order, not that postgres is actually accepting connections yet. I ran into this bug when my app finished starting before postgres had warmed up, causing connection failures right from the start.