2:00 AM. The phone rings off the hook. The Nginx cluster on the Rocky Linux 9 server delivered just this afternoon suddenly throws a 403 Forbidden error.
A quick check reveals the web root is already chmod 755 and the ownership is properly set to nginx:nginx. Even reading index.html as an unprivileged user works without a hitch. Yet access over HTTP is completely blocked.
Half-asleep, many sysadmins hastily run setenforce 0 just to get back to bed. Last week, while auditing over 10 production servers, I was shocked to find more than half had SELINUX=disabled permanently set in their config files ages ago. Disabling SELinux entirely just because of a permission issue is like tearing down your front security door because the key jammed.
1. Quick Start: Troubleshooting in 5 Minutes
Suspecting an SELinux issue in the middle of the night? Don’t disable it just yet. Follow these 3 quick triage steps:
Step 1: Check the System Status
# Check detailed status
sestatus
# Or quickly check the current mode
getenforce
If the terminal returns Enforcing, SELinux is actively blocking actions that violate policy. Permissive only logs warnings without blocking. If it returns Disabled, the security layer is completely turned off.
Step 2: Temporarily Switch to Permissive Mode
Instead of disabling it completely, switch SELinux to Permissive mode. Does the website start working immediately? Then SELinux is 100% the culprit.
# Change mode in RAM, no reboot required
sudo setenforce 0
# Confirm the change
getenforce
# Output: Permissive
Step 3: Trace the Generated AVC Logs
# Find the most recently blocked actions
sudo ausearch -m avc -ts recent
# Inspect raw logs if auditd is not installed
sudo tail -n 20 /var/log/audit/audit.log | grep AVC
2. The Core Concept: Understanding SELinux Architecture
Unlike traditional DAC (Discretionary Access Control) based on rwx permissions and user/group ownership, SELinux operates on MAC (Mandatory Access Control). This means even if a process runs as root, its hands are tied unless explicitly permitted by policy.
2.1. Security Context Structure
Every file, directory, process, and network port is labeled with a security tag called a Security Context. Its structure consists of four fields: user:role:type:level.
Add the -Z flag to the ls command to view labels:
ls -lZ /var/www/html/
Actual output:
-rw-r--r--. 1 nginx nginx system_u:object_r:httpd_sys_content_t:s0 index.html
95% of your day-to-day work revolves around the Type field (in this case, httpd_sys_content_t). Web server processes are only permitted to access files that have a compatible type defined by policy.
2.2. Fixing Context Mismatches with semanage and restorecon
Back to our 2:00 AM shift. Source code was cloned into /home/deploy/myweb/ and rsync’d directly to /var/www/html/ using the -a flag. As a result, the files retained the user_home_t label from the home directory. Naturally, Nginx was forbidden from touching them.
To change the label temporarily for quick testing:
# Temporarily change label (lost upon filesystem relabeling)
sudo chcon -t httpd_sys_content_t /var/www/html/index.html
For a production-ready fix, define the context in the policy database using semanage, then apply it with restorecon:
# Install policy tools if command is not found
sudo dnf install policycoreutils-python-utils -y
# Register permanent rule for the entire web directory
sudo semanage fcontext -a -t httpd_sys_content_t "/var/www/html(/.*)?"
# Write the default context to disk
sudo restorecon -Rv /var/www/html
3. Advanced: Booleans and Custom Policies with audit2allow
3.1. Managing SELinux Booleans
SELinux includes hundreds of built-in toggle switches called Booleans. You can simply turn them on or off without writing custom rules from scratch. A classic scenario is Nginx or PHP-FPM failing to connect to a remote MySQL host or having outbound internet access blocked.
# List web server related booleans
getsebool -a | grep httpd
# Allow web server outbound network connections (cURL third-party APIs)
sudo setsebool -P httpd_can_network_connect on
# Allow web server to connect to remote databases (MySQL port 3306, PostgreSQL 5432)
sudo setsebool -P httpd_can_network_connect_db on
Always remember to include the -P (Persistent) flag. Without it, your changes will be discarded as soon as the server reboots.
3.2. Running Services on Non-Standard Ports
Changing your SSH port to 2222 to evade automated scanners? The sshd service will crash on restart. The reason: SELinux only allows SSH to bind to port 22 by default.
# List allowed ports for SSH
sudo semanage port -l | grep ssh
# Allow TCP port 2222 for SSH
sudo semanage port -a -t ssh_port_t -p tcp 2222
# Verify and restart sshd
sudo systemctl restart sshd
3.3. Creating Custom Modules with audit2allow
When running custom applications or specialized monitoring agents like Prometheus node-exporter, SELinux might block specific behaviors. In these cases, audit2allow can analyze AVC denial logs and generate a tailored policy module.
5-step production workflow:
# Step 1: Temporarily switch to Permissive to collect all denials
sudo setenforce 0
# Step 2: Trigger the failing application workflow again
curl -X POST https://api.myweb.local/upload
# Step 3: Inspect denial reasons with audit2why
sudo ausearch -m avc -ts recent | audit2why
# Step 4: Generate a policy module named myapp_custom
sudo ausearch -m avc -ts recent | audit2allow -M myapp_custom
# Step 5: Load module into the kernel and re-enable Enforcing mode
sudo semodule -i myapp_custom.pp
sudo setenforce 1
4. Hard-Earned Production Lessons
- Never set
SELINUX=disabledin/etc/selinux/config: When re-enablingEnforcing, the server will be forced to relabel the entire filesystem at boot (touch /.autorelabel). On disks storing over 500GB of data or millions of small files, the system can get stuck at the boot screen for 30 to 60 minutes. - Always run
audit2whybeforeaudit2allow: Many issues can be resolved simply by flipping an existing Boolean. Avoid needlessly generating.ppfiles that clutter the kernel policy store. - Bake it into your CI/CD pipelines: Add
restorecon -Rv /var/www/htmlas the final step of your deployment script (GitLab CI, GitHub Actions, or Jenkins) to permanently prevent label mismatch errors after rsyncing code. - Audit custom modules regularly: Periodically review installed custom modules with
semodule -l. Clean up obsolete modules or those from decommissioned applications usingsemodule -r <module_name>.

