Installing Prometheus, Grafana, and Node Exporter on Fedora Server: A Production-Ready Monitoring Stack

Fedora tutorial - IT technology blog
Fedora tutorial - IT technology blog

Managing a server without monitoring dashboards is like driving blind in thick fog. A single rogue web scraper bottlenecking the CPU or an unmanaged application log eating up 100% of your disk space can bring down your entire system before you even realize what happened.

To tackle this challenge, the Prometheus + Grafana + Node Exporter triad is always a top-tier choice. This stack is lightweight, rock-solid, and exceptionally easy to scale. On a basic 2GB RAM Fedora VPS, the entire monitoring stack consumes only about 250MB to 350MB of memory.

Quick Deployment: Spin Up the Monitoring Stack in 5 Minutes

If you want to get hands-on right away, follow these streamlined steps below.

1. Install Node Exporter and Prometheus from Binaries

# Create dedicated system users without shell login access
sudo useradd --no-create-home --shell /bin/false prometheus
sudo useradd --no-create-home --shell /bin/false node_exporter

# Download and configure Node Exporter
NODE_VER="1.8.2"
curl -LO https://github.com/prometheus/node_exporter/releases/download/v${NODE_VER}/node_exporter-${NODE_VER}.linux-amd64.tar.gz
tar -xvf node_exporter-${NODE_VER}.linux-amd64.tar.gz
sudo mv node_exporter-${NODE_VER}.linux-amd64/node_exporter /usr/local/bin/
sudo chown node_exporter:node_exporter /usr/local/bin/node_exporter

# Download and configure Prometheus Server
PROM_VER="2.54.1"
curl -LO https://github.com/prometheus/prometheus/releases/download/v${PROM_VER}/prometheus-${PROM_VER}.linux-amd64.tar.gz
tar -xvf prometheus-${PROM_VER}.linux-amd64.tar.gz
sudo mkdir -p /etc/prometheus /var/lib/prometheus
sudo mv prometheus-${PROM_VER}.linux-amd64/prometheus /usr/local/bin/
sudo mv prometheus-${PROM_VER}.linux-amd64/promtool /usr/local/bin/
sudo mv prometheus-${PROM_VER}.linux-amd64/consoles /etc/prometheus
sudo mv prometheus-${PROM_VER}.linux-amd64/console_libraries /etc/prometheus
sudo chown -R prometheus:prometheus /etc/prometheus /var/lib/prometheus /usr/local/bin/prometheus /usr/local/bin/promtool

2. Install Grafana via DNF Repository

Add the official Grafana Labs repository to install and receive the latest updates via DNF:

sudo tee /etc/yum.repos.d/grafana.repo << 'EOF'
[grafana]
name=grafana
baseurl=https://rpm.grafana.com
gpgcheck=1
gpgkey=https://rpm.grafana.com/gpg.key
enabled=1
EOF

sudo dnf install -y grafana
sudo systemctl daemon-reload
sudo systemctl enable --now grafana-server

3. Open Ports in Firewalld

By default, Fedora enforces strict firewall rules. You need to open port 3000 (Grafana) and port 9090 (Prometheus Web UI):

sudo firewall-cmd --permanent --add-port={3000/tcp,9090/tcp}
sudo firewall-cmd --reload

Open your browser and navigate to http://<SERVER_IP>:3000. The default initial credentials are admin / admin.

Under the Hood: How Does the Pull Model Work?

This monitoring setup operates on a pull-based architecture (Pull Model), with responsibilities cleanly divided among the three components:

  • Node Exporter (The Collector): A lightweight background daemon on the OS consuming just around 15MB of RAM. It reads host metrics directly from /proc and /sys, exposing them in HTTP text format on port 9100/metrics.
  • Prometheus (Storage & Processing Engine): Every 15 seconds, Prometheus actively issues an HTTP request to scrape metrics from Node Exporter. Data is compressed and written directly to its time-series database (TSDB) located at /var/lib/prometheus.
  • Grafana (Visualization Layer): Grafana does not store server metrics. Instead, it queries Prometheus using PromQL and renders the data into intuitive, real-time dashboards.

Create Systemd Services for Node Exporter and Prometheus

To ensure services launch on system boot and automatically recover from failures, configure systemd unit files.

Create /etc/systemd/system/node_exporter.service:

[Unit]
Description=Node Exporter
After=network.target

[Service]
User=node_exporter
Group=node_exporter
Type=simple
ExecStart=/usr/local/bin/node_exporter --web.listen-address="127.0.0.1:9100"
Restart=always
RestartSec=5

[Install]
WantedBy=multi-user.target

Next, create the Prometheus scrape configuration file at /etc/prometheus/prometheus.yml:

global:
  scrape_interval: 15s
  evaluation_interval: 15s

scrape_configs:
  - job_name: "prometheus"
    static_configs:
      - targets: ["localhost:9090"]

  - job_name: "fedora_node"
    static_configs:
      - targets: ["localhost:9100"]
        labels:
          instance: "fedora-srv-01"

Create the unit file /etc/systemd/system/prometheus.service:

[Unit]
Description=Prometheus Server
Wants=network-online.target
After=network-online.target

[Service]
User=prometheus
Group=prometheus
Type=simple
ExecStart=/usr/local/bin/prometheus \
  --config.file=/etc/prometheus/prometheus.yml \
  --storage.tsdb.path=/var/lib/prometheus/ \
  --storage.tsdb.retention.time=30d \
  --storage.tsdb.retention.size=10GB \
  --web.console.templates=/etc/prometheus/consoles \
  --web.console.libraries=/etc/prometheus/console_libraries \
  --web.enable-lifecycle
Restart=always

[Install]
WantedBy=multi-user.target

Enable and start both services concurrently:

sudo systemctl daemon-reload
sudo systemctl enable --now node_exporter prometheus
sudo systemctl status node_exporter prometheus --no-pager

Visualizing Metrics and Setting Up Alerts

1. Connect Data Source and Import Dashboard

  1. Log in to Grafana; the system will prompt you to set a new admin password.
  2. From the left navigation menu, go to Connections > Data Sources > click Add data source > select Prometheus.
  3. In the URL field, enter http://localhost:9090, scroll to the bottom, and click Save & test. A green success banner confirms the connection.
  4. Navigate to Dashboards > New > Import. Enter dashboard ID 1860 (the community-favorite Node Exporter Full dashboard).
  5. Select the Prometheus data source you just created and click Import.

Instantly, you’ll see per-core CPU load, memory utilization (Cached, Free, Buffers), NVMe/SSD disk I/O throughput, and network interface traffic updated in real time.

2. Configure Basic Alert Rules

Create /etc/prometheus/alert_rules.yml to catch issues before your server goes down:

groups:
  - name: HostAlerts
    rules:
      - alert: HighCpuLoad
        expr: 100 - (avg by(instance) (rate(node_cpu_seconds_total{mode="idle"}[5m])) * 100) > 85
        for: 5m
        labels:
          severity: warning
        annotations:
          summary: "High CPU load on {{ $labels.instance }}"
          description: "CPU utilization has stayed above 85% for the last 5 minutes."

      - alert: DiskSpaceLow
        expr: (node_filesystem_free_bytes{mountpoint="/"} / node_filesystem_size_bytes{mountpoint="/"}) * 100 < 15
        for: 2m
        labels:
          severity: critical
        annotations:
          summary: "Low disk space on root partition: {{ $labels.instance }}"
          description: "Root partition (/) has less than 15% free space remaining."

      - alert: NodeDown
        expr: up{job="fedora_node"} == 0
        for: 1m
        labels:
          severity: critical
        annotations:
          summary: "Connection lost to host {{ $labels.instance }}"
          description: "Prometheus has failed to scrape metrics from Node Exporter for over 1 minute."

Remember to register the rule file in /etc/prometheus/prometheus.yml:

rule_files:
  - "alert_rules.yml"

Before applying changes, always validate syntax errors using the promtool utility:

promtool check config /etc/prometheus/prometheus.yml

Production Best Practices on Fedora

Hot-Reload Configurations Without Downtime

Thanks to the --web.enable-lifecycle flag configured in your systemd service, you don’t need to run systemctl restart prometheus every time you update alert rules. Simply trigger a POST request:

curl -X POST http://localhost:9090/-/reload

Prometheus will reload the new configuration within milliseconds—without interrupting metric ingestion for even a single second.

Manage TSDB Disk Usage

By default, Prometheus retains data for 15 days without capping total disk space. If your server captures thousands of metrics per second, your storage can fill up quickly. The flags --storage.tsdb.retention.time=30d and --storage.tsdb.retention.size=10GB provide peace of mind: Prometheus automatically purges the oldest data blocks when storage reaches 10GB or exceeds 30 days.

Secure the Node Exporter Port

Never expose port 9100 directly to the public Internet unless protected behind an authenticated reverse proxy. Hardware metrics, mount points, and system user details provide valuable intelligence for reconnaissance attacks. The safest practice is binding Node Exporter strictly to 127.0.0.1:9100 as shown in the instructions above.

Share: