Installing and Configuring Fluent Bit on Linux: High-Performance Log Collection, Filtering, and Forwarding to Loki and Elasticsearch - ITFROMZERO - Share tobe shared! A practical guide to configuring Fluent Bit on Linux: Collect, parse, and route Nginx logs and syslog to Grafana Loki and Elasticsearch with minimal resource overhead.

Table of Contents

Quick start: Set up Fluent Bit and ship logs in 5 minutes

Let’s skip the lengthy theoretical explanations and dive straight into practice to get a lightweight log collector running smoothly on your machine.

1. Installing Fluent Bit on Ubuntu / Debian

Open your terminal and run the official Fluent Bit installation script to add the repository and install the latest release:

curl https://raw.githubusercontent.com/fluent/fluent-bit/master/install.sh | sh
sudo systemctl daemon-reload
sudo systemctl enable fluent-bit

For RHEL, Rocky Linux, or AlmaLinux, run the equivalent command:

curl https://raw.githubusercontent.com/fluent/fluent-bit/master/install.sh | sh
sudo systemctl enable --now fluent-bit

2. Creating a minimal configuration to stream metrics to the console

The default configuration file is located at /etc/fluent-bit/fluent-bit.conf. Create the sample configuration below to quickly test your pipeline:

sudo tee /etc/fluent-bit/fluent-bit.conf << 'EOF'
[SERVICE]
    Flush         1
    Log_Level     info
    Daemon        off

[INPUT]
    Name          cpu
    Tag           my_cpu
    Interval_Sec  2

[OUTPUT]
    Name          stdout
    Match         *
EOF

3. Testing directly in the foreground

Run the binary directly to inspect the data outputting to your terminal:

/opt/fluent-bit/bin/fluent-bit -c /etc/fluent-bit/fluent-bit.conf

Are JSON lines containing CPU metrics printing steadily every 2 seconds? Congratulations, your pipeline is up and running.

Fluent Bit Architecture: How lightweight is it?

Many people wonder: “Why not just use Logstash or Fluentd since they are so widely adopted?” The answer comes down to resource overhead. Logstash consumes anywhere from 500MB to 1GB of RAM because it runs on the JVM. Fluentd, written in Ruby, still takes around 50-100MB. In contrast, Fluent Bit is written entirely in pure C. It consumes merely 15-30MB of RAM and under 1% CPU even while processing thousands of logs per second. On a 1 vCPU – 1GB RAM VPS or a Kubernetes cluster with thousands of nodes, this distinction is critical.

The log processing pipeline consists of 5 sequential stages:

INPUT: Ingests data from log files (tail), syslog, systemd journal, or metrics.
PARSER: Converts raw text (Nginx, Apache) into structured JSON.
FILTER: Modifies fields, extracts IP addresses, discards noise, or attaches host metadata.
BUFFER: Buffers logs in memory or writes them to disk (filesystem buffer) in case destinations experience downtime.
OUTPUT: Sends logs to target destinations such as Grafana Loki, Elasticsearch, OpenSearch, or Kafka.

Data reaches the right destination thanks to the Tag and Match routing mechanism. In the INPUT block, you assign Tag web.nginx.access. In the OUTPUT block, you specify Match web.nginx.*. Fluent Bit automatically evaluates prefix matching to route records accordingly.

Production-Ready Setup: Dual-shipping to Loki and Elasticsearch

A common architectural pattern across engineering teams is log branching. Loki is used for live tailing and quick querying on Grafana dashboards. Meanwhile, Elasticsearch archives logs for security audits or data teams requiring deep full-text search capabilities.

Collecting Nginx access logs and system Syslog

Here is a production-grade /etc/fluent-bit/fluent-bit.conf configuration for Linux server environments:

[SERVICE]
    Flush         1
    Log_Level     info
    Parsers_File  parsers.conf
    Storage.path  /var/log/fluent-bit/buffer
    Storage.sync  normal
    Storage.checksum off
    Storage.backlog.mem_limit 10M

# 1. INPUT: Ingest Nginx logs
[INPUT]
    Name              tail
    Tag               web.nginx.access
    Path              /var/log/nginx/access.log
    Parser            nginx
    DB                /var/log/fluent-bit/nginx.db
    Mem_Buf_Limit     15MB
    Storage.type      filesystem

# 2. INPUT: Ingest Systemd Journal (Syslog)
[INPUT]
    Name              systemd
    Tag               host.systemd
    Read_From_Tail    On
    Storage.type      filesystem

# 3. FILTER: Append host metadata
[FILTER]
    Name              record_modifier
    Match             *
    Record env        production
    Record cluster    vps-sg-01

# 4. OUTPUT 1: Forward logs to Grafana Loki
[OUTPUT]
    Name              loki
    Match             web.nginx.*
    Host              loki-server.internal
    Port              3100
    Labels            job=fluent-bit, app=nginx, env=$env
    Auto_Kubernetes_Labels off

# 5. OUTPUT 2: Forward logs to Elasticsearch
[OUTPUT]
    Name              es
    Match             host.*
    Host              es-cluster.internal
    Port              9200
    Index             linux-systemd-logs
    Type              _doc
    HTTP_User         elastic
    HTTP_Passwd       SecretPassword123
    TLS               On
    TLS.verify        Off
    Retry_Limit       5

3 Essential Parameters You Shouldn’t Overlook:

DB /var/log/fluent-bit/nginx.db: Fluent Bit records the current read offset into a compact SQLite database file. When the service restarts or the OS reboots, log ingestion resumes right where it left off without duplicate or missed log lines.
Storage.type filesystem: Enables disk buffering. If Elasticsearch experiences a 15-minute outage, your logs stay safely backed up on disk instead of bloating memory and triggering out-of-memory errors.
Retry_Limit 5: Caps retry attempts during network failures, preventing worker threads from hanging indefinitely.

Production Tuning & Lessons Learned

The stability of any monitoring stack relies on thoughtful tuning. Early on, a brief 5-minute maintenance window on the Loki backend would trigger OOM alerts across multiple servers as in-memory buffers became exhausted. Here are 3 hard-earned lessons you should adopt immediately:

1. Always enable Disk Buffering instead of relying solely on RAM

By default, Fluent Bit buffers log entries strictly in RAM. When a downstream endpoint encounters network latency or returns HTTP 429/503 errors, memory consumption quickly hits Mem_Buf_Limit. The system is then forced to drop logs or trigger the Linux OOM Killer, potentially taking down your web server alongside it. Always configure Storage.type filesystem on all critical INPUT blocks.

2. Drop noisy logs before transmitting over the network

Avoid shipping unnecessary logs across the wire. Health checks from an AWS ALB or Kubernetes liveness probes hitting /healthz every 2 seconds will rapidly burn through your storage budget. Use the grep filter to discard them right at the source:

[FILTER]
    Name    grep
    Match   web.nginx.access
    Exclude log ^.*"GET /healthz.*200.*$

This single rule can cut down daily log ingestion volume by 40-60% across your clusters.

3. Validate syntax with dry-run before reloading

Do not rush to run systemctl restart fluent-bit immediately after editing configs. Verify the syntax first using the -d (dry-run) flag:

/opt/fluent-bit/bin/fluent-bit -c /etc/fluent-bit/fluent-bit.conf --dry-run

Once you see configuration syntax ok in your terminal, you can safely reload the service:

sudo systemctl restart fluent-bit
sudo journalctl -u fluent-bit -f

By mastering the processing pipeline and disk buffering mechanics, you will ensure a rock-solid, resource-efficient log shipper that runs reliably at scale.