Installing Headscale on Fedora Server: Self-Host a Private Tailscale Mesh VPN

Fedora tutorial - IT technology blog
Fedora tutorial - IT technology blog

Tailscale is the top choice for deploying a mesh VPN thanks to its lightning-fast setup and virtually zero WireGuard configuration. However, the free SaaS tier is limited to 3 users and 100 devices. For organizations with strict compliance requirements or homelabs running numerous nodes, routing all metadata through a third-party server is far from ideal. Headscale solves this dilemma: an open-source, independent coordination server fully compatible with official Tailscale clients.

On the server side, Fedora shines with modern 6.x kernels, in-tree WireGuard kernel modules, and hardened SELinux policies out of the box. Below is a practical, production-ready guide to deploying Headscale on Fedora Server 39/40.

1. Quick Start: Deploy Headscale in 5 Minutes

This guide assumes you have a clean Fedora Server instance running at IP 203.0.113.10 (or a domain like vpn.example.com with an A record pointing to this address).

Step 1: Download the Binary and Configure System Permissions

# Fetch the latest stable release
HEADSCALE_VERSION=$(curl -s https://api.github.com/repos/juanfont/headscale/releases/latest | grep tag_name | cut -d '"' -f 4 | sed 's/v//')
wget https://github.com/juanfont/headscale/releases/download/v${HEADSCALE_VERSION}/headscale_${HEADSCALE_VERSION}_linux_amd64 -O /usr/local/bin/headscale
chmod +x /usr/local/bin/headscale

# Create a dedicated system user without login shell for security
useradd --system --shell /sbin/nologin --create-home headscale
mkdir -p /etc/headscale /var/lib/headscale /var/run/headscale
chown -R headscale:headscale /etc/headscale /var/lib/headscale /var/run/headscale

Step 2: Set Up a Minimal Configuration File

Create /etc/headscale/config.yaml with essential settings:

cat <<'EOF' > /etc/headscale/config.yaml
server_url: http://203.0.113.10:8080
listen_addr: 0.0.0.0:8080
metrics_listen_addr: 127.0.0.1:9090
grpc_listen_addr: 127.0.0.1:50443
grpc_allow_insecure: true

prefixes:
  v4: 100.64.0.0/10
  v6: fd7a:115c:a1e0::/48

derp:
  server:
    enabled: false
  urls:
    - https://controlplane.tailscale.com/derpmap/default
  auto_update_enabled: true
  update_frequency: 24h

disable_check_updates: true
ephemeral_node_inactivity_timeout: 30m
database:
  type: sqlite3
  sqlite:
    path: /var/lib/headscale/db.sqlite
EOF

Step 3: Launch the Systemd Service and Open Firewall Ports

cat <<'EOF' > /etc/systemd/system/headscale.service
[Unit]
Description=Headscale Coordination Server
After=network.target

[Service]
User=headscale
Group=headscale
ExecStart=/usr/local/bin/headscale serve
Restart=always
RestartSec=5
LimitNOFILE=65536

[Install]
WantedBy=multi-user.target
EOF

systemctl daemon-reload
systemctl enable --now headscale

# Open coordination port 8080 via firewalld
firewall-cmd --add-port=8080/tcp --permanent
firewall-cmd --reload

Step 4: Create a User and Register Your First Node

Create an administrative user (previously referred to as a namespace):

headscale users create devteam

Switch to your client machine (with the tailscale package installed) and initiate authentication pointing to your self-hosted server:

tailscale up --login-server http://203.0.113.10:8080

The terminal will output an authentication key in the format mkey:0123456789abcdef.... Copy this key, return to your Fedora server terminal, and register the node:

headscale nodes register --user devteam --key <NODE_KEY_FROM_CLIENT>

Verify connectivity on your client by running tailscale status. The machine is now assigned an IP from the 100.64.x.x subnet and ready for communication.

2. How It Works: Why Headscale Is Secure and Lightweight

Decoupled Control Plane and Data Plane

Traditional VPN architectures like OpenVPN or IPsec route all traffic through a centralized gateway (Hub-and-Spoke). If that central server bottlenecks or fails, the entire network goes down. A WireGuard mesh architecture cleanly separates these concerns:

  • Data Plane (Actual Traffic): Encrypted data travels Peer-to-Peer (P2P) directly between Client A and Client B. Two devices on the same local network transfer files at line speed (1–10 Gbps). Headscale never sits in the middle, decrypts packets, or bears any traffic load.
  • Control Plane (Coordination Server): Headscale only stores public keys, assigns internal virtual IPs, and assists with NAT traversal (STUN). Once two nodes discover each other and establish an encrypted session, the coordination server could go offline without disrupting ongoing P2P transfers.

3 Core Configuration Parameters to Know

  • server_url: The public endpoint clients use to reach the control plane. When behind an Nginx or Caddy reverse proxy handling HTTPS, update this to your full domain (e.g., https://headscale.company.internal). Never set a private IP here if remote clients connect over the Internet.
  • prefixes.v4: The IP subnet allocated to your mesh network. Tailscale adheres to RFC6598 CGNAT space (100.64.0.0/10). Using this default avoids IP conflicts with typical home networks (192.168.1.0/24) or corporate subnets (10.0.0.0/8).
  • derp.urls: A list of public DERP relay servers. If two client devices reside behind Symmetric NATs (common in 4G/5G mobile networks or public Wi-Fi) and cannot punch a direct UDP hole, Headscale falls back to an intermediary DERP relay to keep the tunnel alive.

3. Production-Grade Advanced Setups

Pre-authentication Keys: Automating CI/CD and Server Provisioning

When provisioning Kubernetes worker nodes or ephemeral staging virtual machines, manually approving mkey tokens is not scalable. Headscale supports reusable pre-authenticated keys:

# Generate a reusable auth key valid for 24 hours for the dev cluster
headscale preauthkeys create --user devteam --reusable --expiration 24h

Inject the generated key into your cloud-init scripts or CI/CD deployment pipelines:

tailscale up --login-server http://203.0.113.10:8080 --authkey <PREAUTH_KEY> --accept-routes

Subnet Routers: Bridging Mesh Nodes into Your Local LAN

Legacy hardware such as network printers, managed switches, or IP cameras cannot run Tailscale. You can configure your Fedora Server node as a subnet gateway, granting remote devices seamless access to the entire 192.168.1.0/24 LAN.

# 1. Enable packet forwarding at the kernel level
echo 'net.ipv4.ip_forward = 1' | tee -a /etc/sysctl.d/99-tailscale.conf
echo 'net.ipv6.conf.all.forwarding = 1' | tee -a /etc/sysctl.d/99-tailscale.conf
sysctl -p /etc/sysctl.d/99-tailscale.conf

# 2. Advertise the local LAN subnet from the node
tailscale up --login-server http://203.0.113.10:8080 --advertise-routes=192.168.1.0/24

# 3. Enable the route on Headscale
headscale routes list
headscale routes enable -r 1

Now, while working from a coffee shop, a laptop connected to Tailscale can directly open a local Proxmox dashboard at https://192.168.1.50:8006 without exposing any ports on your home or office router.

Access Control with ACLs

By default, all nodes belonging to the same user can communicate unconditionally. When you need fine-grained isolation (e.g., granting developers access only to ports 80 and 443 on web servers), create /etc/headscale/acl.json:

{
  "acls": [
    {
      "action": "accept",
      "src": ["devteam"],
      "dst": ["tag:web:80,443"]
    }
  ],
  "tagOwners": {
    "tag:web": ["devteam"]
  }
}

Specify acl.policy_path: /etc/headscale/acl.json in your main configuration and restart the service. Any connection attempts to SSH port 22 or PostgreSQL port 5432 on nodes tagged with tag:web will be dropped immediately.

4. Operations and Troubleshooting

Optimizing P2P Connections: Direct vs. Relay

To help peers successfully traverse NAT and communicate via direct UDP instead of falling back to DERP relays, open the default Tailscale WireGuard listening port on your server firewall:

firewall-cmd --add-port=41641/udp --permanent
firewall-cmd --reload

Execute tailscale status on your client machine to inspect link health:

# Ideal connection (latency ~15ms, bandwidth 800+ Mbps)
100.64.0.2   workstation   devteam   linux   active; direct 14.225.2.18:41641

# Fallback connection over relay (latency spikes to 90-180ms, throughput throttled to 5-15 Mbps)
100.64.0.2   workstation   devteam   linux   active; relay "hkg"

If you see relay "hkg" (Hong Kong), inspect your firewall and router configurations to ensure outbound UDP traffic is not being restricted or filtered.

Online Backups for the SQLite Database

All registered nodes, authorization keys, and route tables live inside a single database file: /var/lib/headscale/db.sqlite. Avoid copying the raw file using cp while the service is active, as concurrent writes can corrupt the database. Use SQLite's built-in online backup mechanism instead:

sqlite3 /var/lib/headscale/db.sqlite ".backup '/var/backups/headscale_$(date +%F_%H%M).sqlite'"

Since the resulting snapshot is typically only a few hundred kilobytes, you can easily automate it via a nightly cron job scheduled for 2:00 AM.

Resolving MagicDNS Issues on Fedora Clients

If your client can ping the node IP 100.64.0.2 but fails to resolve its hostname with ping workstation, the issue is almost always a conflict in systemd-resolved:

resolvectl status tailscale0

Check whether the tailscale0 interface received DNS server 100.100.100.100. If the search domain is missing, run the following command to instruct the workstation to route DNS queries through Tailscale:

resolvectl domain tailscale0 "~example.com"
Share: