It’s 2 AM, and PagerDuty is blaring. The primary database cluster suddenly vanished from the monitoring system following an automatic kernel update. An SSH attempt returns Connection timed out. Opening the remote console via IPMI reveals that dreaded prompt: dracut: Please enter passphrase for disk. With no one there to manually enter the passphrase, the server remained down for 45 minutes.
In practice, when auditing infrastructures across various organizations, I frequently encounter two common mistakes: leaving data unencrypted in plain text for fear of reboot hangs, or storing the plaintext password directly in /etc/crypttab. Both approaches completely defeat the purpose of disk encryption.
Comparing 3 Ways to Unlock LUKS on Server Reboot
To protect data-at-rest while ensuring smooth, unattended server reboots, system administrators typically choose between three approaches:
- Method 1: Manually entering passphrase via IPMI or Dropbear SSH
After every reboot, you must open the console or SSH into the initramfs environment and type the password manually. - Method 2: Reading key files directly from a USB drive or /boot partition
The system automatically reads the decryption key from a locally stored file to unlock the LUKS partition. - Method 3: Network-Bound Disk Encryption (NBDE) with Clevis and Tang
The client only decrypts the drive upon successfully connecting to an internal Tang server over a trusted network. If an attacker pulls the drive out of the datacenter, all they get is meaningless scrambled data.
Pros and Cons of Each Approach
1. Manual Passphrase Entry
- Pros: Completely independent. You do not rely on any external network infrastructure or intermediate servers.
- Cons: High operational overhead. If a server reboots automatically at midnight or recovers from a power outage, all dependent services suffer extended downtime waiting for an operator to type the passphrase.
2. Storing Local Key Files on the Host
- Pros: Instant boot process with no network dependencies.
- Cons: A false sense of security. Anyone with physical access to the server or capable of cloning the cloud disk image can retrieve the key file in seconds, rendering encryption practically useless.
3. Using Network-Bound Disk Encryption (Clevis + Tang)
- Pros: Fully automated within the local network. The Tang server utilizes the ECDH (Elliptic-curve Diffie-Hellman) handshake protocol and never stores the client’s private key. Even if the Tang server is compromised, attackers cannot obtain keys to unlock the disks.
- Cons: Requires early-boot networking to function reliably. You also need at least two Tang servers to avoid a single point of failure (SPOF).
When Should You Deploy This Solution?
The Clevis and Tang (NBDE) model is ideal for colocation servers, on-premises data centers, or home labs. The server reboots autonomously as long as it resides within the trusted LAN. However, if someone unplugs the hard drive or walks away with the entire chassis, the data remains securely locked.
Step-by-Step Configuration Guide on Ubuntu / Debian
Lab Environment Setup
- Tang Server: IP
192.168.10.50(running Ubuntu Server 22.04/24.04) - Target Client: IP
192.168.10.100(partition to encrypt:/dev/sdb1)
Step 1: Set Up the Tang Server
The Tang service consumes less than 50MB of RAM. You can easily install it on a secondary server or an internal container:
sudo apt update
sudo apt install -y tang
sudo systemctl enable --now tangd.socketThe service runs as a systemd socket listener on TCP port 80. Test the Tang response with curl:
curl -s http://localhost/adv | head -n 5If you see a JSON payload containing public key metadata, your Tang server is operational.
Step 2: Create a LUKS2 Encrypted Partition on the Client
Log in to the client machine. Initialize a new encrypted partition and configure a fallback administrative passphrase:
# Format partition with LUKS2
sudo cryptsetup luksFormat --type luks2 /dev/sdb1
# Open partition to format ext4 filesystem
sudo cryptsetup open /dev/sdb1 secure_data
sudo mkfs.ext4 /dev/mapper/secure_data
sudo cryptsetup close secure_dataNote: Always save this passphrase in your team’s password manager (such as Bitwarden or Vault) for emergency recovery.
Step 3: Install Clevis and Bind the Partition to the Tang Server
Install the required packages on the client machine:
sudo apt update
sudo apt install -y clevis clevis-luks clevis-initramfsBind partition /dev/sdb1 to the Tang server:
sudo clevis luks bind -d /dev/sdb1 tang '{"url":"http://192.168.10.50"}'The system will display the Tang key fingerprint and ask for confirmation. Type y and enter the administrative passphrase you created in Step 2. Clevis creates a new key slot in the LUKS header without overwriting the existing password.
Verify the key slot configuration:
sudo cryptsetup luksDump /dev/sdb1If the Token section lists clevis along with the Tang server URL, the binding was successful.
Step 4: Configure Automatic Mounting via /etc/crypttab
Find the UUID of the underlying encrypted partition:
sudo blkid -s UUID -o value /dev/sdb1Assuming the returned UUID is 12345678-abcd-1234-abcd-1234567890ab, add the following line to /etc/crypttab. The _netdev parameter instructs systemd to wait for network availability before attempting decryption:
secure_data UUID=12345678-abcd-1234-abcd-1234567890ab none luks,_netdevNext, create the mount directory and add the entry to /etc/fstab:
sudo mkdir -p /mnt/data
echo "/dev/mapper/secure_data /mnt/data ext4 defaults,_netdev 0 2" | sudo tee -a /etc/fstabStep 5: Update initramfs and Test Reboot
Rebuild initramfs to incorporate all Clevis network hooks:
sudo update-initramfs -u -k allNow, reboot the client to verify the setup:
sudo rebootMonitor the boot process via IPMI or console. The system automatically acquires an IP address via DHCP, performs the handshake with the Tang server at 192.168.10.50, decrypts the partition, and mounts it to /mnt/data in 3 to 5 seconds—completely hands-free.
Production Operational Tips
- When the Tang Server is unreachable: Clevis retries several times before timing out. Once the timeout expires, the system falls back to the manual passphrase prompt. You will never be locked out of your data.
- High Availability (HA) Configuration: In production environments, avoid relying on a single Tang server. You can leverage Clevis’s Shamir’s Secret Sharing (SSS) pin to bind simultaneously to two Tang servers placed across different network switches:
sudo clevis luks bind -d /dev/sdb1 sss '{"t":1,"pins":{"tang":[{"url":"http://192.168.10.50"},{"url":"http://192.168.10.51"}]}}'With the "t":1 threshold parameter, if either Tang server responds, the drive unlocks successfully.
