Securing SSH with Hardware Security Keys (FIDO2/Yubikey): When Private Keys Are No Longer a ‘Fatal Weakness’ – ITFROMZERO

If you’re a Developer or Sysadmin, SSH keys are likely part of your daily workflow. The usual practice is generating a key with ssh-keygen -t ed25519, setting a passphrase, and assuming your server is ‘locked down tight’.

But let’s look at reality: a single accidental commit of your id_ed25519 file to GitHub or a malware infection can spin things out of control. According to GitGuardian, millions of secrets are leaked annually in public repos. In those cases, no matter how long the passphrase is, hackers can brute-force it offline instantly. Through recent audits of over 10 systems, I’ve found that 80% of vulnerabilities stem from loose private key file management.

The most effective way to solve this is by using a Hardware Security Key (like Yubikey or other FIDO2 keys). With this method, the private key resides permanently within the secure chip. Even if a hacker steals the key file from your machine, they can’t do anything without a physical touch confirmation on the device.

Table of Contents

Quick start: Setup SSH with Yubikey in 5 minutes

For those who already have a key. I’ll assume you’re using Ubuntu or macOS and your key supports the FIDO2 protocol.

Step 1: Check OpenSSH version

FIDO2 support (known as sk – Security Key) requires OpenSSH version 8.2 or higher. Check your version quickly with:

ssh -V

If your version is lower than 8.2, you’ll need to upgrade your OS or install a newer OpenSSH version to proceed.

Step 2: Generate keys directly on the device

Plug your Yubikey into a USB port and run the following command:

ssh-keygen -t ed25519-sk -C "[email protected]"

At this point, the light on the key will flash. Simply tap the metal circle to confirm User Presence. Then, set a passphrase for the key handle as usual.

Step 3: Transfer the Public Key to the Server

Use the familiar command to push the key to your target server:

ssh-copy-id -i ~/.ssh/id_ed25519_sk user@your_server_ip

Everything is ready! From now on, when you log in, the server will require a touch on the key. No touch, no entry. It provides immense peace of mind.

Why Hardware Keys ‘outclass’ traditional SSH Keys?

Many ask: ‘If it still generates an id_ed25519_sk file on my machine, what happens if I lose that file?’

This is where FIDO2/U2F differs. The file you see is actually just a Key Handle (an identification fragment), not the real Private Key. The actual key stays deep inside the secure chip and is never exposed as a file that can be copied.

With regular SSH Keys: The file on your hard drive is everything. Losing the file + exposing the passphrase = Loss of server control.
With SSH FIDO2 (sk): Key = File handle + Physical key. Even if a hacker gets the file, it’s useless without your physical Yubikey in hand.

This is essentially Multi-Factor Authentication (MFA) at the SSH protocol layer. You don’t need to install Google Authenticator or configure complex scripts on the server anymore.

Advanced: Resident Keys (On-device keys)

On newer Yubikey models (like the Series 5), the Resident Keys feature is a major leap in convenience.

Typically, when moving to a new computer, you have to copy the file handle over. With Resident Keys, this fragment is stored directly in the Yubikey’s memory. Add the -O resident option during creation:

ssh-keygen -t ed25519-sk -O resident -O verify-required

(Note: verify-required forces you to enter the key’s PIN before touching, creating three layers of security: something you have, something you know, and something you do).

When switching workstations, just plug in the key and type:

ssh-add -K

This command automatically ‘downloads’ the key from the Yubikey into the current machine’s SSH Agent. You no longer need to worry about manual file transfers.

Hardening: Configuring the Server to only accept Hardware Keys

Don’t stop at just creating the key; lock the door from the server side to ensure absolute security. We will configure the server to reject all types of regular keys.

Open the SSH configuration file on the server:

sudo nano /etc/ssh/sshd_config

Add the following line to accept only algorithms with the -sk suffix:

PubkeyAcceptedAlgorithms [email protected],[email protected]

Save and restart the service:

sudo systemctl restart ssh

Now, even if a hacker has millions of traditional private keys, the server will reject them right at the gate.

Real-world implementation experience

Security is always a trade-off. Here are 4 ‘battle-tested’ tips for a smoother deployment:

Always have a backup plan: Never rely on a single key. If it’s lost or damaged (though the failure rate is extremely low), you’ll be locked out. Buy at least two (e.g., Yubikey 5 NFC), set both up, and keep one as a backup.
Set a FIDO2 PIN: Never leave the key ‘open’. A PIN prevents unauthorized use even if someone finds it on your desk.
Note for WSL2: On Windows, mapping USB to Linux is a bit tricky. You need to install usbipd so WSL can recognize the Yubikey.
OS Compatibility: Some older operating systems like CentOS 7 won’t support ed25519-sk. In that case, ecdsa-sk is a more compatible alternative.

In an era where servers are probed 24/7, investing in a security key is a highly worthwhile decision. It helps you sleep better, eliminating worries about Private Key leaks. If you encounter the error sign_and_send_pubkey: signing failed, check if your local machine has the libfido2 library installed!