Ditch SSH Bastion Hosts: Effortless Infrastructure Access Management with HashiCorp Boundary – ITFROMZERO

Table of Contents

The “SSH Key Management” Nightmare

When I first started working, I once spent an entire morning just setting up access for a dev team of 20. The process was extremely manual: devs sent their Public Keys, and I copy-pasted them into the authorized_keys file on the Bastion Host and each target server.

Things started getting messy when personnel changed. Someone leaving meant I had to audit dozens of servers to delete their keys. After a security audit of a system with 15+ servers, I discovered shocking numbers: over 30% of old keys still existed even though the employees had left long ago. The Bastion Host was no longer a layer of protection but a “black hole” of risk. Just one leaked private key, and the entire infrastructure was exposed to hackers.

Why the Traditional Bastion Host Model is Obsolete

The problem isn’t with the SSH protocol. It’s how we manage it. There are 3 critical weaknesses making SREs turn away from Bastion Hosts:

Opaque Identity: Bastion Hosts only care if the key matches. They don’t know who is actually typing the commands. They are completely disconnected from Identity Providers (IdP) like Google or Okta.
Over-privileged Access: Once inside the Bastion, users can often roam freely across the entire internal network. This violates the principle of Least Privilege.
Audit Nightmares: Seeing a log showing the ubuntu user deleted the database at 2 AM is one thing, but which of the 20 devs was that ubuntu user? Finding the culprit usually takes a full day of manual SSH log reviews.

Common Alternatives

Before discovering Boundary, I tried a few stopgap solutions:

Using a VPN: This is more secure but grants access at the Network level (Layer 3). Once on the VPN, users can “see” servers they shouldn’t even touch.
SSH Certificate Authority (SSH CA): A professional solution using short-lived keys. However, operating a dedicated CA system is overkill for small teams or startups.
Identity-aware Proxy (Boundary): The perfect match. Instead of checking IPs or keys, the system authenticates based on Identity. You only see what you’re authorized to see based on who you are.

HashiCorp Boundary: Keyless Infrastructure Access

Boundary allows you to connect to servers without worrying about IPs or complex key management. Just log in with your company account, and Boundary automatically establishes a temporary secure connection to the required resource.

Step 1: Quick Start with Dev Mode

Don’t rush into complex setups. Try dev mode to understand how it works in 5 minutes. It will spin up a demo environment right in your machine’s memory.

# Install on Ubuntu
wget -O- https://apt.releases.hashicorp.com/gpg | sudo gpg --dearmor -o /usr/share/keyrings/hashicorp-archive-keyring.gpg
echo "deb [signed-by=/usr/share/keyrings/hashicorp-archive-keyring.gpg] https://apt.releases.hashicorp.com $(lsb_release -cs) main" | sudo tee /etc/apt/sources.list.d/hashicorp.list
sudo apt update && sudo apt install boundary

# Run Boundary dev mode
boundary dev

After this command, the terminal will display the Auth Method ID and password. Open your browser and go to http://127.0.0.1:9200 to start exploring the admin interface.

Step 2: Setting up Targets and Hosts

Boundary’s structure follows a hierarchy: Host Catalog -> Host Group -> Host -> Target.

In practice, subnetting for Workers is crucial. I often use the IP Subnet Calculator to calculate IP ranges for separate network zones. This ensures Workers connect to targets accurately, avoiding IP conflicts between Staging and Prod environments.

Here is how to create an SSH Target using the command line:

# Create scope for the project
boundary scopes create -name "Project_Alpha" -scope-id "global"

# Declare the server list
boundary host-catalogs create static -name "Backend_Pool" -scope-id "p_1234567890"

# Add a specific server
boundary hosts create static -name "DB_Master" -address "10.0.1.50" -host-catalog-id "hc_1234567890"

Step 3: “One-touch” Connection

Forget the ssh root@ip command. With Boundary, you only need a single command. The system automatically creates an intermediate proxy right on your local machine.

boundary connect ssh -target-id t_1234567890

Boundary automatically opens a local port and maps it directly to the destination server. You can still use familiar tools like Termius or iTerm2. The biggest plus is that every session is recorded (Session Recording). If an incident occurs, you can just “rewind the tape” to see exactly what happened.

Pro Tips for Real-world Deployment

When configuring complex YAML files for Boundary, I often use the YAML ↔ JSON Converter to validate syntax. This tool runs 100% in the browser, making it extremely secure. You don’t have to worry about leaking sensitive infrastructure info to third-party servers.

If you need to generate unique tokens or IDs for your system, check out the UUID Generator. It helps quickly create standard UUID v4 strings to assign to resources in Boundary.

Advice for Production Environments

dev mode is only for learning. When deploying for real, keep these 3 principles in mind:

Isolate the Controller: Place the Controller in the most secure network zone. Only Workers need direct contact with Target servers.
Secure the Root Key: Always use AWS KMS or HashiCorp Vault to encrypt Boundary’s root keys.
Integrate OIDC immediately: Never create users manually. Connect with Google Workspace. When an employee leaves, simply lock their email, and all their server access evaporates in a second.

Shifting from a “key-holding” mindset to “identity authentication” is a vital step for modern DevOps. It not only protects the system better but also frees the SRE team from tedious, repetitive tasks.