Why Do 2-Node Clusters Need a Witness?
Setting up a standard 3-node vSAN cluster for remote offices or branch offices (Edge sites) can sometimes feel like overkill. In my experience managing systems at small branches, the biggest challenge is cost optimization. Typically, vSAN requires at least 3 physical nodes to maintain a majority vote (Quorum). If one node fails, the remaining two still have the authority to keep the system running.
However, investing in an extra server costing thousands of dollars just to act as a “tie-breaker” for voting is a waste of budget. This is where the vSAN 2-node Cluster combined with the Witness Appliance comes into play. This solution allows you to completely eliminate the hardware costs of a third node while still ensuring High Availability (HA).
Comparing Quorum Approaches
To get approval from management or clients, you need to clarify the difference between buying an additional server and using a Witness virtual machine.
| Criteria | Physical 3-Node Cluster | 2-Node Cluster + Witness Appliance |
|---|---|---|
| Hardware Cost | Expensive (Requires 3 identical servers) | Cheap (2 Servers + 1 free VM) |
| Witness Location | Located inside the cluster | Must be outside the cluster (PC, Cloud, or another Host) |
| Required Bandwidth | 10Gbps+ (Data traffic) | ~2 Mbps (Metadata only) |
| Network Latency (RTT) | Low (< 1ms) | Supports up to 500ms |
Key Strengths and Limitations to Consider
Why use it?
- Maximum Savings: The Witness Appliance is an OVA file provided for free by VMware. You don’t spend extra on vSphere or Windows licenses for this entity.
- Extremely Lightweight: It does not contain virtual machine data (VMDK). It only stores metadata to act as an arbitrator in the event of a network split (split-brain).
- Low Requirements: An old PC running ESXi or a small VM in the Cloud is enough to act as a Witness.
Technical Hurdles:
- Network Infrastructure: You must ensure Layer 2 or Layer 3 connectivity between the 2 main nodes and the Witness node.
- Risk Scenarios: If the Witness node and one main node fail simultaneously, the vSAN cluster will stop operating due to the loss of Quorum.
Real-World Deployment Steps
Below is the process I usually apply for rapid deployment in actual projects.
Step 1: Network Infrastructure Preparation
Network connectivity is the soul of this model. The Witness Appliance has two separate network interfaces:
- Management Network: Used to connect to vCenter.
- vSAN Network: Used to communicate with the vSAN interfaces of the 2 physical nodes.
Make sure you can ping between these interfaces. I always use the vmkping command to check thoroughly before official configuration.
# Test connectivity from the physical host to the Witness vSAN IP
vmkping -I vmk1 [WITNESS_VSAN_IP]Step 2: Deploy vSAN Witness Appliance
- Download the OVA file from the Broadcom homepage. Choose a version that exactly matches the ESXi version you are running.
- Right-click on vCenter and select Deploy OVF Template.
- Select the appropriate size. For clusters with fewer than 10 VMs, the Tiny option (2 vCPUs, 8GB RAM, 15GB Disk) is sufficient.
- Configure a static IP. Absolutely do not use DHCP because if the IP changes after a power outage, the entire vSAN cluster will lose connection.
Step 3: Configure the Witness Host in vCenter
After booting up, the Witness Appliance appears as a nested ESXi host. Add it to vCenter like a regular host, but keep it outside the cluster.
Steps to activate on the Cluster:
- Go to Configure -> vSAN -> Services.
- In the Fault Domains & Stretched Cluster section, click Configure.
- Select Two-node vSAN cluster mode.
- Point to the Witness Appliance you just prepared.
- Confirm the Disk Group. Usually, the OVA has already pre-created 1 virtual SSD (Cache) and 1 virtual HDD (Capacity) for you.
Hard-Won Lessons from Operations
During the process of troubleshooting for customers, I have summarized 3 important notes:
1. Avoid “Self-Sabotage”: Never place the Witness VM on the very vSAN cluster it is protecting. If the vSAN cluster fails, the Witness also goes down, resulting in no arbitrator to help recover the system.
2. MTU Synchronization: If the physical switches use Jumbo Frames (MTU 9000) for vSAN, the Witness network card must also be set to 9000. MTU mismatch is the most common cause for the Witness to go into a “Disconnected” state without warning.
# Check the current MTU on the vmk interface
esxcfg-vmknic -l3. Licensing: VMware provides a free license included within the Witness OVA. You don’t need to spend extra money on this part; just focus on licensing the 2 main nodes and vCenter.
Conclusion
Using the vSAN Witness Appliance is the smartest way to own an Enterprise system at a budget-friendly price. This solution is particularly effective for telecom stations or retail stores. As long as you set up the network correctly from the start, the system will run extremely stably.
If you encounter the error “Witness host is not in the same subnet” or have difficulty Claiming Disks, please leave a comment below so I can assist you!
