How to Install VMware ESXi 8 for Free on a Physical Server and Create Your First VM

VMware tutorial - IT technology blog
VMware tutorial - IT technology blog

Why Choose ESXi for a Physical Server Instead of Workstation?

If you’re familiar with VMware Workstation on a desktop, ESXi is a completely different beast. It doesn’t run on top of Windows or Linux — it replaces the operating system entirely, installing directly on bare-metal. Every resource on the server goes straight to your VMs, with no intermediate OS eating up RAM or CPU.

I started using ESXi when I picked up a used Dell PowerEdge R720 for around $350. Instead of installing Ubuntu and running KVM like I had before, I gave ESXi a shot and noticed an immediate difference: managing VMs through a web browser is incredibly convenient — no more SSH-ing into the server every time. When I later migrated to Proxmox for my home lab, I came to appreciate ESXi’s strengths even more — particularly its stability and hardware compatibility with older servers.

Starting with version 8.0, ESXi can be used for free with a free license. Broadcom acquired VMware and changed some policies, but at the time of writing, free registration is still available. The main limitations: a maximum of 8 vCPUs per VM and no vCenter — more than enough for a personal lab.

Prerequisites Before Installing

Minimum Hardware Requirements

  • CPU: 64-bit Intel or AMD with virtualization support (VT-x/AMD-V)
  • RAM: 8GB minimum — but ESXi itself uses around 4GB, leaving only ~4GB for VMs. 16GB+ is strongly recommended
  • Storage: 32GB for ESXi, a separate drive for the VM datastore
  • NIC: At least one network card supported by ESXi (Intel i350 and Broadcom BCM5719 are the most common)

Important note: Check the hardware compatibility list before installing. ESXi is far more strict than Workstation — cheap Realtek NICs are usually not supported out of the box. I once bought an RTL8111 card, and ESXi wouldn’t detect it at all. I had to track down a custom ISO with the driver bundled in just to get it working.

Downloading ESXi 8 and Registering for a Free License

  1. Go to customerconnect.vmware.com and create a free account
  2. Navigate to Products & Accounts → Free Products and find VMware vSphere Hypervisor
  3. Download the ISO file (around 400MB) and save the free license serial key

Create a bootable USB using Rufus (Windows) or dd (Linux):

# Linux: write the ISO to a USB drive (replace /dev/sdX with your USB device)
sudo dd if=VMware-VMvisor-Installer-8.0-*.iso of=/dev/sdX bs=4M status=progress
sync

Step-by-Step ESXi 8 Installation

Step 1: Configure BIOS/UEFI

Before booting from the USB, enter your BIOS and verify the following:

  • Enable Intel VT-x or AMD-V (Virtualization Technology)
  • Enable VT-d (if you need PCI device passthrough)
  • Disable Secure Boot (some servers require this before ESXi can be installed)
  • Set boot order: USB first

Step 2: The Installation Process

Once you boot from the USB, the ESXi installer is fairly straightforward — far fewer choices than Ubuntu or CentOS:

  1. Press Enter to begin and accept the EULA
  2. Select the disk to install ESXi on (a dedicated drive of at least 32GB is recommended)
  3. Choose a keyboard layout (US default works fine)
  4. Set a password for root — this is the only administrative account
  5. Confirm the installation and wait about 5–10 minutes
  6. Reboot and remove the USB drive

After booting, the DCUI (Direct Console User Interface) screen appears showing the server’s IP address — this is what you’ll need to access the web UI.

Step 3: Basic Network Configuration via DCUI

Press F2 on the DCUI screen to enter configuration, then log in as root. Select Configure Management Network:

  • Network Adapters: Select the NIC connected to your switch/router
  • IPv4 Configuration: Assign a static IP (strongly recommended) — e.g. 192.168.1.10/24, gateway 192.168.1.1
  • DNS Configuration: Set a hostname and DNS server

Press Escape → Y to save and restart the network.

Detailed Configuration via the Web UI

Accessing the ESXi Host Client

Open a browser and navigate to https://192.168.1.10 (replace with your configured IP). Log in with root and the password you set during installation.

The ESXi Host Client runs entirely in the browser — no plugins or separate client required. This is one of my favorite things about it compared to managing KVM from the command line. CPU usage, network throughput, VM list, storage status — everything is visible on a single tab.

Entering the Free License

Go to Manage → Licensing → Assign License and paste the serial key you saved when downloading the ISO. Without a license, ESXi only runs as a 60-day trial — once it expires, all VMs are locked and cannot be powered on.

Creating a Datastore for VMs

A datastore is where VM files (vmdk, vmx, etc.) are stored. If your server has a second drive, create the datastore there to keep it separate from the ESXi installation drive:

  1. Go to Storage → New Datastore
  2. Select Create new VMFS datastore
  3. Choose the disk, give it a name (e.g. datastore1), and select VMFS 6
  4. Complete the wizard — the datastore is ready to use

Uploading an OS ISO to the Datastore

To install an OS on a VM, you first need to upload an ISO to ESXi:

  1. Go to Storage → datastore1 → Datastore Browser
  2. Create a folder called ISOs, then click Upload
  3. Select the ISO file from your computer (e.g. Ubuntu Server 24.04, ~2.6GB)

Creating Your First Virtual Machine

The VM Creation Wizard

Go to Virtual Machines → Create / Register VM:

  1. Select creation type: Create a new virtual machine
  2. Select a name and guest OS: Name the VM, choose Guest OS family (Linux) and version (Ubuntu Linux 64-bit)
  3. Select storage: Choose datastore1
  4. Customize settings: This is the most important step

Configuring VM Resources

In the Customize settings step, here’s how I typically configure an Ubuntu Server VM:

  • CPU: 2 vCPUs (can be increased while the VM is running — no shutdown required)
  • Memory: 2GB RAM (bump to 4GB if you have enough to spare)
  • Hard disk 1: 40GB, select Thin provisioning to save actual disk space
  • Network Adapter: Keep the default VM Network (bridged to the physical network)
  • CD/DVD Drive: Select Datastore ISO file and point it to the ISO you uploaded

Thin provisioning means the vmdk file only occupies the space actually in use. A freshly installed Ubuntu VM typically takes up around 5–6GB even if you allocated 40GB — a significant saving when running 5–10 VMs on the same datastore.

Powering On and Installing the OS

Click Power On, then click the console icon to open the VM display. The Ubuntu Server installation proceeds exactly as it would on a physical machine.

Once the install is done, remember to go into VM settings → CD/DVD Drive → disconnect the ISO. If you skip this, the next reboot will boot straight back into the installer.

Monitoring and Verification After Launch

Tracking Resource Usage

The ESXi Host Client has a Monitor tab for both the host and individual VMs. The two metrics I check most often:

  • Host → Monitor → Performance: CPU ready time — if it exceeds 5%, VMs are being starved for CPU; consider reducing vCPU counts or the number of running VMs
  • VM → Monitor → Performance: Memory balloon — ESXi reclaims RAM from VMs when the host runs low, which noticeably slows them down

For a deeper look at tuning these numbers, the VMware VM performance optimization checklist from production covers CPU, memory, and storage settings that make a real difference under load.

Installing VMware Tools Inside the VM

VMware Tools allows the VM to receive shutdown commands from ESXi, enables copy-paste support, and keeps the system clock in sync. For Ubuntu:

sudo apt update
sudo apt install open-vm-tools -y

# Verify it's running
systemctl status open-vm-tools

After installation, the ESXi Host Client will show the VMware Tools status change from Not installed to Running.

Take a Snapshot Before Major Changes

A habit I picked up from my sysadmin days: before updating the OS, changing critical config, or installing new software — take a snapshot first. If something breaks, you can revert in about 30 seconds without reinstalling from scratch.

Right-click the VM → Snapshots → Take Snapshot, and give it a descriptive name like before-nginx-install.

Useful SSH Commands on the ESXi Host

To SSH into the ESXi host, first enable SSH under Manage → Services. Then:

# List all VMs and their status
vim-cmd vmsvc/getallvms

# Check the power state of VM ID 1
vim-cmd vmsvc/power.getstate 1

# View real-time resource usage
esxtop

esxtop is like top on Linux but purpose-built for ESXi — it shows CPU ready time, memory balloon, and disk latency per VM. Extremely useful when debugging performance issues. If you want to go further and automate routine tasks like bulk VM provisioning or scheduled reporting, PowerCLI is the natural next step.

A Few Things Worth Knowing After Installation

Having used both ESXi and Proxmox extensively, I find ESXi better suited for older enterprise servers like Dell PowerEdge or HP ProLiant — driver support is stronger, and when things go wrong, VMware’s documentation is thorough, with dedicated KB articles for specific errors.

Proxmox is more flexible and fully open-source with no feature limitations on the free tier. But on older servers, I’ve run into situations where NICs weren’t detected and RAID cards lacked drivers — issues that ESXi handled more gracefully.

Once the basics are in place, where you go next depends on your goals: configure vSwitches to isolate networks for different VM groups, attach an NFS/iSCSI datastore from a NAS for shared storage, or — if you have three or more hosts — consider vCenter. For a single-host home lab, standalone ESXi is more than sufficient.

Share: