Rescuing CentOS Stream 9: Using Rescue Mode and Chroot to Fix GRUB Errors – ITFROMZERO

Table of Contents

When the “GRUB rescue>” black screen appears

You’ve just updated the Kernel or changed partitions on your CentOS Stream 9 server and typed the reboot command. Instead of the login screen, the system returns the haunting grub rescue> prompt. All running services suddenly stop, and the pressure mounts on the administrator’s shoulders.

I once found myself in this situation after accidentally cleaning up the wrong files in the /boot directory. At that moment, my first thought was to reinstall the OS, but doing so would mean losing all previous complex configurations. Luckily, the duo of Rescue Mode and Chroot saved me. They allow you to “break into” a broken system to fix errors from within without losing a single byte of data.

Why did your server suddenly fail to boot?

Operational experience shows there are 3 most common scenarios where a Linux system “goes on strike”:

Corrupted Bootloader (GRUB): Often caused by errors during the Kernel update process (which might occur during automatic security updates) or the Boot record being overwritten.
Errors in mount configuration files: Just one typo in /etc/fstab and the OS will hang because it cannot find the necessary partitions.
Driver Conflicts: The newly upgraded Kernel is incompatible with the server’s old hardware.

If you cannot SSH, you must use the Console (on a VM) or plug a monitor directly into the physical machine to operate.

Don’t rush to reinstall the OS when encountering boot errors

Many new administrators often choose to Restore a Snapshot or reinstall the OS. Restoring a Snapshot is very fast, but you will lose all the latest data since the last backup. And reinstalling the operating system? That is the most time-consuming option and extremely unprofessional in a Production environment, especially if you have already performed the 10 essential post-installation steps for CentOS Stream 9.

Using Rescue Mode is the shortest path. You just need to boot from an ISO file, mount the old hard drive, and use the chroot command. At this point, the rescue environment turns into the real server environment, allowing you to operate as if no error ever occurred.

The CentOS Stream 9 server rescue process

Here are the steps I usually apply to handle the situation quickly in about 15 minutes.

Step 1: Accessing Rescue Mode

Prepare a CentOS Stream 9 ISO file and attach it to the machine (via USB or Virtual Drive).

Boot and select Boot from ISO.
In the main menu, navigate down to Troubleshooting.
Select Rescue a CentOS Stream system.

When the screen displays 4 options, press 1 (Continue). The system will scan the disks and automatically mount the old operating system into the temporary directory /mnt/sysroot.

# Press Enter when you see the notification to enter the Shell
sh-5.1#

Step 2: Using Chroot to “enter” the old system

You are currently in the ISO disk environment. To edit the server’s files, you need to change the root directory to the machine’s hard drive using the following command:

chroot /mnt/sysroot

The command prompt will change. Now, every change you make will directly affect the faulty system.

Step 3: Handling GRUB errors

If the error prevents the machine from showing the Kernel selection menu, you need to reinstall GRUB. Use the lsblk command to accurately identify the hard drive name (e.g., /dev/sda or /dev/vda).

For legacy BIOS:

grub2-install /dev/sda
grub2-mkconfig -o /boot/grub2/grub.cfg

For modern UEFI:

dnf reinstall grub2-efi shim
grub2-mkconfig -o /boot/efi/EFI/centos/grub.cfg

Step 4: Editing system files and Resetting the password

If the error is in /etc/fstab, use vi to open the file. Delete or comment out the unfamiliar mount lines you added before the error occurred.

vi /etc/fstab

While you’re at it, if you’ve forgotten the root password, change it right here:

passwd root

Step 5: Activating SELinux Relabel and rebooting

This is the most easily forgotten step. When intervening from an external environment, the SELinux security labels of files might become incorrect, preventing you from logging in after rebooting.

# Request the system to rescan SELinux labels on boot
touch /.autorelabel
exit
reboot

After rebooting, the autorelabel process will run (taking about 2-5 minutes depending on the data volume). Please be patient.

Practical experience

The chroot technique isn’t just for fixing boot errors. I once had to urgently handle 5 servers that needed their IPs changed and network reconfigured simultaneously after moving Datacenters. Using Rescue Mode allowed me to intervene deeply in the system configuration without worrying about running service barriers. For advanced diagnostics, knowing how to use sosreport on CentOS Stream 9 can also be a lifesaver.

Quick tip: Always check if the /boot partition is mounted using the ls /boot command. If the directory is empty, you must mount it manually before running the grub2-install commands.

Mastering Rescue Mode will help you stay calm when a server encounters an issue, because you know for sure that you always have the key to reopen the door to the system.