iSCSI Multipathing on Linux: Up and Running in 5 Minutes
If you’re running storage over SAN with only a single connection, that’s a ticking time bomb. A broken cable, a failed NIC, a faulty switch — any of these means immediate downtime. iSCSI Multipathing solves this by using multiple parallel network paths to the same storage device.
This quick start assumes you already have an iSCSI target running (TrueNAS, Synology, or targetcli on Linux) and a Linux server with at least 2 network interfaces connected to the storage network.
Step 1: Install Required Packages
# Ubuntu/Debian
apt install open-iscsi multipath-tools -y
# RHEL/CentOS/Rocky
dnf install iscsi-initiator-utils device-mapper-multipath -yStep 2: Retrieve the Initiator IQN
cat /etc/iscsi/initiatorname.iscsi
# Output: InitiatorName=iqn.1993-08.org.debian:01:abc123def456Copy this IQN and add it to the ACL list on your iSCSI target.
Step 3: Discover and Log In to the Target via Both Network Paths
# Discover from interface 1 (e.g., 192.168.10.10 is the server's storage IP)
iscsiadm -m discovery -t st -p 192.168.100.1 -I 192.168.10.10
# Discover from interface 2
iscsiadm -m discovery -t st -p 192.168.101.1 -I 192.168.11.10
# Log in to all discovered targets
iscsiadm -m node --loginStep 4: Verify Both Paths Are Established
iscsiadm -m session
# Should show 2 sessions, each through a different IP
# tcp: [1] 192.168.100.1:3260,1 iqn.2023-01.com.example:storage
# tcp: [2] 192.168.101.1:3260,1 iqn.2023-01.com.example:storageStep 5: Enable Multipath
mpathconf --enable --with_multipathd y
systemctl enable --now multipathd
# Verify multipath has detected the device
multipath -llIf the output of multipath -ll shows a device with 2 paths underneath, you’ve completed the basic setup.
Going Deeper: How iSCSI and Multipathing Work
What is iSCSI?
iSCSI (Internet Small Computer Systems Interface) enables block storage access over standard TCP/IP networks — without the need for expensive Fibre Channel. Your server (the initiator) connects to a storage server (the target) and sees the storage device as a regular local disk.
Why Do You Need Multipathing?
Multipath I/O (MPIO) creates a single virtual device from multiple physical paths to the same LUN. Two key benefits:
- Failover: When one path fails, I/O automatically switches to the remaining path — zero service interruption
- Load balancing: Distributes I/O across multiple paths, increasing actual throughput
Configuration File Structure
cat /etc/multipath.confDefault file after running mpathconf --enable:
defaults {
user_friendly_names yes
find_multipaths yes
}
blacklist {
devnode "^sda" # Exclude the OS disk from multipathing
}Advanced Configuration
Choosing the Right Load Balancing Policy
Linux multipath supports several load distribution modes. Here are the ones I use most:
defaults {
user_friendly_names yes
path_grouping_policy multibus
path_selector "round-robin 0" # Round-robin evenly across all paths
failback immediate # Revert to primary path upon recovery
rr_weight priorities
no_path_retry fail
}
devices {
device {
vendor "SYNOLOGY" # Adjust to match your vendor
product "Storage"
path_grouping_policy multibus
path_checker tur
features "0"
hardware_handler "0"
prio "const"
rr_weight uniform
}
}After editing the config:
systemctl restart multipathd
multipath -F && multipath # Flush and rediscover
multipath -ll # View the new statusAssigning a Persistent Name to the Multipath Device
Instead of relying on /dev/mapper/mpatha (which may change after a reboot), assign an alias based on the WWID:
# Get the device WWID
multipath -ll | grep -E "^[a-z0-9]+"
# Output: 360000000000000001 dm-0 SYNOLOGY,iSCSI Storage
# Add to /etc/multipath.conf
multipaths {
multipath {
wwid 360000000000000001
alias san_storage_lun0
}
}The device will then always be /dev/mapper/san_storage_lun0 — easy to reference in fstab and scripts.
Configuring iSCSI to Auto-Login After Reboot
# Enable auto login for all targets
iscsiadm -m node --op update -n node.startup -v automatic
# Enable the services
systemctl enable iscsid open-iscsi
# Test by rebooting and verifying
iscsiadm -m session
multipath -llPractical Tips from Real-World Experience
Isolate Storage Traffic with a Dedicated VLAN
Never let iSCSI traffic share the management network. I once dealt with intermittent packet loss that only appeared during peak hours — it took a long time to figure out that iSCSI was competing for bandwidth with a nightly backup job on the same interface. After splitting it onto a dedicated VLAN with MTU 9000 (jumbo frames), throughput nearly doubled and I/O timeouts disappeared entirely.
Configure jumbo frames for the storage interface:
# Temporary (for testing)
ip link set eth1 mtu 9000
ip link set eth2 mtu 9000
# Test end-to-end MTU path
ping -M do -s 8972 192.168.100.1
# -M do: no fragmentation, -s 8972: 8972 + 28 byte header = 9000 MTU
# If ping succeeds → the network supports jumbo frames end-to-endMonitor Multipath Path Health
# View real-time status
watch -n 2 'multipath -ll'
# Script to check for failed paths and send an alert
#!/bin/bash
FAILED=$(multipath -ll | grep -c 'failed faulty')
if [ "$FAILED" -gt 0 ]; then
echo "WARNING: $FAILED multipath path(s) failed" | mail -s "Multipath Alert" [email protected]
fiTesting Failover for Real
Don’t wait for an actual incident to test failover. Do this periodically:
# Run continuous I/O in the background
dd if=/dev/zero of=/dev/mapper/san_storage_lun0 bs=1M count=1000 &
# While dd is running, bring down one path
multipath -f mpatha # or bring down the physical interface
ip link set eth1 down
# Check whether I/O continues
iostat -x 2
# Bring it back up
ip link set eth1 up
multipathTroubleshooting Missing Paths
# View multipathd logs
journalctl -u multipathd -f
# Rescan iSCSI session
iscsiadm -m session --rescan
# If the path still doesn't come up, try removing and re-adding it
iscsiadm -m node --logout
iscsiadm -m node --login
multipath -F && multipathSeparating Two Network Interfaces into Different Subnets
Many people plug both interfaces into the same storage subnet and then wonder why multipath isn’t picking up both paths. Linux will route I/O through a single default interface for both connections if they share the same subnet. The solution is to use policy routing:
# Create two separate routing tables
echo '100 storage1' >> /etc/iproute2/rt_tables
echo '101 storage2' >> /etc/iproute2/rt_tables
# Routing rules
ip rule add from 192.168.10.10 table storage1
ip rule add from 192.168.11.10 table storage2
ip route add 192.168.100.0/24 dev eth1 src 192.168.10.10 table storage1
ip route add 192.168.101.0/24 dev eth2 src 192.168.11.10 table storage2Then run iscsiadm discover with the -I flag to specify the source IP — only then will multipath truly traverse two separate physical paths.
