[SOLVED] Boot From SAN - Multipath boot disk - Proxmox Installation

E

Explisit

New Member
Proxmox Subscriber
Mar 25, 2025
4
0
1
I have a 3-node Proxmox cluster running on an HP C7000 chassis with three BL460 Gen10 blades. The shared storage is provided by an HPE 3PAR SAN. I've configured a shared LUN to host the virtual machines, and boot-from-SAN is working successfully.


However, I'm encountering an issue with multipath configuration on the boot-from-SAN disk. When one of the 3PAR controllers is down or rebooting during maintenance or upgrades, it causes disruptions in Proxmox due to the lack of multipath redundancy.


I came across a thread suggesting the use of a vanilla Debian 12 installation, which apparently supports multipath for the boot disk. However, I'm unsure whether this approach is officially supported or if it's feasible to convert an existing Proxmox installation to support boot disk multipathing.


Has anyone successfully configured Proxmox to boot from a SAN using multipath after the initial installation? I suspect it involves adding the appropriate GRUB modules for SAN multipath and possibly adjusting the initramfs and other configurations. Any guidance, insights, or steps from someone who’s done this would be greatly appreciated.
 
Hi Explisit, can you provide following commands output?
Bash: 
multipath -ll
multipath -v3 -d
cat /etc/multipath.conf
lsblk
cat /proc/scsi/scsi
ls -ld /sys/block/sd*/device
ls -la /dev/disk/by-id/
And have you make sure the root filesystem is mount from multipath device instead of device file?
 
bbgeek17 said:
Hi @Explisit , welcome to the forum.

Do you have all required tools installed? https://forum.proxmox.com/multipath -ll<br>multipath -v3 -d<br><span>cat</span> /etc/multipath.conf<br>lsblk<br><span>cat</span> /proc/scsi/scsi<br><span>ls</span> -ld /sys/block/sd*/device<br><span>ls</span> -la /dev/disk/by-id/
threads/fiber-channel-san-and-multipath-after-a-reboot.151594/#post-686938

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox
Click to expand...
Hi, I have the following two multipath packages installed.

ii multipath-tools 0.9.4-5~bpo11+1 amd64 maintain multipath block device access
ii multipath-tools-boot 0.9.4-5~bpo11+1 all Support booting from multipath devices
 
david_tao said:
Hi Explisit, can you provide following commands output?
Bash: 
multipath -ll
multipath -v3 -d
cat /etc/multipath.conf
lsblk
cat /proc/scsi/scsi
ls -ld /sys/block/sd*/device
ls -la /dev/disk/by-id/
And have you make sure the root filesystem is mount from multipath device instead of device file?
Click to expand...
Hi, attached is the requested output.
 

Attachments

After some back and forth, I have manage to solve it. It is not really a Proxmox issue, but rather Linux specific. Just make sure that the LVM is discovered after the multipath, adjust the multipath setup, have the multipath package installed, loaded the multipath modules, and fix the race condition between multipath and lvm for creating the boot device.

Hereunder is a brief list of the steps, in case anyone is stumbled across same/similar issue.

Markdown (GitHub flavored): 
# Debian 12: Boot from SAN via Multipath + LVM (UEFI) - Setup Guide

This guide outlines how to configure Debian 12 to boot from a SAN LUN using device-mapper **multipath**, with an LVM root volume and UEFI boot. It ensures reliable failover if one SAN path goes down.

---

## ✅ 1. Install Required Packages

```bash
sudo apt update
sudo apt install multipath-tools multipath-tools-boot
```

- `multipath-tools` enables multipath functionality.
- `multipath-tools-boot` ensures multipath is usable early in the boot process.

---

## ✅ 2. Find the WWID of the Boot LUN

Use the following to identify the WWID of your boot disk:

```bash
sudo /lib/udev/scsi_id --whitelisted --device=/dev/sdX
```

Replace `/dev/sdX` with your boot disk (e.g., `/dev/sdd`).

Or use:

```bash
sudo multipath -ll
```

Look for the WWID string like: `360002ac0000000000000000d0002811d`

---

## ✅ 3. Configure `/etc/multipath.conf`

Create or edit `/etc/multipath.conf`:

```conf
defaults {
    user_friendly_names yes
    find_multipaths yes
}

blacklist_exceptions {
    wwid "360002ac0000000000000000d0002811d"
}

multipaths {
    multipath {
        wwid  "360002ac0000000000000000d0002811d"
        alias "pve_boot"
    }
}
```

---

## ✅ 4. Add the WWID to `/etc/multipath/wwids`

```bash
sudo multipath -W
```

This ensures your boot LUN is always multipathed, even with a single path.

---

## ✅ 5. Update GRUB Kernel Parameters

Edit `/etc/default/grub`:

```bash
sudo nano /etc/default/grub
```

Set:

```bash
GRUB_CMDLINE_LINUX="root=/dev/mapper/pve-root"
```

Then:

```bash
sudo update-grub
```

---

## ✅ 6. Adjust `/etc/fstab`

Edit `/etc/fstab` to use the LVM on the multipath device:

```fstab
/dev/mapper/pve-root   /         ext4    defaults        0 1
/dev/mapper/pve-swap   none      swap    sw              0 0
```

Avoid using `/dev/sdX` or UUIDs tied to raw paths.

---

## ✅ 7. Create Initramfs Hook Script with Udev Wait

Create a custom initramfs script:

```bash
sudo nano /etc/initramfs-tools/scripts/init-top/zz-multipath-init
```

Paste this content:

```bash
#!/bin/sh
PREREQ="udev"

prereqs() {
    echo "$PREREQ"
}

case $1 in
    prereqs)
        prereqs
        exit 0
        ;;
esac

. /scripts/functions

modprobe qede || true
modprobe dm_mod
modprobe dm_multipath
modprobe dm_round_robin
modprobe scsi_dh_alua


udevadm trigger --action=add
udevadm settle

# Wait for SAN paths to appear
tries=20
while [ $tries -gt 0 ]; do
  if ls /dev/sd[a-z] 2>/dev/null | grep -q .; then
    break
  fi
  sleep 1
  tries=$((tries - 1))
done

multipath -v2
```

Make it executable:

```bash
sudo chmod +x /etc/initramfs-tools/scripts/init-top/zz-multipath-init
```

---

## ✅ 8. Add Multipath Modules to Initramfs

Edit `/etc/initramfs-tools/modules`:

```text
dm_mod
dm_multipath
dm_round_robin
scsi_dh_alua
```

Add your NIC driver if needed (e.g., `qede` for HP 50G CNA).

---

## ✅ 9. Rebuild Initramfs

```bash
sudo update-initramfs -u -k all
```

Ensure your WWID and config files are included:

```bash
lsinitramfs /boot/initrd.img-$(uname -r) | grep multipath
```

Should show:
- `etc/multipath.conf`
- `etc/multipath/wwids`
- `usr/sbin/multipath`

---

## ✅ 10. Reboot and Verify

```bash
sudo reboot
```

Then check:

```bash
multipath -ll
mount | grep " / "
cat /proc/cmdline | grep root=
```

You should see:
- Root is mounted from `/dev/mapper/pve-root`
- Multipath device `pve_boot` with active paths
- No direct use of `/dev/sdX`

---

## ✅ Optional: Test Failover

Temporarily disable a SAN path and confirm the system continues running:

```bash
multipath -ll
```

You’ll see failed paths, but device stays active.

---
 
Last edited:
You must log in or register to reply here.
Top Bottom