Multipath configuration - Proxmox 5.3-1 - Intel Modular Server

[emmanuel.paul]

Hi,

I have configured multipath on 4 nodes in my Intel Modular Server. The problem is that when I stress test one of the storage control module while the system is running, I get different results each time.

Some times, multipath is working and the path is switch automatically, but some times it does not.

Configuration is the same on each of the 4 nodes, but multipath works differently on each node when I pull out SCMs.

I have attached outputs of configuration bellow.

Can someone help?

uname -a
Linux pve221 4.15.18-9-pve #1 SMP PVE 4.15.18-30 (Thu, 15 Nov 2018 13:32:46 +0100) x86_64 GNU/Linux

/etc/multipath.conf
defaults { polling_interval 5 fast_io_fail_tmo 5 dev_loss_tmo 10 checker_timeout 15 user_friendly_names no } blacklist { devnode "^(ram|raw|loop|fd|md|dm-|sr|scd|st)[0-9]*" devnode "^(hd|xvd)[a-z][[0-9]*]" devnode "^cciss!c[0-9]d[0-9]*[p[0-9]*]" } devices { device { vendor "Intel" roduct "Multi-Flex" path_grouping_policy "group_by_prio" uid_attribute ID_SERIAL prio "alua" path_checker tur path_selector "round-robin 0" hardware_handler "1 alua" failback 30 rr_weight uniform rr_min_io 100 no_path_retry queue features "1 queue_if_no_path" retain_attached_hw_handler no product_blacklist "VTrak V‐LUN" } }

multipath -ll
222a0000155aa8365 dm-5 Intel,Multi-Flex size=2.6T features='1 queue_if_no_path' hwhandler='1 alua' wp=rw |-+- policy='round-robin 0' prio=50 status=active | `- 0:0:0:2 sdc 8:32 active ready running `-+- policy='round-robin 0' prio=1 status=enabled `- 0:0:1:2 sdf 8:80 active ready running 222b6000155529315 dm-0 Intel,Multi-Flex size=60G features='1 queue_if_no_path' hwhandler='1 alua' wp=rw |-+- policy='round-robin 0' prio=50 status=active | `- 0:0:0:0 sda 8:0 active ready running `-+- policy='round-robin 0' prio=1 status=enabled `- 0:0:1:0 sdd 8:48 active ready running 222400001552becb2 dm-1 Intel,Multi-Flex size=2.6T features='1 queue_if_no_path' hwhandler='1 alua' wp=rw |-+- policy='round-robin 0' prio=50 status=active | `- 0:0:0:1 sdb 8:16 active ready running `-+- policy='round-robin 0' prio=1 status=enabled `- 0:0:1:1 sde 8:64 active ready running

Thanks,

Emmanuel

 

[LnxBil]

Configuration is the same on each of the 4 nodes, but multipath works differently on each node when I pull out SCMs.

What do you mean by that? Do you get timeouts or very slow I/O response?

 

[emmanuel.paul]

What do you mean by that? Do you get timeouts or very slow I/O response?

Well I get Kernel dumps on screen. Sometimes the console responds right away, sometimes it never comes back unless I reboot the node.

 

[bbgeek17]

Linux pve221 4.15.18-9-pve #1 SMP PVE 4.15.18-30 (Thu, 15 Nov 2018 13:32:46 +0100) x86_64 GNU/Linux

There have been many changes in disk/dm/etc areas since kernel 4.15 been released. Both the kernel and PVE you have are EoL. You will get a much more stable environment if you upgrade your systems.

---

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox

 

[emmanuel.paul]

You will get a much more stable environment if you upgrade your systems.

I tried the same configuration on Proxmox 7.2 - Linux pve225 5.15.30-2-pve #1 SMP PVE 5.15.30-3 (Fri, 22 Apr 2022 18:08:27 +0200) x86_64 GNU/Linux but it wasn't working. Couldn't find a way to boot in multipath.

 

[LnxBil]

Well I get Kernel dumps on screen.

Oh, that is not good and normally not a configuration problem. Can you setup a serial or network console to log the kernel traces? It may yield to something useful.

Couldn't find a way to boot in multipath.

Oh that ... yes, I gave up on that A LONG TIME AGO. We switched all to internal RAID1 for booting and never looked back (and never had any problem).

 

[emmanuel.paul]

Thanks, will look into all of that!