PVE8 beta: regression with multipath attached storage

[bbgeek17]

Hello Everyone,

This is a quick note to inform folks that our Proxmox storage regression test suites picked up an issue with PVE8 on Debian 12. The regressions are generally applicable to multi-path storage configurations. Our initial analysis suggests that the issue is specific to Debian 12. We're currently investigating and will provide more information in the coming days.

---

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox

 

[LnxBil]

Nice to hear. What are you testing exactly? In recent years, the multipath-tools changed in some significant configuration settings from one release to the next and broke a few times. I haven't tested PVE8 beta in a multipath setup yet.

 

[bbgeek17]

The specific tests that are failing are related to resizing disks that are multipath attached. We see significant changes to the udev ruleset in Debian 12. Our current theory is that the multipath tools package may be broken. We believe files are missing from the installation that are required for correct operation of multipathd.

---

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox

 

[bbgeek17]

A brief update on the regression: One of our developers tracked this back to a packaging issue. The version of multipath-tools shipped with Debian 12 omits a rules file needed by multipathd. For tracking purposes, a bug has been filed with Proxmox, which can be found here:

• https://bugzilla.proxmox.com/show_bug.cgi?id=4775

Our team is also submitting a bug and patch to the Debian bug tracking system. Hopefully, this will be resolved sooner than later.

---

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox

 

[Zerstoiber]

@bbgeek17 - i just want to give you a really well deserved "thank you!"
You guys are amazing, giving back so much to the community.
Not only in this example where you catch this bug to directly improve the product/code quality, but also with your blog and technotes.

I will shamelessly promote them here, i guess not enough people read this nerdy stuff yet ;-)
e.g. https://kb.blockbridge.com/technote/proxmox-tuning-low-latency-storage/
or https://kb.blockbridge.com/technote/proxmox-aio-vs-iouring/

 

[bbgeek17]

Thanks, @Zerstoiber ! The team truly appreciates your feedback.

Here's an update on the progress:

We submitted the bug to Debian yesterday: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1037539.
As of this morning, it has been accepted and resolved/closed with a fix. The multipath-tools version containing a fix will be 0.9.4-5. It should be available on Debian FTP shortly.

We'll validate the fix and report back as soon as the package is available.

---

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox

 

[bbgeek17]

Good news. Thanks to @t.lamprecht , the fix that was submitted to Debian has been pulled into the PVE8 testing repo.
We updated PVE8 systems in our CI/CD environment and all testing has passed with flying colors.

Here is the version that contains the regression fix for PVE repo: multipath-tools amd64 0.9.4-5~bpo11+1

Cheers.

---

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox