Newbie: new install on Supermicro platform - disks with 4 paths show as 4 separate disks that are unusable

[SamCarson]

PLEASE HELP!
New install on Supermicro server completes normally. SAS drives show in Proxmox (and Linux) as 4 separate drives whose SCSI ids are identical. Proxmox does not set up multipath access, Therefore Proxmox cannot use the drives.
TrueNAS (BSD based) has NO trouble with this configuration.
Youtube tutorial installs with this chassis apparently work ok. The only difference is SATA instead of SAS drives.

Proxmox v6.3-3
2x Broadcom 9300-8i IT firmware (v16)
Supermicro SAS3 Dual Expander Backplane (BPN-SAS3-826EL2)
3x HGST 4TB SAS disks
Supermicro X10DRH-iT Motherboard
Supermicro CSE-826 14 Bay Chassis

 

[Stoiko Ivanov]

On a hunch - are the `multipath-tools` installed in the PVE installation?
(they are not installed by default)

if yes - what's the output of `multipath -ll`?

maybe the wiki-article: https://pve.proxmox.com/wiki/ISCSI_Multipath can provide some pointers (it is focused on iscsi and might be a bit dated)

I hope this helps!

 

[SamCarson]

root@Carson:~# multipath -ll
B1D2 (35000cca05c95d838) dm-2 HGST,HMRP4000S5xnN7.2
size=3.6T features='0' hwhandler='0' wp=rw
`-+- policy='service-time 0' prio=1 status=active
|- 0:0:2:0 sde 8:64 active ready running
|- 0:0:6:0 sdh 8:112 active ready running
|- 11:0:2:0 sdk 8:160 active ready running
`- 11:0:6:0 sdn 8:208 active ready running
B1D1 (35000cca05c941a60) dm-1 HGST,HMRP4000S5xnN7.2
size=3.6T features='0' hwhandler='0' wp=rw
`-+- policy='service-time 0' prio=1 status=active
|- 0:0:1:0 sdd 8:48 active ready running
|- 0:0:5:0 sdg 8:96 active ready running
|- 11:0:1:0 sdj 8:144 active ready running
`- 11:0:5:0 sdm 8:192 active ready running
B1D0 (35000cca05c964654) dm-0 HGST,HMRP4000S5xnN7.2
size=3.6T features='0' hwhandler='0' wp=rw
`-+- policy='service-time 0' prio=1 status=active
|- 0:0:0:0 sdc 8:32 active ready running
|- 0:0:4:0 sdf 8:80 active ready running
|- 11:0:0:0 sdi 8:128 active ready running
`- 11:0:4:0 sdl 8:176 active ready running


The disk ids (B1D0, B1D1, B1D2) do not show up in Proxmox gui, only the paths (sd*). The multipath disks cannot be used!

 

[Stoiko Ivanov]

The disk GUI is meant for creating storages on simple local disks - it could not distinguish between a manually setup multipath or whether the disk is already used somewhere apart from PVE

you need to manually create the storage technology you wish to use with this node (if it's meant to be used in a clustered environment, which shares the SAS disks probably regular LVM (not lvmthin)) and then add that to the storage.cfg (GUI->Datacenter->Storage (not the menu of the node))

https://pve.proxmox.com/pve-docs/chapter-pvesm.html

i hope this helps!

 

[SamCarson]

Despite 35+ years Windows experience I am a complete Linux novice. Will you show me how the get Proxmox to use the multipath disks as local ZFS storage for ISO image, Disk image, Container template, and Container files. I will also need to pass some multipath physical disks to a virtual machine. Should the multipath disk or the 4 physical disks (for the multipath) be be passed to the VM. The final configuration will contain 14 HDDs.
Any help will be greatly appreciated.

 

[Stoiko Ivanov]

Please make sure to read the documentation regarding the different storage-plugins and their capabilities.
In this case - if you want to use the disks on multiple nodes as shared storage - it is not supported to format them with ZFS.
(it could work fine in a single node environment)

Check the output of `lsblk` it should give you a good overview of how the disks are called.

creating a zpool then works via command line - see `man zpool` (or one of the many howtos on the internet)
the created zpool then needs to be added to PVE's storage definition (can be done via GUI->Datacenter->Storage->Add->ZFS)

I hope this helps to get you started!