Proxmox 8.2.2 not detecting Huawei ES3000 NVMe

[sergeiwaigant]

Hi all.

I am in the process of moving all the different hardware servers, that I am still running form ESXi to Proxmox.
So far I was able to migrate 3 different hardware types and I am stuck with my Huawei Tecal RH5885 V2 server with an installed Huawei Tecal ES3000 (PCIe NVMe SSD-Card).
In ESXi I had just to download the driver and install them... finished story. But for that drive, I am only able to find the driver up to Debian 9 on the Huawei support site...

On the freshly installed Proxmox 8.2.2 I am not able to see the disk, the other USB SSD (which is hosting the system partitions) and the RAID10 at LSI MegaRAID SAS 9270-8I are working fine.
Please find attached the output of lspci for further reference.

Maybe someone can make a clue out of it and guide me to a solution...

dmesg -H | grep -i nvme
[  +0.000004] megaraid_sas 0000:05:00.0: NVMe passthru support  : No
[May17 22:59] nvme nvme0: pci function 0000:01:00.0
[  +0.001817] nvme nvme0: Device not ready; aborting initialisation, CSTS=0xc

lspci -v | grep -i Hua
01:00.0 Mass storage controller: Huawei Technologies Co., Ltd. Device 0007
        Subsystem: Huawei Technologies Co., Ltd. Device 0007

lspci -nn | grep -i hua
01:00.0 Mass storage controller [0180]: Huawei Technologies Co., Ltd. Device [19e5:0007]

lsblk
NAME               MAJ:MIN RM   SIZE RO TYPE MOUNTPOINTS
sda                  8:0    0   1.8T  0 disk
└─sda5               8:5    0   1.8T  0 part
sdb                  8:16   0 931.5G  0 disk
├─sdb1               8:17   0  1007K  0 part
├─sdb2               8:18   0     1G  0 part /boot/efi
└─sdb3               8:19   0 930.5G  0 part
  ├─pve-swap       252:0    0     8G  0 lvm  [SWAP]
  ├─pve-root       252:1    0    96G  0 lvm  /
  ├─pve-data_tmeta 252:2    0   8.1G  0 lvm
  │ └─pve-data     252:4    0 794.3G  0 lvm
  └─pve-data_tdata 252:3    0 794.3G  0 lvm
    └─pve-data     252:4    0 794.3G  0 lvm
sr0                 11:0    1  1024M  0 rom

00:1f.2 SATA controller: Intel Corporation 82801JI (ICH10 Family) SATA AHCI Controller (prog-if 01 [AHCI 1.0])
        Subsystem: Intel Corporation 82801JI (ICH10 Family) SATA AHCI Controller
        Flags: bus master, 66MHz, medium devsel, latency 0, IRQ 74, IOMMU group 30
        I/O ports at 9130 [size=8]
        I/O ports at 9120 [size=4]
        I/O ports at 9110 [size=8]
        I/O ports at 9100 [size=4]
        I/O ports at 9020 [size=32]
        Memory at cf221000 (32-bit, non-prefetchable) [size=2K]
        Capabilities: [80] MSI: Enable+ Count=16/16 Maskable- 64bit-
        Capabilities: [70] Power Management version 3
        Capabilities: [a8] SATA HBA v1.0
        Capabilities: [b0] PCI Advanced Features
        Kernel driver in use: ahci
        Kernel modules: ahci

01:00.0 Mass storage controller: Huawei Technologies Co., Ltd. Device 0007
        Subsystem: Huawei Technologies Co., Ltd. Device 0007
        Physical Slot: 1
        Flags: fast devsel, IRQ 24, IOMMU group 31
        Memory at ce000000 (32-bit, non-prefetchable) [size=2M]
        Capabilities: [40] Power Management version 3
        Capabilities: [48] MSI: Enable- Count=1/1 Maskable- 64bit+
        Capabilities: [60] Express Endpoint, MSI 00
        Capabilities: [9c] MSI-X: Enable- Count=8 Masked-
        Capabilities: [100] Device Serial Number 00-00-00-01-01-00-0a-35
        Capabilities: [10c] Advanced Error Reporting

05:00.0 RAID bus controller: Broadcom / LSI MegaRAID SAS 2208 [Thunderbolt] (rev 05)
        Subsystem: Broadcom / LSI MegaRAID SAS 9270-8i
        Flags: bus master, fast devsel, latency 0, IRQ 28, IOMMU group 32
        I/O ports at 6000 [size=256]
        Memory at cf160000 (64-bit, non-prefetchable) [size=16K]
        Memory at cf100000 (64-bit, non-prefetchable) [size=256K]
        Expansion ROM at cf140000 [disabled] [size=128K]
        Capabilities: [50] Power Management version 3
        Capabilities: [68] Express Endpoint, MSI 00
        Capabilities: [d0] Vital Product Data
        Capabilities: [a8] MSI: Enable- Count=1/1 Maskable- 64bit+
        Capabilities: [c0] MSI-X: Enable+ Count=16 Masked-
        Capabilities: [100] Advanced Error Reporting
        Capabilities: [1e0] Secondary PCI Express
        Capabilities: [1c0] Power Budgeting <?>
        Capabilities: [190] Dynamic Power Allocation <?>
        Capabilities: [148] Alternative Routing-ID Interpretation (ARI)
        Kernel driver in use: megaraid_sas
        Kernel modules: megaraid_sas

hostnamectl
 Static hostname: pve01
       Icon name: computer-desktop
         Chassis: desktop ️
      Machine ID: 6a26527996de4c38bd0c1077f00d533a
         Boot ID: db5170c4ec5b4d3280ab2f473d8fb70a
Operating System: Debian GNU/Linux 12 (bookworm) 
          Kernel: Linux 6.8.4-3-pve
    Architecture: x86-64
 Hardware Vendor: HUAWEI TECHNOLOGIES CO.,LTD.
  Hardware Model: RH5885 V2
Firmware Version: RGPUC-BIOS-V035

Best Regards
Sergei

 

[bbgeek17]

I'd try vanilla Debian12 installation. If you can see your hardware, then you can install PVE as packages on top of it.
https://pve.proxmox.com/wiki/Install_Proxmox_VE_on_Debian_12_Bookworm

Good luck

---

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox

 

[zzz09700]

My guess is that drive is not NVME compliant and uses some custom protocol on top of PCIe (just like your regular GPU, sound card, etc) and would require a specific driver (which, obviously won't be in the kernel tree since its a pretty uncommon device).

If you can't find driver for Debian 12 you could be out of luck. Try contact Huawei and nagging to make you a driver for Debian 12 maybe?

 

[zzz09700]

I've seen this type of devices before, that was one with onboard raid-10. Basically it was a custom raid controller + 4 SSD combined into one single board and present it self to the OS as "Mass storage controller" (just like the one OP has).

 

[zzz09700]

Just Googled the card.
If what OP has is the one without V2/3/4/5 postfix then the card is not a NVME SSD. It's a "High Performance PCIe SSD Card".
In fact in the days of that card (2012), NVME was not really a thing and practically no MB supports that.

So no driver, no go.

 

[sergeiwaigant]

Thank you very much guys!
Then I will just use that card with PCI passthrough in some VM and try to get some alternative SSD storage.

Can you guys recommend any NVMe PCIe Adapter?
Since the server is quite old I only have 2.5" SAS slots and 8x PCIe x8 slots.
I am thinking to get some NVMe drives and put them on a PCIe adapter ... but dont know how Debian / Proxmox will recognize them.

 

[sergeiwaigant]

I am thinking to order 2x the following and put it into a ZFS RAID1... anyone knows if that would work?

https://www.galaxus.de/en/s1/product/crucial-p3-plus-4000-gb-m2-2280-ssd-21575426?ip=Crucial+nvme

https://www.galaxus.de/en/s1/produc...e-ssd-pcie-4-host-storage-controller-15988460

 

[leesteken]

I am thinking to order 2x the following and put it into a ZFS RAID1... anyone knows if that would work?

https://www.galaxus.de/en/s1/product/crucial-p3-plus-4000-gb-m2-2280-ssd-21575426?ip=Crucial+nvme

That looks like a QLC drive. Please search the forum for QLC and ZFS and you'll find people who cannot believe that performance is worse than a HDD and that they wasted their money. Those threads usually also contain suggestions for enterprise SSDs with PLP (for fast sync writes).

 

[sergeiwaigant]

Thank you very much for the hint... checked it now also and its a QLC as you said.
Since this is "just" a test system to setup some k3s cluster and playground for PoCs it might be ok from performance perspective.
Toms Hardware tested the drive and is reporting 1.2Gbps write which is fair for the price I guess.
2TB Performance Results - Crucial P3 Plus SSD Review: Capacity on the Cheap - Page 2 | Tom's Hardware (tomshardware.com)

But the write cycles are a concern of course when it comes to QLC...
Going for SLC or MLC drives for a PoC system would be over the top from cost perspective.

So the way in between with TLC would be fair enough I think... at least its giving up 1000 to 3000 write cycles compared to 500 to 1000 with QLC.
The price is close to the same as the Crucial P3 Plus with QLC anyway but with better performance.
Lexar NM790 (4000 GB, M.2 2280) - buy at Galaxus
4TB Performance Results - Lexar NM790 SSD Review: A Pleasant Surprise - Page 2 | Tom's Hardware (tomshardware.com)

Thank you so much guys, the quick response and support is very apprieciated and I will provide back reports about how it went.