only one of four nvme detected (duplicate cntlid)

[Quickly]

Hello.
I am trying to install ProxMox 6.x (plus dist-upgrade) on a HP ProLiant ML350p Gen8.

In the process I use:
1x Raid for OS (2x 120GB) - Ok
4x NVMe on controller for ZFS - Error

# Problem
Only 1 of the 4x NVMe is mounted.

Here it stops with me really totally.
There are 4x ONE NVMe single adapter.
So nothing with Bifructuatiuon.

Can anyone Help me please!

root@proxmox1:~# dmesg | grep -i nvme
[    2.490263] nvme nvme0: pci function 0000:07:00.0
[    2.490400] nvme nvme1: pci function 0000:0d:00.0
[    2.490677] nvme nvme2: pci function 0000:2a:00.0
[    2.490811] nvme nvme3: pci function 0000:24:00.0
[    2.597059] nvme nvme1: Duplicate cntlid 0 with nvme0, rejecting
[    2.597134] nvme nvme1: Removing after probe failure status: -22
[    2.597140] nvme nvme3: Duplicate cntlid 0 with nvme0, rejecting
[    2.597225] nvme nvme3: Removing after probe failure status: -22
[    2.597229] nvme nvme2: Duplicate cntlid 0 with nvme0, rejecting
[    2.597309] nvme nvme2: Removing after probe failure status: -22
[    2.608279] nvme nvme0: 64/0/0 default/read/poll queues
[    2.626451]  nvme0n1: p1 p2

 

[apoc]

Only 1 of the 4x NVMe is mounted.

This reminds me of the early days of NVMe-devices (which would likely fit to your Gen8-Server).
We experienced it with multiple vendors (NVMe-flash-units) which were not correctly filling the serial number field in the SCSI vendor codes.
A lot of solutions do rely on these fields so we needed firmware-updates on the NVMe flash-devices as well as Bios-Updates IIRC.

Check if your vendor provides FW-updates.