Hello,
I have a hpe dl20 gen11 with proxmox 9.
As this server don't support bifurcation i get an axagon pcem2-nd pcie. I install two nvme and configure a zfspool in proxmox, late july.
But now, after some time i get a warning that the pool is degrated because one of the disk are missing.
In bios and in the boot menu the two disk are visible, in proxmox i still only get one. I try remove one of this from the slot and boot to see if the missing disk is detected, but no luck.
I dig into the boot log and get this information...
Then i try the card in other computer (with windows) and both nvme are detected... the last test i did was using two different nvmes in the proxmox server and then the two nvmes are again detected...
So i don't know what happen because when i install the proxmox the both disk are detected and i'm able to create a zpool and then after some time (i do all the pve upgrades) i lose one nvme... but if i use two different brands it works...
Anyone can help?
I have a hpe dl20 gen11 with proxmox 9.
As this server don't support bifurcation i get an axagon pcem2-nd pcie. I install two nvme and configure a zfspool in proxmox, late july.
But now, after some time i get a warning that the pool is degrated because one of the disk are missing.
In bios and in the boot menu the two disk are visible, in proxmox i still only get one. I try remove one of this from the slot and boot to see if the missing disk is detected, but no luck.
I dig into the boot log and get this information...
Code:
Oct 30 10:56:07 pvemmi kernel: pci 0000:05:08.0: bridge window [mem 0xd5d00000-0xd5dfffff]
Oct 30 10:56:07 pvemmi kernel: pci 0000:05:08.0: enabling Extended Tags
Oct 30 10:56:07 pvemmi kernel: pci 0000:05:08.0: PME# supported from D0 D3hot D3cold
Oct 30 10:56:07 pvemmi kernel: pci 0000:04:00.0: PCI bridge to [bus 05-07]
Oct 30 10:56:07 pvemmi kernel: pci 0000:05:00.0: bridge configuration invalid ([bus 06-06]), reconfiguring
Oct 30 10:56:07 pvemmi kernel: pci 0000:07:00.0: [c0a9:5426] type 00 class 0x010802 PCIe Endpoint
Oct 30 10:56:07 pvemmi kernel: pci 0000:07:00.0: BAR 0 [mem 0xd5d00000-0xd5d03fff 64bit]
Oct 30 10:56:07 pvemmi kernel: pci 0000:07:00.0: 31.504 Gb/s available PCIe bandwidth, limited by 8.0 GT/s PCIe x4 link at 0000:00:01.0 (capable of 63.012 Gb/s with 16.0 GT/s PCIe x4 link)
Oct 30 10:56:07 pvemmi kernel: pci 0000:05:08.0: PCI bridge to [bus 07]
Oct 30 10:56:07 pvemmi kernel: pci_bus 0000:08: busn_res: can not insert [bus 08-07] under [bus 05-07] (conflicts with (null) [bus 05-07])
Oct 30 10:56:07 pvemmi kernel: pci 0000:05:00.0: PCI bridge to [bus 08-07]
Oct 30 10:56:07 pvemmi kernel: pci_bus 0000:08: busn_res: [bus 08-07] end is updated to 08
Oct 30 10:56:07 pvemmi kernel: pci_bus 0000:08: busn_res: can not insert [bus 08] under [bus 05-07] (conflicts with (null) [bus 05-07])
Oct 30 10:56:07 pvemmi kernel: pci 0000:05:00.0: devices behind bridge are unusable because [bus 08] cannot be assigned for them
Oct 30 10:56:07 pvemmi kernel: pci 0000:04:00.0: bridge has subordinate 07 but max busn 08
Oct 30 10:56:07 pvemmi kernel: pci 0000:01:00.0: [14e4:1657] type 00 class 0x020000 PCIe Endpoint
Oct 30 10:56:07 pvemmi kernel: pci 0000:01:00.0: BAR 0 [mem 0x40000b0000-0x40000bffff 64bit pref]
Oct 30 10:56:07 pvemmi kernel: pci 0000:01:00.0: BAR 2 [mem 0x40000a0000-0x40000affff 64bit pref]
Oct 30 10:56:07 pvemmi kernel: pci 0000:01:00.0: BAR 4 [mem 0x4000090000-0x400009ffff 64bit pref]
Oct 30 10:56:07 pvemmi kernel: pci 0000:01:00.0: ROM [mem 0x00000000-0x0003ffff pref]
Oct 30 10:56:07 pvemmi kernel: pci 0000:01:00.0: PME# supported from D0 D3hot D3cold
Oct 30 10:56:07 pvemmi kernel: pci 0000:01:00.0: 8.000 Gb/s available PCIe bandwidth, limited by 5.0 GT/s PCIe x2 link at 0000:00:1c.0 (capable of 16.000 Gb/s with 5.0 GT/s PCIe x4 link)
Oct 30 10:56:07 pvemmi kernel: pci 0000:01:00.1: [14e4:1657] type 00 class 0x020000 PCIe Endpoint
Oct 30 10:56:07 pvemmi kernel: pci 0000:01:00.1: BAR 0 [mem 0x4000080000-0x400008ffff 64bit pref]
Oct 30 10:56:07 pvemmi kernel: pci 0000:01:00.1: BAR 2 [mem 0x4000070000-0x400007ffff 64bit pref]
Oct 30 10:56:07 pvemmi kernel: pci 0000:01:00.1: BAR 4 [mem 0x4000060000-0x400006ffff 64bit pref]
Oct 30 10:56:07 pvemmi kernel: pci 0000:01:00.1: ROM [mem 0x00000000-0x0003ffff pref]
Oct 30 10:56:07 pvemmi kernel: pci 0000:01:00.1: PME# supported from D0 D3hot D3cold
Oct 30 10:56:07 pvemmi kernel: pci 0000:01:00.2: [14e4:1657] type 00 class 0x020000 PCIe Endpoint
Oct 30 10:56:07 pvemmi kernel: pci 0000:01:00.2: BAR 0 [mem 0x4000050000-0x400005ffff 64bit pref]
Oct 30 10:56:07 pvemmi kernel: pci 0000:01:00.2: BAR 2 [mem 0x4000040000-0x400004ffff 64bit pref]
Oct 30 10:56:07 pvemmi kernel: pci 0000:01:00.2: BAR 4 [mem 0x4000030000-0x400003ffff 64bit pref]
Oct 30 10:56:07 pvemmi kernel: pci 0000:01:00.2: ROM [mem 0x00000000-0x0007ffff pref]
Oct 30 10:56:07 pvemmi kernel: pci 0000:01:00.2: PME# supported from D0 D3hot D3cold
Oct 30 10:56:07 pvemmi kernel: pci 0000:01:00.3: [14e4:1657] type 00 class 0x020000 PCIe Endpoint
Oct 30 10:56:07 pvemmi kernel: pci 0000:01:00.3: BAR 0 [mem 0x4000020000-0x400002ffff 64bit pref]
Oct 30 10:56:07 pvemmi kernel: pci 0000:01:00.3: BAR 2 [mem 0x4000010000-0x400001ffff 64bit pref]
Oct 30 10:56:07 pvemmi kernel: pci 0000:01:00.3: BAR 4 [mem 0x4000000000-0x400000ffff 64bit pref]
Oct 30 10:56:07 pvemmi kernel: pci 0000:01:00.3: ROM [mem 0x00000000-0x0007ffff pref]
Oct 30 10:56:07 pvemmi kernel: pci 0000:01:00.3: PME# supported from D0 D3hot D3cold
Oct 30 10:56:07 pvemmi kernel: pci 0000:00:1c.0: PCI bridge to [bus 01]Then i try the card in other computer (with windows) and both nvme are detected... the last test i did was using two different nvmes in the proxmox server and then the two nvmes are again detected...
So i don't know what happen because when i install the proxmox the both disk are detected and i'm able to create a zpool and then after some time (i do all the pve upgrades) i lose one nvme... but if i use two different brands it works...
Anyone can help?