add Infiniband kernel module to proxmox

[lessfoobar]

I'm missing the Infiniband kernel module for my Qlogic IB card.

lspci | grep -i infiniband
07:00.0 InfiniBand: QLogic Corp. IBA7322 QDR InfiniBand HCA (rev 02)

Are there a prebuild modules to add it in the kernel in proxmox? Do I have to build the kmod myself?

dmesg | grep qib
[ 12.352184] ib_qib 0000:07:00.0: Unable to enable pcie error reporting: -5
[ 12.352753] ib_qib 0000:07:00.0: DCA enabled
[ 12.352756] ib_qib 0000:07:00.0: set notifier irq 140 rcv 0 notify 00000000df00ca29
[ 12.352760] ib_qib 0000:07:00.0: set notifier irq 141 rcv 0 notify 000000003a959b07
[ 12.352762] ib_qib 0000:07:00.0: set notifier irq 142 rcv 0 notify 00000000898beee0
[ 12.352764] ib_qib 0000:07:00.0: set notifier irq 143 rcv 0 notify 000000007b98a1f5
[ 12.352766] ib_qib 0000:07:00.0: set notifier irq 144 rcv 0 notify 00000000535e852e
[ 12.352768] ib_qib 0000:07:00.0: set notifier irq 145 rcv 0 notify 000000002c5d5afe
[ 12.352770] ib_qib 0000:07:00.0: set notifier irq 148 rcv 1 notify 000000005fc84d43
[ 12.352771] ib_qib 0000:07:00.0: set notifier irq 149 rcv 1 notify 00000000f63ce0e1
[ 13.211782] ib_qib 0000:07:00.0: qib0: Reserving QPNs from 0x656b78 to 0x656b78 for non-verbs use
[ 13.252703] ib_qib 0000:07:00.0: qib0: Registration with rdmavt done.
[ 15.273494] ib_qib 0000:07:00.0: qib0: IB0:2 QSFP byte0 is 0x11, S/B 0x0C/D
[ 15.329492] ib_qib 0000:07:00.0: qib0: IB0:2 QSFP ID byte is 0x11, S/B 0x0C/D
[ 15.389491] ib_qib 0000:07:00.0: qib0: IB0:1 QSFP byte0 is 0x11, S/B 0x0C/D
[ 15.445491] ib_qib 0000:07:00.0: qib0: IB0:1 QSFP ID byte is 0x11, S/B 0x0C/D
[ 20.931632] ib_qib IB0:2 Turning LOS off
[ 23.012766] ib_qib IB0:1 Turning LOS off

 

[zelenij]

Having the same problem after trying to upgrade from 6 to 7

has there been any solution?

dmesg | grep -i ib_qib | tail -25
[ 9.272726] ib_qib 0000:88:00.0: Unable to enable pcie error reporting: -5
[ 9.613637] ib_qib 0000:88:00.0: qib0: Reserving QPNs from 0x656b78 to 0x656b78 for non-verbs use
[ 9.635270] ib_qib 0000:88:00.0: qib0: Registration with rdmavt done.
[ 9.722911] ib_qib 0000:88:00.0: DCA enabled
[ 9.722921] ib_qib 0000:88:00.0: set notifier irq 141 rcv 0 notify 000000000ad0ba6f
[ 9.722928] ib_qib 0000:88:00.0: set notifier irq 142 rcv 0 notify 0000000004cdc2bd
[ 9.722932] ib_qib 0000:88:00.0: set notifier irq 143 rcv 0 notify 000000000305ea8e
[ 9.722936] ib_qib 0000:88:00.0: set notifier irq 145 rcv 1 notify 00000000a6973348
[ 9.781564] ib_qib 0000:88:00.0 ibs4: renamed from ib0
[ 9.817072] Modules linked in: ib_srp(+) scsi_transport_srp intel_rapl_msr(+) intel_rapl_common sb_edac rpcrdma ib_umad rdma_ucm ib_ipoib x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel kvm irqbypass crct10dif_pclmul ghash_clmulni_intel aesni_intel crypto_simd cryptd rapl intel_cstate ib_qib pcspkr serio_raw rdmavt mgag200 ib_uverbs drm_kms_helper input_leds joydev cec rc_core i2c_algo_bit fb_sys_fops syscopyarea sysfillrect sysimgblt iomemory_vsl(O+) hpilo ioatdma(+) dca acpi_ipmi ipmi_si(+) ipmi_devintf ipmi_msghandler mac_hid acpi_power_meter zfs(PO) zunicode(PO) zzstd(O) zlua(O) zavl(PO) icp(PO) zcommon(PO) znvpair(PO) spl(O) vhost_net vhost vhost_iotlb tap ib_iser rdma_cm iw_cm ib_cm ib_core iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi drm sunrpc ip_tables x_tables autofs4 raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor hid_generic usbkbd usbmouse usbhid hid raid6_pq raid1 raid0 multipath linear crc32_pclmul psmouse xhci_pci
[ 16.013059] ib_qib IB0:1 Turning LOS off
[ 16.121716] ib_qib 0000:88:00.0: IB0:1 got a lid: 0x6

dmesg
[ 175.168561] scsi host1: ib_srp: Failed to map data (-22)
[ 175.182646] scsi host1: ib_srp: Failed to map data (-22)
[ 175.198652] scsi host1: ib_srp: Failed to map data (-22)
[ 175.214649] scsi host1: ib_srp: Failed to map data (-22)
[ 175.230658] scsi host1: ib_srp: Failed to map data (-22)
[ 175.246659] scsi host1: ib_srp: Failed to map data (-22)
[ 175.247915] sd 1:0:0:0: [sdb] tag#56 FAILED Result: hostbyte=DID_ERROR driverbyte=DRIVER_OK cmd_age=0s
[ 175.247925] sd 1:0:0:0: [sdb] tag#56 CDB: Read(16) 88 00 00 00 00 00 00 00 00 00 00 00 01 00 00 00
[ 175.247942] blk_update_request: I/O error, dev sdb, sector 0 op 0x0READ) flags 0x0 phys_seg 8 prio class 0

most likely, that the driver, that comes with PVE 7 (and maybe with debian) is not correct.
Hardware problems excluded.
On buster with 6.4 everything works fine!

PS:
uname -a
Linux nodexx 5.13.19-6-pve #1 SMP PVE 5.13.19-15 (Tue, 29 Mar 2022 15:59:50 +0200) x86_64 GNU/Linux

 

[lessfoobar]

Unfortunately no, I've scrapped that idea and sold my cards and switch