vfio-pci used when it shouldnt

K

kromberg

Active Member
Nov 24, 2021
92
6
28
54
After an update today, promox is assigning/using hr vfio-pci driver when it should. I have a NIC that it is doing this to.

Code: 
8c:00.0 0200: 15b3:1007
        Subsystem: 15b3:0006
        Physical Slot: 7
        Flags: fast devsel, IRQ 901, NUMA node 1, IOMMU group 3
        Memory at fba00000 (64-bit, non-prefetchable) [disabled] [size=1M]
        Memory at e9800000 (64-bit, prefetchable) [disabled] [size=8M]
        Expansion ROM at fb900000 [disabled] [size=1M]
        Capabilities: [40] Power Management version 3
        Capabilities: [48] Vital Product Data
        Capabilities: [9c] MSI-X: Enable- Count=128 Masked-
        Capabilities: [60] Express Endpoint, MSI 00
        Capabilities: [c0] Vendor Specific Information: Len=18 <?>
        Capabilities: [100] Alternative Routing-ID Interpretation (ARI)
        Capabilities: [148] Device Serial Number e4-1d-2d-03-00-2f-f8-50
        Capabilities: [108] Single Root I/O Virtualization (SR-IOV)
        Capabilities: [154] Advanced Error Reporting
        Capabilities: [18c] Secondary PCI Express
        Kernel driver in use: vfio-pci
        Kernel modules: mlx4_core

The /etc/modprobe.d/vfio.conf is empty. I have looked around and I do not see where/what is telling proxmox should be using the vfio-pci driver. I have a same second NIC where the correct driver is being assigned:

Code: 
8a:00.0 0200: 15b3:1007
        Subsystem: 15b3:0006
        Physical Slot: 9
        Flags: bus master, fast devsel, latency 0, IRQ 39, NUMA node 1, IOMMU group 2
        Memory at fbc00000 (64-bit, non-prefetchable) [size=1M]
        Memory at d9000000 (64-bit, prefetchable) [size=8M]
        Expansion ROM at fbb00000 [disabled] [size=1M]
        Capabilities: [40] Power Management version 3
        Capabilities: [48] Vital Product Data
        Capabilities: [9c] MSI-X: Enable+ Count=128 Masked-
        Capabilities: [60] Express Endpoint, MSI 00
        Capabilities: [c0] Vendor Specific Information: Len=18 <?>
        Capabilities: [100] Alternative Routing-ID Interpretation (ARI)
        Capabilities: [148] Device Serial Number e4-1d-2d-03-00-de-a5-10
        Capabilities: [108] Single Root I/O Virtualization (SR-IOV)
        Capabilities: [154] Advanced Error Reporting
        Capabilities: [18c] Secondary PCI Express
        Kernel driver in use: mlx4_core
        Kernel modules: mlx4_core
 
Hmmmm...... just noticed this:

Code: 
8b:00.0 0200: 8086:151c (rev 01)
        Subsystem: 8086:a03c
        Physical Slot: 6
        Flags: bus master, fast devsel, latency 0, IRQ 56, NUMA node 1, IOMMU group 3
        Memory at fae00000 (64-bit, non-prefetchable) [size=2M]
        I/O ports at f020 [size=32]
        Memory at fb204000 (64-bit, non-prefetchable) [size=16K]
        Expansion ROM at fb100000 [disabled] [size=1M]
        Capabilities: [40] Power Management version 3
        Capabilities: [50] MSI: Enable- Count=1/1 Maskable+ 64bit+
        Capabilities: [70] MSI-X: Enable+ Count=64 Masked-
        Capabilities: [a0] Express Endpoint, MSI 00
        Capabilities: [e0] Vital Product Data
        Capabilities: [100] Advanced Error Reporting
        Capabilities: [140] Device Serial Number 00-1b-21-ff-ff-d7-54-1a
        Capabilities: [150] Alternative Routing-ID Interpretation (ARI)
        Capabilities: [160] Single Root I/O Virtualization (SR-IOV)
        Kernel driver in use: vfio-pci
        Kernel modules: ixgbe

8b:00 is a NIC port I am passing through to a VM. I see 8b:00 is in IOMMU group 3, is that why 8c:00 is getting assigned fvio-pci because it is also in group 3? Is there a way to force 8c:00 to use module mlx4_core rather than fvio-pci?
 
You must log in or register to reply here.
Top Bottom
Back