NVME passthrough not working - PCI config conflict?

rakali · Dec 27, 2021

I am passing 2 x NVME to my guest. One is a samsung and it works fine. The other m.2 I have just added is WD/Sandisk and cannot make it work. It is not recognised by the guest.

Tested and working well when booting a live distro on bare metal.

Is there something I am missing?

Code:

06:00.0 Non-Volatile memory controller [0108]: Sandisk Corp WD Black SN750 / PC SN730 NVMe SSD [15b7:5006] (prog-if 02 [NVM Express])
        Subsystem: Sandisk Corp WD Black 2019/PC SN750 NVMe SSD [15b7:5006]
        Flags: fast devsel, IRQ 16, NUMA node 0, IOMMU group 21
        Memory at 62a00000 (64-bit, non-prefetchable) [virtual] [size=16K]
        Memory at 62a04000 (64-bit, non-prefetchable) [virtual] [size=256]
        Capabilities: [80] Power Management version 3
        Capabilities: [90] MSI: Enable- Count=1/32 Maskable- 64bit+
        Capabilities: [b0] MSI-X: Enable- Count=65 Masked-
        Capabilities: [c0] Express Endpoint, MSI 00
        Capabilities: [100] Advanced Error Reporting
        Capabilities: [150] Device Serial Number 00-00-00-00-00-00-00-00
        Capabilities: [1b8] Latency Tolerance Reporting
        Capabilities: [300] Secondary PCI Express
        Capabilities: [900] L1 PM Substates
        Kernel modules: nvme

Code:

[    0.822917] pci 0000:06:00.0: [15b7:5006] type 00 class 0x010802
[    0.822936] pci 0000:06:00.0: reg 0x10: [mem 0x62a00000-0x62a03fff 64bit]
[    0.822961] pci 0000:06:00.0: reg 0x20: [mem 0x62a04000-0x62a040ff 64bit]
[    0.823082] pci 0000:06:00.0: 15.752 Gb/s available PCIe bandwidth, limited by 8.0 GT/s PCIe x2 link at 0000:00:1b.4 (capable of 31.504 Gb/s with 8.0 GT/s PCIe x4 link)
[    0.913909] pci 0000:06:00.0: Adding to iommu group 21
[    1.582970] nvme nvme1: pci function 0000:06:00.0
[ 1086.071892] vfio-pci 0000:06:00.0: not ready 1023ms after FLR; waiting
[ 1087.127897] vfio-pci 0000:06:00.0: not ready 2047ms after FLR; waiting
[ 1089.399909] vfio-pci 0000:06:00.0: not ready 4095ms after FLR; waiting
[ 1093.751930] vfio-pci 0000:06:00.0: not ready 8191ms after FLR; waiting
[ 1102.199988] vfio-pci 0000:06:00.0: not ready 16383ms after FLR; waiting
[ 1119.352084] vfio-pci 0000:06:00.0: not ready 32767ms after FLR; waiting
[ 1154.168258] vfio-pci 0000:06:00.0: not ready 65535ms after FLR; giving up
[ 1162.989757] vfio-pci 0000:06:00.0: enabling device (0000 -> 0002)
[ 1163.096541] vfio-pci 0000:06:00.0: vfio_ecap_init: hiding ecap 0x19@0x300
[ 1163.097242] vfio-pci 0000:06:00.0: vfio_ecap_init: hiding ecap 0x1e@0x900

rakali · Dec 27, 2021

I tried with another guest (windows) and received a lot of errors in dmesg.

Windows device manager says it cannot start the device. An I/O adapter hardware error has occurred.

Can anyone make sense of it?

Code:

[    0.822374] pci 0000:06:00.0: [15b7:5006] type 00 class 0x010802
[    0.822392] pci 0000:06:00.0: reg 0x10: [mem 0x62a00000-0x62a03fff 64bit]
[    0.822418] pci 0000:06:00.0: reg 0x20: [mem 0x62a04000-0x62a040ff 64bit]
[    0.886184] pci 0000:06:00.0: Adding to iommu group 21
[    1.563643] nvme nvme1: pci function 0000:06:00.0
[  190.879948] vfio-pci 0000:06:00.0: not ready 1023ms after FLR; waiting
[  191.935944] vfio-pci 0000:06:00.0: not ready 2047ms after FLR; waiting
[  194.175943] vfio-pci 0000:06:00.0: not ready 4095ms after FLR; waiting
[  198.527948] vfio-pci 0000:06:00.0: not ready 8191ms after FLR; waiting
[  206.975939] vfio-pci 0000:06:00.0: not ready 16383ms after FLR; waiting
[  224.639918] vfio-pci 0000:06:00.0: not ready 32767ms after FLR; waiting
[  259.455783] vfio-pci 0000:06:00.0: not ready 65535ms after FLR; giving up
[  260.542808] vfio-pci 0000:06:00.0: vfio_cap_init: hiding cap 0xff@0xff
[  260.543413] vfio-pci 0000:06:00.0: vfio_cap_init: hiding cap 0xff@0xff
[  260.544075] vfio-pci 0000:06:00.0: vfio_cap_init: hiding cap 0xff@0xff
[  260.544711] vfio-pci 0000:06:00.0: vfio_cap_init: hiding cap 0xff@0xff
[  260.545272] vfio-pci 0000:06:00.0: vfio_cap_init: hiding cap 0xff@0xff
[  260.545819] vfio-pci 0000:06:00.0: vfio_cap_init: hiding cap 0xff@0xff
[  260.546361] vfio-pci 0000:06:00.0: vfio_cap_init: hiding cap 0xff@0xff
[  260.546919] vfio-pci 0000:06:00.0: vfio_cap_init: hiding cap 0xff@0xff
[  260.547460] vfio-pci 0000:06:00.0: vfio_cap_init: hiding cap 0xff@0xff
[  260.548038] vfio-pci 0000:06:00.0: vfio_cap_init: hiding cap 0xff@0xff
[  260.548602] vfio-pci 0000:06:00.0: vfio_cap_init: hiding cap 0xff@0xff
[  260.549123] vfio-pci 0000:06:00.0: vfio_cap_init: hiding cap 0xff@0xff
[  260.549630] vfio-pci 0000:06:00.0: vfio_cap_init: hiding cap 0xff@0xff
[  260.550137] vfio-pci 0000:06:00.0: vfio_cap_init: hiding cap 0xff@0xff
[  260.550642] vfio-pci 0000:06:00.0: vfio_cap_init: hiding cap 0xff@0xff
[  260.551142] vfio-pci 0000:06:00.0: vfio_cap_init: hiding cap 0xff@0xff
[  260.551635] vfio-pci 0000:06:00.0: vfio_cap_init: hiding cap 0xff@0xff
[  260.552174] vfio-pci 0000:06:00.0: vfio_cap_init: hiding cap 0xff@0xff
[  260.552691] vfio-pci 0000:06:00.0: vfio_cap_init: hiding cap 0xff@0xff
[  260.553176] vfio-pci 0000:06:00.0: vfio_cap_init: hiding cap 0xff@0xff
[  260.553627] vfio-pci 0000:06:00.0: vfio_cap_init: hiding cap 0xff@0xff
[  260.554062] vfio-pci 0000:06:00.0: vfio_cap_init: hiding cap 0xff@0xff
[  260.554490] vfio-pci 0000:06:00.0: vfio_cap_init: hiding cap 0xff@0xff
[  260.554909] vfio-pci 0000:06:00.0: vfio_cap_init: hiding cap 0xff@0xff
[  260.555321] vfio-pci 0000:06:00.0: vfio_cap_init: hiding cap 0xff@0xff
[  260.555791] vfio-pci 0000:06:00.0: vfio_cap_init: hiding cap 0xff@0xff
[  260.556232] vfio-pci 0000:06:00.0: vfio_cap_init: hiding cap 0xff@0xff
[  260.556637] vfio-pci 0000:06:00.0: vfio_cap_init: hiding cap 0xff@0xff
[  260.557018] vfio-pci 0000:06:00.0: vfio_cap_init: hiding cap 0xff@0xff
[  260.557391] vfio-pci 0000:06:00.0: vfio_cap_init: hiding cap 0xff@0xff
[  260.557761] vfio-pci 0000:06:00.0: vfio_cap_init: hiding cap 0xff@0xff
[  260.558121] vfio-pci 0000:06:00.0: vfio_cap_init: hiding cap 0xff@0xff
[  260.558481] vfio-pci 0000:06:00.0: vfio_cap_init: hiding cap 0xff@0xff
[  260.558832] vfio-pci 0000:06:00.0: vfio_cap_init: hiding cap 0xff@0xff
[  260.559175] vfio-pci 0000:06:00.0: vfio_cap_init: hiding cap 0xff@0xff
[  260.559515] vfio-pci 0000:06:00.0: vfio_cap_init: hiding cap 0xff@0xff
[  260.559918] vfio-pci 0000:06:00.0: vfio_cap_init: hiding cap 0xff@0xff
[  260.560294] vfio-pci 0000:06:00.0: vfio_cap_init: hiding cap 0xff@0xff
[  260.560627] vfio-pci 0000:06:00.0: vfio_cap_init: hiding cap 0xff@0xff
[  260.560969] vfio-pci 0000:06:00.0: vfio_cap_init: hiding cap 0xff@0xff
[  260.561283] vfio-pci 0000:06:00.0: vfio_cap_init: hiding cap 0xff@0xff
[  260.561592] vfio-pci 0000:06:00.0: vfio_cap_init: hiding cap 0xff@0xff
[  260.561898] vfio-pci 0000:06:00.0: vfio_cap_init: hiding cap 0x0@0xff
[  260.562206] vfio-pci 0000:06:00.0: vfio_cap_init: hiding cap 0x15@0x1
[  260.562506] vfio-pci 0000:06:00.0: vfio_cap_init: PCI config conflict @0x6, was cap 0x0 now cap 0x10
[  260.562801] vfio-pci 0000:06:00.0: vfio_cap_init: PCI config conflict @0x7, was cap 0x0 now cap 0x10
[  260.563085] vfio-pci 0000:06:00.0: vfio_cap_init: PCI config conflict @0x8, was cap 0x0 now cap 0x10
[  260.563357] vfio-pci 0000:06:00.0: vfio_cap_init: PCI config conflict @0x9, was cap 0x0 now cap 0x10
[  260.563613] vfio-pci 0000:06:00.0: vfio_cap_init: PCI config conflict @0xa, was cap 0x0 now cap 0x10
[  260.563920] vfio-pci 0000:06:00.0: vfio_cap_init: PCI config conflict @0xb, was cap 0x0 now cap 0x10
[  260.564226] vfio-pci 0000:06:00.0: vfio_cap_init: PCI config conflict @0xc, was cap 0x0 now cap 0x10
[  260.564478] vfio-pci 0000:06:00.0: vfio_cap_init: PCI config conflict @0xd, was cap 0x0 now cap 0x10
[  260.564724] vfio-pci 0000:06:00.0: vfio_cap_init: PCI config conflict @0xe, was cap 0x0 now cap 0x10
[  260.564968] vfio-pci 0000:06:00.0: vfio_cap_init: PCI config conflict @0xf, was cap 0x0 now cap 0x10
[  260.565224] vfio-pci 0000:06:00.0: vfio_cap_init: PCI config conflict @0x10, was cap 0x0 now cap 0x10
[  260.565458] vfio-pci 0000:06:00.0: vfio_cap_init: PCI config conflict @0x11, was cap 0x0 now cap 0x10
[  260.565684] vfio-pci 0000:06:00.0: vfio_cap_init: PCI config conflict @0x12, was cap 0x0 now cap 0x10
[  260.565896] vfio-pci 0000:06:00.0: vfio_cap_init: PCI config conflict @0x13, was cap 0x0 now cap 0x10
[  260.566095] vfio-pci 0000:06:00.0: vfio_cap_init: PCI config conflict @0x14, was cap 0x0 now cap 0x10
[  260.566284] vfio-pci 0000:06:00.0: vfio_cap_init: PCI config conflict @0x15, was cap 0x0 now cap 0x10
[  260.566464] vfio-pci 0000:06:00.0: vfio_cap_init: PCI config conflict @0x16, was cap 0x0 now cap 0x10
[  260.566645] vfio-pci 0000:06:00.0: vfio_cap_init: PCI config conflict @0x17, was cap 0x0 now cap 0x10
[  260.566823] vfio-pci 0000:06:00.0: vfio_cap_init: PCI config conflict @0x18, was cap 0x0 now cap 0x10
[  260.567000] vfio-pci 0000:06:00.0: vfio_cap_init: PCI config conflict @0x19, was cap 0x0 now cap 0x10
[  260.567173] vfio-pci 0000:06:00.0: vfio_cap_init: PCI config conflict @0x1a, was cap 0x0 now cap 0x10
[  260.567344] vfio-pci 0000:06:00.0: vfio_cap_init: PCI config conflict @0x1b, was cap 0x0 now cap 0x10
[  260.567513] vfio-pci 0000:06:00.0: vfio_cap_init: PCI config conflict @0x1c, was cap 0x0 now cap 0x10
[  260.567712] vfio-pci 0000:06:00.0: vfio_cap_init: PCI config conflict @0x1d, was cap 0x0 now cap 0x10
[  260.567938] vfio-pci 0000:06:00.0: vfio_cap_init: PCI config conflict @0x1e, was cap 0x0 now cap 0x10
[  260.568104] vfio-pci 0000:06:00.0: vfio_cap_init: PCI config conflict @0x1f, was cap 0x0 now cap 0x10
[  260.568270] vfio-pci 0000:06:00.0: vfio_cap_init: PCI config conflict @0x20, was cap 0x0 now cap 0x10
[  260.568437] vfio-pci 0000:06:00.0: vfio_cap_init: PCI config conflict @0x21, was cap 0x0 now cap 0x10
[  260.568605] vfio-pci 0000:06:00.0: vfio_cap_init: PCI config conflict @0x22, was cap 0x0 now cap 0x10
[  260.568774] vfio-pci 0000:06:00.0: vfio_cap_init: PCI config conflict @0x23, was cap 0x0 now cap 0x10
[  260.568942] vfio-pci 0000:06:00.0: vfio_cap_init: PCI config conflict @0x24, was cap 0x0 now cap 0x10
[  260.569111] vfio-pci 0000:06:00.0: vfio_cap_init: PCI config conflict @0x25, was cap 0x0 now cap 0x10
[  260.569281] vfio-pci 0000:06:00.0: vfio_cap_init: PCI config conflict @0x26, was cap 0x0 now cap 0x10
[  260.569452] vfio-pci 0000:06:00.0: vfio_cap_init: PCI config conflict @0x27, was cap 0x0 now cap 0x10
[  260.569637] vfio-pci 0000:06:00.0: vfio_cap_init: PCI config conflict @0x28, was cap 0x0 now cap 0x10
[  260.569809] vfio-pci 0000:06:00.0: vfio_cap_init: PCI config conflict @0x29, was cap 0x0 now cap 0x10
[  260.569980] vfio-pci 0000:06:00.0: vfio_cap_init: PCI config conflict @0x2a, was cap 0x0 now cap 0x10
[  260.570153] vfio-pci 0000:06:00.0: vfio_cap_init: PCI config conflict @0x2b, was cap 0x0 now cap 0x10
[  260.570324] vfio-pci 0000:06:00.0: vfio_cap_init: PCI config conflict @0x2c, was cap 0x0 now cap 0x10
[  260.570496] vfio-pci 0000:06:00.0: vfio_cap_init: PCI config conflict @0x2d, was cap 0x0 now cap 0x10
[  260.570670] vfio-pci 0000:06:00.0: vfio_cap_init: PCI config conflict @0x2e, was cap 0x0 now cap 0x10
[  260.570844] vfio-pci 0000:06:00.0: vfio_cap_init: PCI config conflict @0x2f, was cap 0x0 now cap 0x10
[  260.571019] vfio-pci 0000:06:00.0: vfio_cap_init: PCI config conflict @0x30, was cap 0x0 now cap 0x10
[  260.571195] vfio-pci 0000:06:00.0: vfio_cap_init: PCI config conflict @0x31, was cap 0x0 now cap 0x10
[  260.571371] vfio-pci 0000:06:00.0: vfio_cap_init: PCI config conflict @0x32, was cap 0x0 now cap 0x10
[  260.571548] vfio-pci 0000:06:00.0: vfio_cap_init: PCI config conflict @0x33, was cap 0x0 now cap 0x10
[  260.571779] vfio-pci 0000:06:00.0: vfio_cap_init: PCI config conflict @0x34, was cap 0x0 now cap 0x10
[  260.572015] vfio-pci 0000:06:00.0: vfio_cap_init: PCI config conflict @0x35, was cap 0x0 now cap 0x10
[  260.572199] vfio-pci 0000:06:00.0: vfio_cap_init: PCI config conflict @0x36, was cap 0x0 now cap 0x10
[  260.572381] vfio-pci 0000:06:00.0: vfio_cap_init: PCI config conflict @0x37, was cap 0x0 now cap 0x10
[  260.572563] vfio-pci 0000:06:00.0: vfio_cap_init: PCI config conflict @0x38, was cap 0x0 now cap 0x10
[  260.572747] vfio-pci 0000:06:00.0: vfio_cap_init: PCI config conflict @0x39, was cap 0x0 now cap 0x10
[  260.572964] vfio-pci 0000:06:00.0: vfio_ecap_init: hiding ecap 0x19@0x300
[  260.573160] vfio-pci 0000:06:00.0: vfio_ecap_init: hiding ecap 0x1e@0x900
[  261.965422] vfio-pci 0000:06:00.0: can't change power state from D0 to D3hot (config space inaccessible)
[  278.465663] vfio-pci 0000:06:00.0: enabling device (0000 -> 0002)
[  278.571881] vfio-pci 0000:06:00.0: vfio_ecap_init: hiding ecap 0x19@0x300
[  278.572543] vfio-pci 0000:06:00.0: vfio_ecap_init: hiding ecap 0x1e@0x900
[  294.856028] pcieport 0000:00:1b.4: AER: Multiple Corrected error received: 0000:06:00.0
[  294.856704] vfio-pci 0000:06:00.0: PCIe Bus Error: severity=Corrected, type=Transaction Layer, (Receiver ID)
[  294.857369] vfio-pci 0000:06:00.0:   device [15b7:5006] error status/mask=0000a000/00000000
[  294.858024] vfio-pci 0000:06:00.0:    [13] NonFatalErr
[  294.858682] vfio-pci 0000:06:00.0:    [15] HeaderOF
[  294.859347] pcieport 0000:00:1b.4: AER: Multiple Corrected error received: 0000:06:00.0
[  294.860081] pcieport 0000:00:1b.4: AER: Multiple Corrected error received: 0000:06:00.0
[  294.860887] pcieport 0000:00:1b.4: AER: Multiple Corrected error received: 0000:06:00.0
[  294.861538] pcieport 0000:00:1b.4: AER: Multiple Corrected error received: 0000:06:00.0
[  294.862175] pcieport 0000:00:1b.4: AER: Multiple Corrected error received: 0000:06:00.0
[  294.862799] pcieport 0000:00:1b.4: AER: Multiple Corrected error received: 0000:06:00.0
[  294.863407] pcieport 0000:00:1b.4: AER: Multiple Corrected error received: 0000:06:00.0
[  294.864068] pcieport 0000:00:1b.4: AER: Multiple Corrected error received: 0000:06:00.0
[  294.864712] pcieport 0000:00:1b.4: AER: Multiple Corrected error received: 0000:06:00.0
[  294.865309] pcieport 0000:00:1b.4: AER: Multiple Corrected error received: 0000:06:00.0
[  294.879909] pcieport 0000:00:1b.4: AER: Multiple Corrected error received: 0000:06:00.0
[  294.880509] vfio-pci 0000:06:00.0: PCIe Bus Error: severity=Corrected, type=Transaction Layer, (Receiver ID)
[  294.881110] vfio-pci 0000:06:00.0:   device [15b7:5006] error status/mask=0000a000/00000000
[  294.881793] vfio-pci 0000:06:00.0:    [13] NonFatalErr
[  294.882476] vfio-pci 0000:06:00.0:    [15] HeaderOF
[  294.883085] pcieport 0000:00:1b.4: AER: Multiple Corrected error received: 0000:06:00.0
[  294.883793] vfio-pci 0000:06:00.0: PCIe Bus Error: severity=Corrected, type=Transaction Layer, (Receiver ID)
[  294.884430] vfio-pci 0000:06:00.0:   device [15b7:5006] error status/mask=0000a000/00000000
[  294.885023] vfio-pci 0000:06:00.0:    [13] NonFatalErr
[  294.885599] vfio-pci 0000:06:00.0:    [15] HeaderOF
[  294.886197] pcieport 0000:00:1b.4: AER: Multiple Corrected error received: 0000:06:00.0
[  294.886795] pcieport 0000:00:1b.4: AER: Multiple Corrected error received: 0000:06:00.0
[  294.887376] pcieport 0000:00:1b.4: AER: Multiple Corrected error received: 0000:06:00.0
[  294.888012] pcieport 0000:00:1b.4: AER: Multiple Corrected error received: 0000:06:00.0
[  294.888639] pcieport 0000:00:1b.4: AER: Multiple Corrected error received: 0000:06:00.0
[  294.889200] pcieport 0000:00:1b.4: AER: Multiple Corrected error received: 0000:06:00.0
[  294.889752] pcieport 0000:00:1b.4: AER: Multiple Corrected error received: 0000:06:00.0
[  294.890296] pcieport 0000:00:1b.4: AER: Multiple Corrected error received: 0000:06:00.0
[  294.890836] pcieport 0000:00:1b.4: AER: Multiple Corrected error received: 0000:06:00.0
[  294.891367] pcieport 0000:00:1b.4: AER: Multiple Corrected error received: 0000:06:00.0
[  295.919108] pcieport 0000:00:1b.4: AER: Multiple Corrected error received: 0000:06:00.0
[  295.919656] vfio-pci 0000:06:00.0: PCIe Bus Error: severity=Corrected, type=Transaction Layer, (Receiver ID)
[  295.920231] vfio-pci 0000:06:00.0:   device [15b7:5006] error status/mask=0000a000/00000000
[  295.920812] vfio-pci 0000:06:00.0:    [13] NonFatalErr
[  295.921341] vfio-pci 0000:06:00.0:    [15] HeaderOF
[  295.921878] pcieport 0000:00:1b.4: AER: Multiple Corrected error received: 0000:06:00.0
[  295.922431] pcieport 0000:00:1b.4: AER: Corrected error received: 0000:06:00.0
[  295.923013] pcieport 0000:00:1b.4: AER: Multiple Corrected error received: 0000:06:00.0
[  295.923624] pcieport 0000:00:1b.4: AER: Multiple Corrected error received: 0000:06:00.0
[  295.924209] pcieport 0000:00:1b.4: AER: Multiple Corrected error received: 0000:06:00.0
[  295.924829] pcieport 0000:00:1b.4: AER: Multiple Corrected error received: 0000:06:00.0
[  295.925373] pcieport 0000:00:1b.4: AER: Multiple Corrected error received: 0000:06:00.0
[  295.925906] pcieport 0000:00:1b.4: AER: Multiple Corrected error received: 0000:06:00.0
[  295.926426] pcieport 0000:00:1b.4: AER: Multiple Corrected error received: 0000:06:00.0

leesteken · Dec 27, 2021

It looks like the device does not reset properly. Maybe this device just does not work with passthrough.
Are there other devices in the IOMMU group? Did you use pcie_acs_override?
Can you try switching the two NVMe devices? This will give a clue about whether it is the device or the M.2 slot.
Passthrough often works with a M.2 slot connected to the CPU but not with a M.2 slot connected to the motherboard chipset. What motherboard are you using?

rakali · Dec 27, 2021

I do have pcie_acs_override

Motherboard is Z390 Aorus Ultra. It looks like the three m.2 are connected to CPU, but maybe I have it wrong?

I will have to try physically moving them around.

Code:

dmesg | grep -i iommu
[    0.000000] Command line: initrd=\EFI\proxmox\5.13.19-2-pve\initrd.img-5.13.19-2-pve root=ZFS=rpool/ROOT/pve-1 boot=zfs intel_iommu=on iommu=pt i915.enable_gvt=1 rootdelay=10 pcie_acs_override=downstream i915.alpha_support=1 video=efifb:off
[    0.000000] Warning: PCIe ACS overrides enabled; This may allow non-IOMMU protected peer-to-peer DMA
[    0.061835] Kernel command line: initrd=\EFI\proxmox\5.13.19-2-pve\initrd.img-5.13.19-2-pve root=ZFS=rpool/ROOT/pve-1 boot=zfs intel_iommu=on iommu=pt i915.enable_gvt=1 rootdelay=10 pcie_acs_override=downstream i915.alpha_support=1 video=efifb:off
[    0.061875] DMAR: IOMMU enabled
[    0.149780] DMAR-IR: IOAPIC id 2 under DRHD base  0xfed91000 IOMMU 1
[    0.823981] iommu: Default domain type: Passthrough (set via kernel command line)
[    0.881541] DMAR: IOMMU feature fl1gp_support inconsistent
[    0.881542] DMAR: IOMMU feature pgsel_inv inconsistent
[    0.881543] DMAR: IOMMU feature nwfs inconsistent
[    0.881544] DMAR: IOMMU feature pasid inconsistent
[    0.881545] DMAR: IOMMU feature eafs inconsistent
[    0.881546] DMAR: IOMMU feature prs inconsistent
[    0.881547] DMAR: IOMMU feature nest inconsistent
[    0.881547] DMAR: IOMMU feature mts inconsistent
[    0.881548] DMAR: IOMMU feature sc_support inconsistent
[    0.881549] DMAR: IOMMU feature dev_iotlb_support inconsistent
[    0.881809] pci 0000:00:00.0: Adding to iommu group 0
[    0.881817] pci 0000:00:01.0: Adding to iommu group 1
[    0.881824] pci 0000:00:01.1: Adding to iommu group 2
[    0.881829] pci 0000:00:02.0: Adding to iommu group 3
[    0.881837] pci 0000:00:12.0: Adding to iommu group 4
[    0.881846] pci 0000:00:14.0: Adding to iommu group 5
[    0.881851] pci 0000:00:14.2: Adding to iommu group 5
[    0.881858] pci 0000:00:16.0: Adding to iommu group 6
[    0.881863] pci 0000:00:17.0: Adding to iommu group 7
[    0.881875] pci 0000:00:1b.0: Adding to iommu group 8
[    0.881882] pci 0000:00:1b.4: Adding to iommu group 9
[    0.881890] pci 0000:00:1c.0: Adding to iommu group 10
[    0.881898] pci 0000:00:1c.5: Adding to iommu group 11
[    0.881905] pci 0000:00:1c.7: Adding to iommu group 12
[    0.881912] pci 0000:00:1d.0: Adding to iommu group 13
[    0.881928] pci 0000:00:1f.0: Adding to iommu group 14
[    0.881933] pci 0000:00:1f.3: Adding to iommu group 14
[    0.881939] pci 0000:00:1f.4: Adding to iommu group 14
[    0.881944] pci 0000:00:1f.5: Adding to iommu group 14
[    0.881950] pci 0000:00:1f.6: Adding to iommu group 14
[    0.881956] pci 0000:01:00.0: Adding to iommu group 15
[    0.881964] pci 0000:02:00.0: Adding to iommu group 16
[    0.881978] pci 0000:03:00.0: Adding to iommu group 17
[    0.881991] pci 0000:03:00.1: Adding to iommu group 18
[    0.881996] pci 0000:04:00.0: Adding to iommu group 19
[    0.882008] pci 0000:05:00.0: Adding to iommu group 20
[    0.882015] pci 0000:06:00.0: Adding to iommu group 21
[    0.882023] pci 0000:08:00.0: Adding to iommu group 22
[    0.882030] pci 0000:09:00.0: Adding to iommu group 23
[    0.882038] pci 0000:0a:00.0: Adding to iommu group 24
[    1.454089]     intel_iommu=on

leesteken · Dec 28, 2021

The IOMMU group information you show is useless because of the pcie_acs_override.
According to the Gigabyte website, all three M.2 storage slots are connected to the chipset. One is a little different from the other two, but nothing stands out.
Can you switch the Samsung and WD devices? This will give a clue about whether it is the device or the M.2 slot.

Search

Search

NVME passthrough not working - PCI config conflict?

rakali

Active Member

rakali

Active Member

leesteken

Distinguished Member

rakali

Active Member

leesteken

Distinguished Member