Isolate USB controller on 8845HS

K

karypid

Active Member
Mar 7, 2021
35
10
28
47
Hello,

I have a system with Ryzen 8845HS and the following IOMMU groups:

Code: 
...

├──────────┼────────┼──────────────┼────────────┼────────┼───────────────────────────────────────────────────────────┼──────┼──────────────────┼───────────────────────┼──────────────────┼────────────────────────────────────────┼────────────────────────────────────────┤
│ 0x0c0330 │ 0x15b9 │ 0000:c7:00.3 │         25 │ 0x1022 │                                                           │      │ 0x15b9           │                       │ 0x1022           │ Advanced Micro Devices, Inc. [AMD]     │ Advanced Micro Devices, Inc. [AMD]     │
├──────────┼────────┼──────────────┼────────────┼────────┼───────────────────────────────────────────────────────────┼──────┼──────────────────┼───────────────────────┼──────────────────┼────────────────────────────────────────┼────────────────────────────────────────┤
│ 0x0c0330 │ 0x15ba │ 0000:c7:00.4 │         26 │ 0x1022 │                                                           │      │ 0x15b9           │                       │ 0x1022           │ Advanced Micro Devices, Inc. [AMD]     │ Advanced Micro Devices, Inc. [AMD]     │
├──────────┼────────┼──────────────┼────────────┼────────┼───────────────────────────────────────────────────────────┼──────┼──────────────────┼───────────────────────┼──────────────────┼────────────────────────────────────────┼────────────────────────────────────────┤
│ 0x0c0330 │ 0x15c0 │ 0000:c9:00.3 │         32 │ 0x1022 │                                                           │      │ 0x15c0           │                       │ 0x1022           │ Advanced Micro Devices, Inc. [AMD]     │ Advanced Micro Devices, Inc. [AMD]     │
├──────────┼────────┼──────────────┼────────────┼────────┼───────────────────────────────────────────────────────────┼──────┼──────────────────┼───────────────────────┼──────────────────┼────────────────────────────────────────┼────────────────────────────────────────┤
│ 0x0c0330 │ 0x15c1 │ 0000:c9:00.4 │         33 │ 0x1022 │                                                           │      │ 0x15c1           │                       │ 0x1022           │ Advanced Micro Devices, Inc. [AMD]     │ Advanced Micro Devices, Inc. [AMD]     │
├──────────┼────────┼──────────────┼────────────┼────────┼───────────────────────────────────────────────────────────┼──────┼──────────────────┼───────────────────────┼──────────────────┼────────────────────────────────────────┼────────────────────────────────────────┤
│ 0x0c0340 │ 0x1668 │ 0000:c9:00.5 │         34 │ 0x1022 │ Pink Sardine USB4/Thunderbolt NHI controller #1           │      │ 0x1668           │                       │ 0x1022           │ Advanced Micro Devices, Inc. [AMD]     │ Advanced Micro Devices, Inc. [AMD]     │
├──────────┼────────┼──────────────┼────────────┼────────┼───────────────────────────────────────────────────────────┼──────┼──────────────────┼───────────────────────┼──────────────────┼────────────────────────────────────────┼────────────────────────────────────────┤
│ 0x0c0340 │ 0x1669 │ 0000:c9:00.6 │         35 │ 0x1022 │ Pink Sardine USB4/Thunderbolt NHI controller #2           │      │ 0x1669           │                       │ 0x1022           │ Advanced Micro Devices, Inc. [AMD]     │ Advanced Micro Devices, Inc. [AMD]     │
├──────────┼────────┼──────────────┼────────────┼────────┼───────────────────────────────────────────────────────────┼──────┼──────────────────┼───────────────────────┼──────────────────┼────────────────────────────────────────┼────────────────────────────────────────┤
│ 0x0c0500 │ 0x790b │ 0000:00:14.0 │         15 │ 0x1022 │ FCH SMBus Controller                                      │      │ 0x790b           │                       │ 0x1022           │ Advanced Micro Devices, Inc. [AMD]     │ Advanced Micro Devices, Inc. [AMD]     │
├──────────┼────────┼──────────────┼────────────┼────────┼───────────────────────────────────────────────────────────┼──────┼──────────────────┼───────────────────────┼──────────────────┼────────────────────────────────────────┼────────────────────────────────────────┤
│ 0x108000 │ 0x15c7 │ 0000:c7:00.2 │         24 │ 0x1022 │ Phoenix CCP/PSP 3.0 Device                                │      │ 0x15c7           │                       │ 0x1022           │ Advanced Micro Devices, Inc. [AMD]     │ Advanced Micro Devices, Inc. [AMD]     │
├──────────┼────────┼──────────────┼────────────┼────────┼───────────────────────────────────────────────────────────┼──────┼──────────────────┼───────────────────────┼──────────────────┼────────────────────────────────────────┼────────────────────────────────────────┤
│ 0x118000 │ 0x1502 │ 0000:c8:00.1 │         30 │ 0x1022 │ AMD IPU Device                                            │      │ 0x1502           │                       │ 0x1022           │ Advanced Micro Devices, Inc. [AMD]     │ Advanced Micro Devices, Inc. [AMD]     │
├──────────┼────────┼──────────────┼────────────┼────────┼───────────────────────────────────────────────────────────┼──────┼──────────────────┼───────────────────────┼──────────────────┼────────────────────────────────────────┼────────────────────────────────────────┤
│ 0x130000 │ 0x14ec │ 0000:c8:00.0 │         29 │ 0x1022 │ Phoenix Dummy Function                                    │      │ 0x14ec           │                       │ 0x1022           │ Advanced Micro Devices, Inc. [AMD]     │ Advanced Micro Devices, Inc. [AMD]     │
├──────────┼────────┼──────────────┼────────────┼────────┼───────────────────────────────────────────────────────────┼──────┼──────────────────┼───────────────────────┼──────────────────┼────────────────────────────────────────┼────────────────────────────────────────┤
│ 0x130000 │ 0x14ec │ 0000:c9:00.0 │         31 │ 0x1022 │ Phoenix Dummy Function                                    │      │ 0x14ec           │                       │ 0x1022           │ Advanced Micro Devices, Inc. [AMD]     │ Advanced Micro Devices, Inc. [AMD]     │
└──────────┴────────┴──────────────┴────────────┴────────┴───────────────────────────────────────────────────────────┴──────┴──────────────────┴───────────────────────┴──────────────────┴────────────────────────────────────────┴────────────────────────────────────────┘

Critically, I noticed that the following PCI devices are all on individual isolated groups:

Code: 
Sep 28 16:02:37 kernel: pci 0000:c9:00.0: Adding to iommu group 31
Sep 28 16:02:37 kernel: pci 0000:c9:00.3: Adding to iommu group 32
Sep 28 16:02:37 kernel: pci 0000:c9:00.4: Adding to iommu group 33
Sep 28 16:02:37 kernel: pci 0000:c9:00.5: Adding to iommu group 34
Sep 28 16:02:37 kernel: pci 0000:c9:00.6: Adding to iommu group 35

I was trying to pass-through one or two of these devices to a guest VM (or all of c9:00.x if individual is not possible) but they use the USB drivers , so if I blacklist module "xhci_hcd", then all of the USB controllers would not work, right? So I tried this /etc/modprobe.d/vfio.conf instead:

Code: 
options vfio-pci ids=1022:15c0,1022:15c1,1022:1668,1022:1669 disable_idle_d3=1
#blacklist xhci_pci
#blacklist xhci_hcd
blacklist thunderbolt

But this ends up loading with the devices attached to that controller connecting to the host anyway, even vfio seems to grab the devices:

Code: 
Sep 28 16:02:37 systemd[1]: Finished kmod-static-nodes.service - Create List of Static Device Nodes.
Sep 28 16:02:37 kernel: vfio_pci: add [1022:15c0[ffffffff:ffffffff]] class 0x000000/00000000
Sep 28 16:02:37 systemd[1]: modprobe@configfs.service: Deactivated successfully.
Sep 28 16:02:37 kernel: vfio_pci: add [1022:15c1[ffffffff:ffffffff]] class 0x000000/00000000
Sep 28 16:02:37 systemd[1]: Finished modprobe@configfs.service - Load Kernel Module configfs.
Sep 28 16:02:37 kernel: vfio_pci: add [1022:1668[ffffffff:ffffffff]] class 0x000000/00000000
Sep 28 16:02:37 systemd[1]: modprobe@drm.service: Deactivated successfully.
Sep 28 16:02:37 kernel: vfio_pci: add [1022:1669[ffffffff:ffffffff]] class 0x000000/00000000
Sep 28 16:02:37 systemd[1]: Finished modprobe@drm.service - Load Kernel Module drm.

And here is lspci with vfio module missing and xhci in use:

Code: 
c9:00.0 1300: 1022:14ec
    Subsystem: 1022:14ec
c9:00.3 0c03: 1022:15c0
    Subsystem: 1022:15c0
    Kernel driver in use: xhci_hcd
    Kernel modules: xhci_pci
c9:00.4 0c03: 1022:15c1
    Subsystem: 1022:15c1
    Kernel driver in use: xhci_hcd
    Kernel modules: xhci_pci
c9:00.5 0c03: 1022:1668
    Subsystem: 1022:1668
    Kernel driver in use: thunderbolt
    Kernel modules: thunderbolt
c9:00.6 0c03: 1022:1669
    Subsystem: 1022:1669
    Kernel driver in use: thunderbolt
    Kernel modules: thunderbolt

Is there anyway to isolate these devices only without blacklisting the xhci_pci and xhci_hcd modules?
 
You must log in or register to reply here.
Top Bottom