AMD iGPU passthrough won't work

jhmc93 · Apr 7, 2024

diversity said:
what specific tutorial are you referring to?

Please remember I am just soo busy that sometimes I have time to write a long wided answer like I do now. But reading up on long threads has never been my strong point if I enter as an advice giver rather than the needer of advice.

So.

Please show us your kernel commands. You use grub that is just fine.
Please output the early boot process of your host using dmesg. it can tell us if you properly integrated your kernal command line options into the boot process.

then show an lspci -k

then we talk further should I have any insights

output of lspci -k:

Code:

00:00.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Renoir/Cezanne Root Complex
    Subsystem: Advanced Micro Devices, Inc. [AMD] Renoir/Cezanne Root Complex
00:00.2 IOMMU: Advanced Micro Devices, Inc. [AMD] Renoir/Cezanne IOMMU
    Subsystem: Advanced Micro Devices, Inc. [AMD] Renoir/Cezanne IOMMU
00:01.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Renoir PCIe Dummy Host Bridge
00:01.2 PCI bridge: Advanced Micro Devices, Inc. [AMD] Renoir/Cezanne PCIe GPP Bridge
    Subsystem: Advanced Micro Devices, Inc. [AMD] Renoir/Cezanne PCIe GPP Bridge
    Kernel driver in use: pcieport
00:01.3 PCI bridge: Advanced Micro Devices, Inc. [AMD] Renoir/Cezanne PCIe GPP Bridge
    Subsystem: Advanced Micro Devices, Inc. [AMD] Renoir/Cezanne PCIe GPP Bridge
    Kernel driver in use: pcieport
00:02.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Renoir PCIe Dummy Host Bridge
00:02.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Renoir/Cezanne PCIe GPP Bridge
    Subsystem: Advanced Micro Devices, Inc. [AMD] Renoir/Cezanne PCIe GPP Bridge
    Kernel driver in use: pcieport
00:08.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Renoir PCIe Dummy Host Bridge
00:08.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Renoir Internal PCIe GPP Bridge to Bus
    Subsystem: Advanced Micro Devices, Inc. [AMD] Renoir Internal PCIe GPP Bridge to Bus
    Kernel driver in use: pcieport
00:08.2 PCI bridge: Advanced Micro Devices, Inc. [AMD] Renoir Internal PCIe GPP Bridge to Bus
    Subsystem: Advanced Micro Devices, Inc. [AMD] Renoir Internal PCIe GPP Bridge to Bus
    Kernel driver in use: pcieport
00:14.0 SMBus: Advanced Micro Devices, Inc. [AMD] FCH SMBus Controller (rev 51)
    Subsystem: Advanced Micro Devices, Inc. [AMD] FCH SMBus Controller
    Kernel driver in use: piix4_smbus
    Kernel modules: i2c_piix4, sp5100_tco
00:14.3 ISA bridge: Advanced Micro Devices, Inc. [AMD] FCH LPC Bridge (rev 51)
    Subsystem: Advanced Micro Devices, Inc. [AMD] FCH LPC Bridge
00:18.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Cezanne Data Fabric; Function 0
00:18.1 Host bridge: Advanced Micro Devices, Inc. [AMD] Cezanne Data Fabric; Function 1
00:18.2 Host bridge: Advanced Micro Devices, Inc. [AMD] Cezanne Data Fabric; Function 2
00:18.3 Host bridge: Advanced Micro Devices, Inc. [AMD] Cezanne Data Fabric; Function 3
    Kernel driver in use: k10temp
    Kernel modules: k10temp
00:18.4 Host bridge: Advanced Micro Devices, Inc. [AMD] Cezanne Data Fabric; Function 4
00:18.5 Host bridge: Advanced Micro Devices, Inc. [AMD] Cezanne Data Fabric; Function 5
00:18.6 Host bridge: Advanced Micro Devices, Inc. [AMD] Cezanne Data Fabric; Function 6
00:18.7 Host bridge: Advanced Micro Devices, Inc. [AMD] Cezanne Data Fabric; Function 7
01:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 15)
    Subsystem: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller
    Kernel driver in use: r8169
    Kernel modules: r8169
02:00.0 Network controller: Intel Corporation Wi-Fi 6 AX200 (rev 1a)
    DeviceName: Onboard LAN Brodcom
    Subsystem: Intel Corporation Wi-Fi 6 AX200NGW
    Kernel driver in use: iwlwifi
    Kernel modules: iwlwifi
03:00.0 Non-Volatile memory controller: Micron/Crucial Technology P5 Plus NVMe PCIe SSD
    Subsystem: Micron/Crucial Technology P5 Plus NVMe PCIe SSD
    Kernel driver in use: nvme
    Kernel modules: nvme
04:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Cezanne [Radeon Vega Series / Radeon Vega Mobile Series] (rev c5)
    Subsystem: Advanced Micro Devices, Inc. [AMD/ATI] Cezanne [Radeon Vega Series / Radeon Vega Mobile Series]
    Kernel driver in use: vfio-pci
    Kernel modules: amdgpu
04:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Renoir Radeon High Definition Audio Controller
    Subsystem: Advanced Micro Devices, Inc. [AMD/ATI] Renoir Radeon High Definition Audio Controller
    Kernel driver in use: vfio-pci
    Kernel modules: snd_hda_intel
04:00.2 Encryption controller: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 10h-1fh) Platform Security Processor
    Subsystem: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 10h-1fh) Platform Security Processor
    Kernel driver in use: ccp
    Kernel modules: ccp
04:00.3 USB controller: Advanced Micro Devices, Inc. [AMD] Renoir/Cezanne USB 3.1
    Subsystem: Advanced Micro Devices, Inc. [AMD] Renoir/Cezanne USB 3.1
    Kernel driver in use: xhci_hcd
    Kernel modules: xhci_pci
04:00.4 USB controller: Advanced Micro Devices, Inc. [AMD] Renoir/Cezanne USB 3.1
    Subsystem: Advanced Micro Devices, Inc. [AMD] Renoir/Cezanne USB 3.1
    Kernel driver in use: xhci_hcd
    Kernel modules: xhci_pci
04:00.5 Multimedia controller: Advanced Micro Devices, Inc. [AMD] ACP/ACP3X/ACP6x Audio Coprocessor (rev 01)
    Subsystem: Advanced Micro Devices, Inc. [AMD] ACP/ACP3X/ACP6x Audio Coprocessor
    Kernel driver in use: snd_rn_pci_acp3x
    Kernel modules: snd_pci_acp3x, snd_rn_pci_acp3x, snd_pci_acp5x, snd_pci_acp6x, snd_acp_pci, snd_rpl_pci_acp6x, snd_pci_ps, snd_sof_amd_renoir, snd_sof_amd_rembrandt
04:00.6 Audio device: Advanced Micro Devices, Inc. [AMD] Family 17h/19h HD Audio Controller
    DeviceName: HD Audio Controller
    Subsystem: Advanced Micro Devices, Inc. [AMD] Family 17h/19h HD Audio Controller
    Kernel driver in use: snd_hda_intel
    Kernel modules: snd_hda_intel
05:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Zeppelin/Raven/Raven2 PCIe Dummy Function (rev 81)
    Subsystem: Advanced Micro Devices, Inc. [AMD] Zeppelin/Raven/Raven2 PCIe Dummy Function
05:00.1 SATA controller: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] (rev 81)
    Subsystem: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode]
    Kernel driver in use: ahci
    Kernel modules: ahci

diversity · Apr 7, 2024

ok. it looks like you are missing a lot of command line parameters but that is of later concern.

also I did not see the amd driver being correctly loaded. So please make sure they are on the host by disabling blacklisting and any VM that might want to use it.

then
on the host (no VM just yet or passthrough. i want to see which bios rom is used for your igpu like that)
dmesg | grep amdgpu
and
lspci -k

diversity · Apr 7, 2024

ahh well you know the age old saying. give a man enough to drink and he will start reading up out of sheer boredom

So in order to work from the top down you indeed need to revert a few steps back in order for me to be able to help out.

If things never worked on the host then I am not sure I can ever help out making sense of it.

So in case you are still trying to get this to work then please make sure you put comments around a anything you have done to try and make this work.

You see for me, and I am sure many others, the whole key has been hiding in plain sight for a long time. In where to look for what rom file I needed. that, and nomodeset were the missing pieces of the puzzle for me.

jhmc93 · Apr 8, 2024

Hello! @diversity, sorry for late reply, been so busy the past few days and not had chance to get on my PC,

the output to dmesg | grep amdgpu:

Code:

[    5.530187] [drm] amdgpu kernel modesetting enabled.
[    5.542715] amdgpu: CRAT table disabled by module option
[    5.542719] amdgpu: Virtual CRAT table created for CPU
[    5.542736] amdgpu: Topology: Add CPU node

and lspci -k:

Code:

00:00.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Renoir/Cezanne Root Complex
    Subsystem: Advanced Micro Devices, Inc. [AMD] Renoir/Cezanne Root Complex
00:00.2 IOMMU: Advanced Micro Devices, Inc. [AMD] Renoir/Cezanne IOMMU
    Subsystem: Advanced Micro Devices, Inc. [AMD] Renoir/Cezanne IOMMU
00:01.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Renoir PCIe Dummy Host Bridge
00:01.2 PCI bridge: Advanced Micro Devices, Inc. [AMD] Renoir/Cezanne PCIe GPP Bridge
    Subsystem: Advanced Micro Devices, Inc. [AMD] Renoir/Cezanne PCIe GPP Bridge
    Kernel driver in use: pcieport
00:01.3 PCI bridge: Advanced Micro Devices, Inc. [AMD] Renoir/Cezanne PCIe GPP Bridge
    Subsystem: Advanced Micro Devices, Inc. [AMD] Renoir/Cezanne PCIe GPP Bridge
    Kernel driver in use: pcieport
00:02.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Renoir PCIe Dummy Host Bridge
00:02.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Renoir/Cezanne PCIe GPP Bridge
    Subsystem: Advanced Micro Devices, Inc. [AMD] Renoir/Cezanne PCIe GPP Bridge
    Kernel driver in use: pcieport
00:08.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Renoir PCIe Dummy Host Bridge
00:08.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Renoir Internal PCIe GPP Bridge to Bus
    Subsystem: Advanced Micro Devices, Inc. [AMD] Renoir Internal PCIe GPP Bridge to Bus
    Kernel driver in use: pcieport
00:08.2 PCI bridge: Advanced Micro Devices, Inc. [AMD] Renoir Internal PCIe GPP Bridge to Bus
    Subsystem: Advanced Micro Devices, Inc. [AMD] Renoir Internal PCIe GPP Bridge to Bus
    Kernel driver in use: pcieport
00:14.0 SMBus: Advanced Micro Devices, Inc. [AMD] FCH SMBus Controller (rev 51)
    Subsystem: Advanced Micro Devices, Inc. [AMD] FCH SMBus Controller
    Kernel driver in use: piix4_smbus
    Kernel modules: i2c_piix4, sp5100_tco
00:14.3 ISA bridge: Advanced Micro Devices, Inc. [AMD] FCH LPC Bridge (rev 51)
    Subsystem: Advanced Micro Devices, Inc. [AMD] FCH LPC Bridge
00:18.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Cezanne Data Fabric; Function 0
00:18.1 Host bridge: Advanced Micro Devices, Inc. [AMD] Cezanne Data Fabric; Function 1
00:18.2 Host bridge: Advanced Micro Devices, Inc. [AMD] Cezanne Data Fabric; Function 2
00:18.3 Host bridge: Advanced Micro Devices, Inc. [AMD] Cezanne Data Fabric; Function 3
    Kernel driver in use: k10temp
    Kernel modules: k10temp
00:18.4 Host bridge: Advanced Micro Devices, Inc. [AMD] Cezanne Data Fabric; Function 4
00:18.5 Host bridge: Advanced Micro Devices, Inc. [AMD] Cezanne Data Fabric; Function 5
00:18.6 Host bridge: Advanced Micro Devices, Inc. [AMD] Cezanne Data Fabric; Function 6
00:18.7 Host bridge: Advanced Micro Devices, Inc. [AMD] Cezanne Data Fabric; Function 7
01:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 15)
    Subsystem: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller
    Kernel driver in use: r8169
    Kernel modules: r8169
02:00.0 Network controller: Intel Corporation Wi-Fi 6 AX200 (rev 1a)
    DeviceName: Onboard LAN Brodcom
    Subsystem: Intel Corporation Wi-Fi 6 AX200NGW
    Kernel driver in use: iwlwifi
    Kernel modules: iwlwifi
03:00.0 Non-Volatile memory controller: Micron/Crucial Technology P5 Plus NVMe PCIe SSD
    Subsystem: Micron/Crucial Technology P5 Plus NVMe PCIe SSD
    Kernel driver in use: nvme
    Kernel modules: nvme
04:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Cezanne [Radeon Vega Series / Radeon Vega Mobile Series] (rev c5)
    Subsystem: Advanced Micro Devices, Inc. [AMD/ATI] Cezanne [Radeon Vega Series / Radeon Vega Mobile Series]
    Kernel driver in use: vfio-pci
    Kernel modules: amdgpu
04:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Renoir Radeon High Definition Audio Controller
    Subsystem: Advanced Micro Devices, Inc. [AMD/ATI] Renoir Radeon High Definition Audio Controller
    Kernel driver in use: vfio-pci
    Kernel modules: snd_hda_intel
04:00.2 Encryption controller: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 10h-1fh) Platform Security Processor
    Subsystem: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 10h-1fh) Platform Security Processor
    Kernel driver in use: ccp
    Kernel modules: ccp
04:00.3 USB controller: Advanced Micro Devices, Inc. [AMD] Renoir/Cezanne USB 3.1
    Subsystem: Advanced Micro Devices, Inc. [AMD] Renoir/Cezanne USB 3.1
    Kernel driver in use: xhci_hcd
    Kernel modules: xhci_pci
04:00.4 USB controller: Advanced Micro Devices, Inc. [AMD] Renoir/Cezanne USB 3.1
    Subsystem: Advanced Micro Devices, Inc. [AMD] Renoir/Cezanne USB 3.1
    Kernel driver in use: xhci_hcd
    Kernel modules: xhci_pci
04:00.5 Multimedia controller: Advanced Micro Devices, Inc. [AMD] ACP/ACP3X/ACP6x Audio Coprocessor (rev 01)
    Subsystem: Advanced Micro Devices, Inc. [AMD] ACP/ACP3X/ACP6x Audio Coprocessor
    Kernel driver in use: snd_rn_pci_acp3x
    Kernel modules: snd_pci_acp3x, snd_rn_pci_acp3x, snd_pci_acp5x, snd_pci_acp6x, snd_acp_pci, snd_rpl_pci_acp6x, snd_pci_ps, snd_sof_amd_renoir, snd_sof_amd_rembrandt
04:00.6 Audio device: Advanced Micro Devices, Inc. [AMD] Family 17h/19h HD Audio Controller
    DeviceName: HD Audio Controller
    Subsystem: Advanced Micro Devices, Inc. [AMD] Family 17h/19h HD Audio Controller
    Kernel driver in use: snd_hda_intel
    Kernel modules: snd_hda_intel
05:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Zeppelin/Raven/Raven2 PCIe Dummy Function (rev 81)
    Subsystem: Advanced Micro Devices, Inc. [AMD] Zeppelin/Raven/Raven2 PCIe Dummy Function
05:00.1 SATA controller: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] (rev 81)
    Subsystem: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode]
    Kernel driver in use: ahci
    Kernel modules: ahci

again apologies for the late reply. Hope you can still help.
thanks much appreciated

diversity · Apr 9, 2024

I see you are still blacklisting the amdgpu driver?
or perhaps not blacklisting it but having the vfio-pci being used for your igpu instead.

the idea is to have it load normally to be able to see in the dmesg which BIOS rom file is loaded as supplied by the firmware on the motherboard.

But perhaps we do not need to go that route as this morning in the BIOS of my motherboard I found the name of the VBIOS being used.
113-RENOIR-034
in my case but yours might be different.

which I then found online and passed that file to the VM and it was bliss.

jhmc93 · Apr 9, 2024

diversity said:
I see you are still blacklisting the amdgpu driver?
or perhaps not blacklisting it but having the vfio-pci being used for your igpu instead.

the idea is to have it load normally to be able to see in the dmesg which BIOS rom file is loaded as supplied by the firmware on the motherboard.

But perhaps we do not need to go that route as this morning in the BIOS of my motherboard I found the name of the VBIOS being used.
113-RENOIR-034
in my case but yours might be different.

which I then found online and passed that file to the VM and it was bliss.

I think I have blacklisted the amdgpu driver, I’ve not been able to get hold of a vbios for my motherboard. As i tried saving one in gpu-z and it said it not found. But is there any advice u can give on how to get it working.

diversity · Apr 9, 2024

My advice would be is to first not blacklist it and not pass it through and also not load vfio drivers at any stage.

let your host system figure out which vbios file to use.

or do you get greeted with an error message that a VBIOS file could not be found in your host's dmesg? I have had that as well earlier and had to change some motherboard settings.

jhmc93 · Apr 9, 2024

diversity said:
My advice would be is to first not blacklist it and not pass it through and also not load vfio drivers at any stage.

let your host system figure out which vbios file to use.

or do you get greeted with an error message that a VBIOS file could not be found in your host's dmesg? I have had that as well earlier and had to change some motherboard settings.

See I couldn’t tell you over the dmesg, is there a line in there that would tell you to see if it loading?, so delete it from blacklist? And don’t install vfio drivers on windows 11?

diversity · Apr 9, 2024

dont run any vm that has that igpu passed through.
remove any blacklists
remove the vfio-driver loading for the igpu.

just like in the tutorials but then in reverse.
and then show the output of dmesg | grep amdgpu

you will find the name of the vbios file being used if the motherboard supplies it.
if it is not being supplied by your motherboard then you are in a whole different kind of trouble and will have to troubleshoot there first

jhmc93 · Apr 9, 2024

diversity said:
dont run any vm that has that igpu passed through.
remove any blacklists
remove the vfio-driver loading for the igpu.

just like in the tutorials but then in reverse.
and then show the output of dmesg | grep amdgpu

you will find the name of the vbios file being used if the motherboard supplies it.
if it is not being supplied by your motherboard then you are in a whole different kind of trouble and will have to troubleshoot there first

That tutorial I sent you?
remove the vfio from /etc/modules?

diversity · Apr 9, 2024

I am unaware of any tutorial you have used. but it does not matter. just undo anything you did and see if you can get the amdgpu driver to load normally.

jhmc93 · Apr 9, 2024

diversity said:
I am unaware of any tutorial you have used. but it does not matter. just undo anything you did and see if you can get the amdgpu driver to load normally.

Would it be best to do a fresh install and enable iommu again?

diversity · Apr 9, 2024

I can't make that call for you

I for one am starting over again pretty soon. But I do not know how much you have already tweaked your system.

jhmc93 · Apr 9, 2024

diversity said:
I can't make that call for you I for one am starting over again pretty soon. But I do not know how much you have already tweaked your system.

Not much just setup a few little things, but then again I don’t wanna get rid of a samba share vm I have running on there.
Would I need to tweak the grub settings if I had to disable what I did ?

diversity · Apr 9, 2024

jhmc93 · Apr 9, 2024

diversity said:
yes

So just disable iommu?

diversity · Apr 9, 2024

in your case it is best to start over. as disabling IOMMU is not required in order to not blacklist or load vfio drivers.

So better use a clean slate and document well what you are doing or use a tutorial from proxmox it self like
https://pve.proxmox.com/wiki/PCI(e)_Passthrough

jhmc93 · Apr 9, 2024

diversity said:
in your case it is best to start over. as disabling IOMMU is not required in order to not blacklist or load vfio drivers.

So better use a clean slate and document well what you are doing or use a tutorial from proxmox it self like
https://pve.proxmox.com/wiki/PCI(e)_Passthrough

What should I follow the guide until? Any point in the guide u want me to stop? So I can show you what you need to help me

diversity · Apr 9, 2024

none of it during your reinstall and getting the amdgpu driver to load correctly on the host.
once you have that up and running then it becomes time to think about passing it through

diversity · Apr 10, 2024

something like this is what you want to see on the host when you run
dmesg | grep amdgpu

[ 5.084765] amdgpu 0000:06:00.0: enabling device (0006 -> 0007)
[ 5.087448] amdgpu 0000:06:00.0: amdgpu: Fetched VBIOS from VFCT
[ 5.087458] amdgpu: ATOM BIOS: 113-CEZANNE-017

ATOM BIOS: *****

And then other signs like that the driver is properly loaded in the lspci -k output

Only then it is a good time to embark on trying to pass it through

AMD iGPU passthrough won't work

Member

Active Member

Active Member

Member

Active Member

Member

Active Member

Member

Active Member

Member

Active Member

Member

Active Member

Member

Active Member

Member

Active Member

Member

Active Member

Active Member