GPU Passthrough : Windows 11 problem

[SkyFlash21]

Hello, I can't despite many attempts to create a virtual machine with amd gpu. I followed a tutorial to the letter: https://www.reddit.com/r/homelab/comments/b5xpua/the_ultimate_beginners_guide_to_gpu_passthrough/ but each time I start with the graphics card, the virtual machine does not start and returns the error 1 (QEMU).When I remove the card from the materials, the machine does not work perfectly. Can you help me please?

There is the log file https://pastebin.com/hcXLkhdJ

Edit : When I launch the virtual machine, proxmox crashes instantly and I can see this on the screen to which the graphics card is connected (moreover this is displayed despite the blacklist) : https://youtu.be/uNZNKJdaq1Y (Volume warning ! my mic is broken)

 

[leesteken]

The GPU is not resetting properly. What GPU is that exactly? Do you have another (i)GPU that is used by Proxmox for booting and console or is the AMD GPU used during boot? Did I ask what kind of AMD GPU already?

 

[SkyFlash21]

The GPU is not resetting properly. What GPU is that exactly? Do you have another (i)GPU that is used by Proxmox for booting and console or is the AMD GPU used during boot? Did I ask what kind of AMD GPU already?

RX580 4Go , 16gb ram Motherboard: B450 AORUS ELITE CPU : AMD Ryzen 5 2600. I have only one gpu. The cpu dont have an integrated GPU. But i found that weird that the gpu is used by proxmox depsite the blacklist.

 

[leesteken]

RX580 4Go , 16gb ram Motherboard: B450 AORUS ELITE CPU : AMD Ryzen 5 2600. I have only one gpu. The cpu dont have an integrated GPU. But i found that weird that the gpu is used by proxmox depsite the blacklist.

You need vendor-reset, because that generation of AMD GPUs has issues with resetting. Here's a guide. Even when blacklisted, it will be used by simplefb, which probably requires the initcall_blacklist=sysfb_init since the GPU is used during boot. More threads about vendor-reset and simplefb issues can be found on this forum.

 

[SkyFlash21]

You need vendor-reset, because that generation of AMD GPUs has issues with resetting. Here's a guide. Even when blacklisted, it will be used by simplefb, which probably requires the initcall_blacklist=sysfb_init since the GPU is used during boot. More threads about vendor-reset and simplefb issues can be found on this forum.

Thank you so much, I'll go try it out.

 

[leesteken]

Thank you so much, I'll go try it out.

If you are using pve-kernel 5.15 or higher then you need to enable vendor-reset explicitly.

 

[SkyFlash21]

If you are using pve-kernel 5.15 or higher then you need to enable vendor-reset explicitly.

When I try to execute this command [echo 'device_specific' > /sys/bus/pci/devices/0000:05:00.0/reset_method], the console gives me this error
-bash: echo: write error: Invalid argument

I tried to figure out online and I think it means that my graphics card is not compatible with the reset feature.

 

[leesteken]

What's the output of uname -a? It's unlikely that your GPU is not compatible with vendor-reset. What is the output of lspci -nnks 05:00?

 

[SkyFlash21]

What's the output of uname -a? It's unlikely that your GPU is not compatible with vendor-reset. What is the output of lspci -nnks 05:00?

 

[leesteken]

What is the output of cat /sys/bus/pci/devices/0000\:05\:00.0/reset_method?
It might help if you update your Proxmox as your kernel version shows that you are using an older version.

EDIT: You said it is an RX 580 but it is detected as XFX Radeon RX 570. Is Linux wrong or did you make a mistake?

 

[SkyFlash21]

What is the output of cat /sys/bus/pci/devices/0000\:05\:00.0/reset_method?
It might help if you update your Proxmox as your kernel version shows that you are using an older version.

I will try to update it.

 

[SkyFlash21]

After an update and restart it seem to apply. But the card is still used, the computer display proxmox image throw hdmi.

 

[leesteken]

After an update and restart it seem to apply. But the card is still used, the computer display proxmox image throw hdmi.

Maybe you forgot a modprobe before? That might not be a problem if the GPU reset properly with the help of vendor-reset. What is currently not working?

 

[SkyFlash21]

Maybe you forgot a modprobe before? That might not be a problem if the GPU reset properly with the help of vendor-reset. What is currently not working?

Currently :

1) The proxmox server still seems to be using the graphics card so what is blacklisted. (The screen displays the proxmox console)

2) When I start the windows virtual machine with the graphics card in hardware, the server freezes and it is impossible for me to do anything (all the other virtual machines also freeze)

 

[leesteken]

Currently :

1) The proxmox server still seems to be using the graphics card so what is blacklisted. (The screen displays the proxmox console)

Blacklisting prevents the driver from being used but other stuff might still use it like simplefb.

2) When I start the windows virtual machine with the graphics card in hardware, the server freezes and it is impossible for me to do anything (all the other virtual machines also freeze)

Make sure the GPU is in a separate IOMMU group, but if this is the only one and it is in the first PCIe x16 slot then it should be (unless your BIOS breaks passthrough, which is known to happen with Ryzen motherboards and various AGESA versions).

Most likely BOOTFB is present in cat /proc/iomem and you need the initcall_blacklist=sysfb_init mentioned before. Is it present in cat /proc/cmdline? Otherwise add it to the kernel parameters.

 

[SkyFlash21]

Blacklisting prevents the driver from being used but other stuff might still use it like simplefb.

Make sure the GPU is in a separate IOMMU group, but if this is the only one and it is in the first PCIe x16 slot then it should be (unless your BIOS breaks passthrough, which is known to happen with Ryzen motherboards and various AGESA versions).

Most likely BOOTFB is present in cat /proc/iomem and you need the initcall_blacklist=sysfb_init mentioned before. Is it present in cat /proc/cmdline? Otherwise add it to the kernel parameters.

It worked for 1 time, I just restarted the proxmox server and after that I can't start the virtual machine.

The card is in an isolated group and "initcall_blacklist=sysfb_init" is already set

 

[leesteken]

The card is in an isolated group and "initcall_blacklist=sysfb_init" is already set

Good. And you also always set device_specific for the reset_method after each reboot?

It worked for 1 time, I just restarted the proxmox server and after that I can't start the virtual machine.

I can't explain that except if you changed something that only took effect after a reboot or if you did not do something like before (and that it gets reset with a reboot).
What does "can't start" mean exactly? How does it manifest, what are the symptoms? What errors do you get? What is in the journalctl -b 0 around the time of starting the VM?

 

[SkyFlash21]

Good. And you also always set device_specific for the reset_method after each reboot?

Yes, i set it after each reboot.

I can't explain that except if you changed something that only took effect after a reboot or if you did not do something like before (and that it gets reset with a reboot).
What does "can't start" mean exactly? How does it manifest, what are the symptoms? What errors do you get? What is in the journalctl -b 0 around the time of starting the VM?

To be more specific. This is what i can see in the web pannel ;
In the log ; https://pastebin.com/TYqDSd9W

Step :
1) I click to start the vam
2) The console menu goes blank and i can see the task "start" apear
3) after 5s (aproximatly) the console menu show (Guest not running) and the task return Qemu exit code 1

 

[leesteken]

Yes, i set it after each reboot.

To be more specific. This is what i can see in the web pannel ;
In the log ; https://pastebin.com/TYqDSd9W

can't change power state from D3cold to D0 (config space inaccessible) make me think that you motherboard BIOS version does not work well with passhtrough. What's you current BIOS version?
Have you tried BIOS version F64c (double check if that's your motherboard)? AMD AGESA V2 1.2.0.7 works very well for a RX 570. But also AMD AGESA 1.0.0.4 B, which would be F50 for you. Versions earlier and in between roke passthrough in various ways (on another motherboard).

EDIT: Can you share the VM configuration file (attach it) from the (/etc/pve/qemu-server/104)? Usually with passthrough there is no virtual console.

 

[SkyFlash21]

can't change power state from D3cold to D0 (config space inaccessible) make me think that you motherboard BIOS version does not work well with passhtrough. What's you current BIOS version?
Have you tried BIOS version F64c (double check if that's your motherboard)? AMD AGESA V2 1.2.0.7 works very well for a RX 570. But also AMD AGESA 1.0.0.4 B, which would be F50 for you. Versions earlier and in between roke passthrough in various ways (on another motherboard).

Bios version : F41