GPU Passthrough issues

S

Simon9116

New Member
Aug 1, 2024
11
1
3
Hi everyone,
I am encountering issues with GPU passthrough in a Windows 11 VM.

My build:
- Ryzen 9 9950x
- Gigabyte x870 Gaming Wifi 6
- NVIDIA RTX 5070ti

UEFI settings:
- Resizable BAR: enabled
- Above 4G decoding: enabled
- CSM: disabled

GRUB config:
GRUB_CMDLINE_LINUX_DEFAULT="quiet pcie_acs_override=downstream,multifunction nofb nomodeset video=vesafb:off video=efifb:off video=simplefb:off"

VM config:
args: -cpu host,-hypervisor,kvm=off, -smbios type=0,vendor="American Megatrends International",version=F5,date="03/12/2025"
bios: ovmf
boot: order=scsi0;net0
cores: 16
cpu: host,hidden=1
efidisk0: vms:vm-110-disk-0,efitype=4m,pre-enrolled-keys=1,size=1M
hostpci0: 0000:01:00.0,pcie=1,romfile=vbios.bin,vendor-id=0x10de,x-vga=1
machine: pc-q35-9.2+pve1
memory: 32768
meta: creation-qemu=9.2.0,ctime=1745758477
name: Win11-gaming
net0: virtio=BC:24:11:B9:D3:58,bridge=vmbr0,firewall=1
ostype: win11
scsi0: vms:vm-110-disk-1,iothread=1,size=150G
scsihw: virtio-scsi-single
smbios1: uuid=67dbbf21-678b-45dd-a5d4-94f1d3ac6487
sockets: 1
tpmstate0: vms:vm-110-disk-2,size=4M,version=v2.0
vga: virtio
vmgenid: c9eef372-e262-4aa4-83bf-b78268cda3a8


Basically, I set up IOMMU correctly and I can pass the GPU to the VM. Then I configure the basic stuff (RDP, VirtIO drivers for the NIC, ...).

The issues arise when I try to download the nvidia drivers: at first the GPU seems to get recognized correctly (no code 43), but when I reboot it, it gets stuck on the loading screen (see image) and sometimes windows goes into recovery.
Further tests revealed that the issues start as soon as the drivers finish installing. I discovered this because I tried not to connect through RDP but with an HDMI cable to a physical monitor and there I could see the image until the drivers finished installing, then I could only see a black screen.

I already tried multiple "solutions" which did not work: changing the GRUB config, recreating the VM, removing the drivers through DDU and then reinstalling them even using NVCleanstall to check the MSI flag, but still no luck.

Can someone help me?
 

Attachments

  • Immagine 2025-04-27 163449.png
    Immagine 2025-04-27 163449.png
    52.2 KB · Views: 3
Last edited:
hey,
For passingtrough my rtx3060, i only need theses 3 options:
Code: 
GRUB_CMDLINE_LINUX_DEFAULT="quiet amd_iommu=on iommu=pt"

Did you have tried with only theses options ?
 
After you installed the video-card driver, shut down the VM, edit the config:
Code: 
vga: none

Start the VM, done.
 
Last edited:
Pifouney said:
hey,
For passingtrough my rtx3060, i only need theses 3 options:
Code: 
GRUB_CMDLINE_LINUX_DEFAULT="quiet amd_iommu=on iommu=pt"

Did you have tried with only theses options ?
Click to expand...
Hi, thank you for your response. I did try with only those options in the beginning, but it didn't work and also I need acs for isolation, otherwise other devices will be passed to the VM. The other options are there because I was reading some solutions on other posts on the forum but didn't work so I'll remove them.
 
Last edited:
  • Like
Reactions: Pifouney
How many physical video cards do you have in your HOST system?
If you have only 1-card and using the motherboard as the primary card, it will not work.
You need at least 2card:
-1card for your motherboard (primary)
-1card for passthrough.

The problem is, when you power on the HOST system it will be initialize the video-card (primary), and wont work as it should be. Thats why need the 2nd card to left unitialized, which will handle the VM when starts it.
 
emunt6 said:
How many physical video cards do you have in your HOST system?
If you have only 1-card and using the motherboard as the primary card, it will not work.
You need at least 2card:
-1card for your motherboard (primary)
-1card for passthrough.

The problem is, when you power on the HOST system it will be initialize the video-card (primary), and wont work as it should be. Thats why need the 2nd card to left unitialized, which will handle the VM when starts it.
Click to expand...
Yep, already considered it. In fact, I am using the CPU integrated graphics as the primary card.
 
i don't know if it will be of any help at all, but when trying to pass my Intel ARC A310 to my windows 10/11 VMs, with it being the primary GPU with no iGPU (using i7-7820X cpu and Gigabyte X299 ud4 mbo), i had to remove and rescan it to basically take it away from proxmox, since proxmox by default has control over it for terminal video output, etc as the main gpu. although it wouldn't really passthrough at all until i did this, so sounds like a totally different problem since you have gotten it to pass before the driver install, where as with mine it would error out since its the primary gpu.

here are the commands from my terminal, that i used in the process, its been some months since i did it so i don't remember perfectly but i went back in my history and grabbed the commands.

Code: 
echo 1 > /sys/bus/pci/devices/0000\:67\:00.0/remove # the hdmi audio controller
echo 1 > /sys/bus/pci/devices/0000\:68\:00.0/remove # the GPU
echo 1 > /sys/bus/pci/devices/0000\:68\:00.0/rescan # rescan GPU
echo 1 > /sys/bus/pci/rescan # rescan all
ls /sys/bus/pci/devices/0000\:68\:00.0

# replace 67-68 with device IIMOU group ID number

NOTE: if this is the problem, not all motherboards will allow it, some reboot instantly if the GPU is taken from the OS
 
Last edited:
Simon9116 said:
Thank you very much, but if I remember correctly I had already tried rescaning the GPU with no luck :(
Click to expand...
you're welcome, i hope you find the fix for this issue, it could just be that it is a blackwell card and there could be problems with support still since it is really new. but did you remove then rescan or just rescan?
 
  • Like
Reactions: Simon9116
zenowl77 said:
you're welcome, i hope you find the fix for this issue, it could just be that it is a blackwell card and there could be problems with support still since it is really new. but did you remove then rescan or just rescan?
Click to expand...
I think I removed it as well, but I'll try doing it again, it certainly won't hurt.
 
zenowl77 said:
i don't know if it will be of any help at all, but when trying to pass my Intel ARC A310 to my windows 10/11 VMs, with it being the primary GPU with no iGPU (using i7-7820X cpu and Gigabyte X299 ud4 mbo), i had to remove and rescan it to basically take it away from proxmox, since proxmox by default has control over it for terminal video output, etc as the main gpu. although it wouldn't really passthrough at all until i did this, so sounds like a totally different problem since you have gotten it to pass before the driver install, where as with mine it would error out since its the primary gpu.

here are the commands from my terminal, that i used in the process, its been some months since i did it so i don't remember perfectly but i went back in my history and grabbed the commands.

Code: 
echo 1 > /sys/bus/pci/devices/0000\:67\:00.0/remove # the hdmi audio controller
echo 1 > /sys/bus/pci/devices/0000\:68\:00.0/remove # the GPU
echo 1 > /sys/bus/pci/devices/0000\:68\:00.0/rescan # rescan GPU
echo 1 > /sys/bus/pci/rescan # rescan all
ls /sys/bus/pci/devices/0000\:68\:00.0

# replace 67-68 with device IIMOU group ID number

NOTE: if this is the problem, not all motherboards will allow it, some reboot instantly if the GPU is taken from the OS
Click to expand...
Okay, I tried Win10 and it has the same problem. I even tried to run all updates in win11 but still no luck.

I tried running the commands you gave me and it let me remove the gpu but not rescan it, I don't no why. It threw an error telling there was no such file or directory and I had to reboot the system to get the GPU to show up again. Maybe that can tell us something more about the issue.

Or maybe it's just like you said: the GPU is very new and maybe has some compatibility issues.
Thanks anyways.
 
You must log in or register to reply here.
Top Bottom