[SOLVED] X99 Extreme4 - GPU Passthrough Windows 10 with Radeon RX570 (Black screen)

leesteken said:
That should work. Check in journalctl if you see messages like AMD_POLARIS10: reset result = 0. If you don't see several lines like that when starting the VM, then vendor-reset is not setup correctly.
Click to expand...

Passing through my RX570 is not working with Ubuntu. I will now setup again vendor-reset.


this is my UBUNTU setup:

bios: ovmf boot: order=scsi0;ide2;net0 cores: 6 efidisk0: VMssd:vm-1000-disk-0,efitype=4m,pre-enrolled-keys=1,size=1M ide2: Synology-Backup:iso/ubuntu-22.04.1-desktop-amd64.iso,media=cdrom,size=3737140K machine: q35 memory: 8192 meta: creation-qemu=7.1.0,ctime=1676025834 name: Ubuntu net0: virtio=F6:2A:4E:D2:D5:B3,bridge=vmbr0,firewall=1 numa: 0 ostype: l26 scsi0: VMssd:vm-1000-disk-1,iothread=1,size=32G scsihw: virtio-scsi-single smbios1: uuid=712aec89-d82e-4b32-b17c-4ecdbfc0d4c2 sockets: 1 vmgenid: f9e13872-917f-4c82-a0c6-b6bf3dc3e5a3
 
Last edited:
sutlnet said:
Passing through my RX570 is not working with Ubuntu. I will now setup again vendor-reset.


this is my UBUNTU setup:

bios: ovmf boot: order=scsi0;ide2;net0 cores: 6 efidisk0: VMssd:vm-1000-disk-0,efitype=4m,pre-enrolled-keys=1,size=1M ide2: Synology-Backup:iso/ubuntu-22.04.1-desktop-amd64.iso,media=cdrom,size=3737140K machine: q35 memory: 8192 meta: creation-qemu=7.1.0,ctime=1676025834 name: Ubuntu net0: virtio=F6:2A:4E:D2:D5:B3,bridge=vmbr0,firewall=1 numa: 0 ostype: l26 scsi0: VMssd:vm-1000-disk-1,iothread=1,size=32G scsihw: virtio-scsi-single smbios1: uuid=712aec89-d82e-4b32-b17c-4ecdbfc0d4c2 sockets: 1 vmgenid: f9e13872-917f-4c82-a0c6-b6bf3dc3e5a3
Click to expand...
GPU is not working as well with UBUNTU. I did the vendor-reset on Proxmox and I created this script:

lib/systemd/system/vrwa.service:
[Unit]
Description=vrwa Service
After=multi-user.target
[Service]
ExecStart=/usr/bin/bash -c 'echo device_specific > /sys/bus/pci/devices/0000:0c:00.0/reset_method'
[Install]
WantedBy=multi-user.target
and i enabled it with systemd: systemctl enable vrwa

I am probably missing something. How do I run journalctl as the output gives me strange dates
 
sutlnet said:
I am probably missing something. How do I run journalctl as the output gives me strange dates
Click to expand...
Run journalctl -e to start at the end and use arrow keys and PageUp and PageDown to scroll to the right day and time.
Use journalctl -b 0 | grep vfio-pci to see all vfio-pci messages since the last reboot. It should contain reset messages from vendor-reset.

What is the output of lspci -nnks 0c:00 after a Proxmox reboot and before starting the VM?
What is your VM configuration with passthrough (the last one you showed did not have a hostpci0 line)?
 
Last edited:
leesteken said:
Run journalctl -e to start at the end and use arrow keys and PageUp and PageDown to scroll to the right day and time.
Use journalctl -b 0 | grep vfio-pci to see all vfio-pci messages since the last reboot. It should contain reset messages from vendor-reset.

What is the output of lspci -nnks 0c:00 after a Proxmox reboot and before starting the VM?
What is your VM configuration with passthrough (the last one you showed did not have a hostpci0 line)?
Click to expand...
Is this what you mean:

Feb 10 14:10:04 pve pvescheduler[1959]: starting server
Feb 10 14:10:04 pve systemd[1]: Started Proxmox VE scheduler.
Feb 10 14:10:04 pve systemd[1]: Reached target Multi-User System.
Feb 10 14:10:04 pve systemd[1]: Reached target Graphical Interface.
Feb 10 14:10:04 pve systemd[1]: Starting Update UTMP about System Runlevel Changes...
Feb 10 14:10:04 pve systemd[1]: Started vrwa Service.
Feb 10 14:10:04 pve bash[1963]: /usr/bin/bash: line 1: echo: write error: Invalid argument
Feb 10 14:10:04 pve systemd[1]: vrwa.service: Main process exited, code=exited, status=1/FAILURE
Feb 10 14:10:04 pve systemd[1]: vrwa.service: Failed with result 'exit-code'.
Feb 10 14:10:04 pve kernel: amdgpu 0000:02:00.0: Unsupported reset method 'device_specific'
Feb 10 14:10:04 pve systemd[1]: systemd-update-utmp-runlevel.service: Succeeded.
Feb 10 14:10:04 pve systemd[1]: Finished Update UTMP about System Runlevel Changes.
Feb 10 14:10:04 pve systemd[1]: Startup finished in 4.335s (kernel) + 10.136s (userspace) = 14.472s.
Feb 10 14:10:06 pve chronyd[1694]: Selected source 212.18.3.19 (2.debian.pool.ntp.org)
Feb 10 14:10:06 pve chronyd[1694]: System clock TAI offset set to 37 seconds
Feb 10 14:10:10 pve kernel: L1TF CPU bug present and SMT on, data leak possible. See CVE-2018-3646 and https://>
Feb 10 14:10:10 pve kernel: FS-Cache: Loaded

For this journalctl -b 0 | grep vfio-pci I am not getting anything out

Ubuntu Config
Code: 
  GNU nano 5.4                             /etc/pve/qemu-server/1000.conf                                       
bios: ovmf
boot: order=scsi0;ide2;net0
cores: 6
efidisk0: VMssd:vm-1000-disk-0,efitype=4m,pre-enrolled-keys=1,size=1M
hostpci0: 0000:02:00,pcie=1
ide2: Synology-Backup:iso/ubuntu-22.04.1-desktop-amd64.iso,media=cdrom,size=3737140K
machine: q35
memory: 8192
meta: creation-qemu=7.1.0,ctime=1676025834
name: Ubuntu
net0: virtio=F6:2A:4E:D2:D5:B3,bridge=vmbr0,firewall=1
numa: 0
ostype: l26
scsi0: VMssd:vm-1000-disk-1,iothread=1,size=32G
scsihw: virtio-scsi-single
smbios1: uuid=712aec89-d82e-4b32-b17c-4ecdbfc0d4c2
sockets: 1
vmgenid: f9e13872-917f-4c82-a0c6-b6bf3dc3e5a3
 
Last edited:
sutlnet said:
Feb 10 14:10:04 pve systemd[1]: Started vrwa Service.
Feb 10 14:10:04 pve bash[1963]: /usr/bin/bash: line 1: echo: write error: Invalid argument
Feb 10 14:10:04 pve systemd[1]: vrwa.service: Main process exited, code=exited, status=1/FAILURE
Feb 10 14:10:04 pve systemd[1]: vrwa.service: Failed with result 'exit-code'.
Feb 10 14:10:04 pve kernel: amdgpu 0000:02:00.0: Unsupported reset method 'device_specific'
Click to expand...
Looks like vendor-reset is not loaded. What is the output of lsmod | grep vendor? Is vendor-reset in /etc/modules? Did you run update-initramfs -u? Does your script run before kernel modules are loaded maybe? Try running the command manually after running modprobe vendor-reset to load vendor-reset.
sutlnet said:
For this journalctl -b 0 | grep vfio-pci I am not getting anything out
Click to expand...
That's probably because vendor-reset is not loaded and you did not start a VM with passthrough.
What is the output of lspci -nnks 02:00 after a Proxmox reboot and before starting the VM?
Then start the VM and run journalctl -b 0 | grep vfio-pci afterwards.
 
leesteken said:
/etc/modules
Click to expand...

root@pve:~# cat /etc/modules # /etc/modules: kernel modules to load at boot time. # # This file contains the names of kernel modules that should be loaded # at boot time, one per line. Lines beginning with "#" are ignored. vfio vfio_iommu_type1 vfio_pci vfio_virqfd vendor-reset

leesteken said:
lsmod | grep vendor
Click to expand...

I do not getting any output
leesteken said:
update-initramfs -u
Click to expand...
I did run it


leesteken said:
lspci -nnks 02:00
Click to expand...
lspci -nnks 02:00 02:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Ellesmere [Radeon RX 470/480/570/570X/580/580X/590] [1002:67df] (rev ef) Subsystem: Sapphire Technology Limited Radeon RX 570 Pulse 4GB [1da2:e353] Kernel driver in use: amdgpu Kernel modules: amdgpu 02:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Ellesmere HDMI Audio [Radeon RX 470/480 / 570/580/590] [1002:aaf0] Subsystem: Sapphire Technology Limited Ellesmere HDMI Audio [Radeon RX 470/480 / 570/580/590] [1da2:aaf0] Kernel driver in use: snd_hda_intel Kernel modules: snd_hda_intel

leesteken said:
journalctl -b 0 | grep vfio-pci
Click to expand...
root@pve:~# journalctl -b 0 | grep vfio-pci Feb 10 16:04:15 pve kernel: vfio-pci 0000:02:00.0: vgaarb: deactivate vga console Feb 10 16:04:15 pve kernel: vfio-pci 0000:02:00.0: vgaarb: changed VGA decodes: olddecodes=io+mem,decodes=io+mem:owns=io+mem Feb 10 16:04:17 pve kernel: vfio-pci 0000:02:00.0: vfio_ecap_init: hiding ecap 0x19@0x270 Feb 10 16:04:17 pve kernel: vfio-pci 0000:02:00.0: vfio_ecap_init: hiding ecap 0x1b@0x2d0 Feb 10 16:04:17 pve kernel: vfio-pci 0000:02:00.0: vfio_ecap_init: hiding ecap 0x1e@0x370
 
sutlnet said:
lspci -nnks 02:00 02:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Ellesmere [Radeon RX 470/480/570/570X/580/580X/590] [1002:67df] (rev ef) Subsystem: Sapphire Technology Limited Radeon RX 570 Pulse 4GB [1da2:e353] Kernel driver in use: amdgpu Kernel modules: amdgpu 02:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Ellesmere HDMI Audio [Radeon RX 470/480 / 570/580/590] [1002:aaf0] Subsystem: Sapphire Technology Limited Ellesmere HDMI Audio [Radeon RX 470/480 / 570/580/590] [1da2:aaf0] Kernel driver in use: snd_hda_intel Kernel modules: snd_hda_intel
Click to expand...
This looks good.
sutlnet said:
root@pve:~# journalctl -b 0 | grep vfio-pci Feb 10 16:04:15 pve kernel: vfio-pci 0000:02:00.0: vgaarb: deactivate vga console Feb 10 16:04:15 pve kernel: vfio-pci 0000:02:00.0: vgaarb: changed VGA decodes: olddecodes=io+mem,decodes=io+mem:owns=io+mem Feb 10 16:04:17 pve kernel: vfio-pci 0000:02:00.0: vfio_ecap_init: hiding ecap 0x19@0x270 Feb 10 16:04:17 pve kernel: vfio-pci 0000:02:00.0: vfio_ecap_init: hiding ecap 0x1b@0x2d0 Feb 10 16:04:17 pve kernel: vfio-pci 0000:02:00.0: vfio_ecap_init: hiding ecap 0x1e@0x370
Click to expand...
This also shows that vendor-reset is not working.
sutlnet said:
I do not getting any output

I did run it
Click to expand...
Vendor-reset is not loaded and therefore the AMD GPU will not reset properly for passthrough.

What is the output of modprobe vendor-reset?
What is the output of cat /sys/bus/pci/devices/0000:02:00.0/reset_method?
 
leesteken said:
modprobe vendor-rese
Click to expand...
I am getting an error:

root@pve:~# modprobe vendor-rese modprobe: FATAL: Module vendor-rese not found in directory /lib/modules/6.1.6-1-pve

leesteken said:
cat /sys/bus/pci/devices/0000:02:00.0/reset_method
Click to expand...
root@pve:~# cat /sys/bus/pci/devices/0000:02:00.0/reset_method bus

I did all the following steps from NickSherlocks' page:

# Get latest Proxmox kernel headers: apt install pve-headers # Did that fail? If so make sure you have Proxmox repository set up properly! https://pve.proxmox.com/wiki/Package_Repositories # Get required build tools: apt install git dkms build-essential # Perform the build: git clone https://github.com/gnif/vendor-reset.git cd vendor-reset dkms install . # Enable vendor-reset to be loaded automatically on startup: echo "vendor-reset" >> /etc/modules update-initramfs -u # Reboot to load the module: shutdown -r now

Struggling with the getup script
 
sutlnet said:
I am getting an error:

root@pve:~# modprobe vendor-rese modprobe: FATAL: Module vendor-rese not found in directory /lib/modules/6.1.6-1-pve
Click to expand...
It's vendor-reset, not vendor-rese. Sorry, I must have typed it wrong. What is the output of modprobe vendor-reset?
What is the output of cat /sys/bus/pci/devices/0000:02:00.0/reset_method afterwards?
 
sutlnet said:
I am still getting this error message:

modprobe: FATAL: Module vendor-reset not found in directory /lib/modules/6.1.6-1-pve
Click to expand...
Looks like you did not build the module for this kernel. Maybe you updated to 6.1 after installing vendor-reset? Did you also install pve-headers-6.1? Otherwise it cannot build the vendor-reset module for pve-kernel-6.1.6-1-pve.
 
leesteken said:
Looks like you did not build the module for this kernel. Maybe you updated to 6.1 after installing vendor-reset? Did you also install pve-headers-6.1? Otherwise it cannot build the vendor-reset module for pve-kernel-6.1.6-1-pve.
Click to expand...

I just ran everyone and still getting the fatal error
 
sutlnet said:
I just ran everyone and still getting the fatal error
Click to expand...
I have vendor reset, but not under
/lib/modules/6.1.6-1-pve
it is under
/vendor/reset

Can I just copy the directory into
/lib/modules/6.1.6-1-pve
 
Last edited:
sutlnet said:
I have vendor reset, but not under
/lib/modules/6.1.6-1-pve
it is under
/vendor/reset
Click to expand...
That is probably the source code, not the compiled module.
sutlnet said:
Can I just copy the directory into
/lib/modules/6.1.6-1-pve
Click to expand...
Did that work?
Did you install the right headers with apt install pve-headers-6.1.6-1-pve?
What is the output of dkms install . when you run it from the directory where you cloned the vendor-reset git repository?
 
sutlnet said:
I have now installed it and run update-initramfs -u



this is the output

Error! DKMS tree already contains: vendor-reset-0.1.1

should I now reboot and try with ubuntu
Click to expand...
Maybe it will work. Or maybe you need to remove the module first. I'm not sure. Does modprobe vendor-reset now work?

If you run apt update and apt install pve-headers-6.1 and apt dist-upgrade, you'll get a new kernel version 6.1.10 (from today) and vendor-reset should be installed correctly the first time.
 
leesteken said:
modprobe vendor-reset
Click to expand...
Still getting this message
modprobe: FATAL: Module vendor-reset not found in directory /lib/modules/6.1.6-1-pve

I will now remove the module and upgrade - and then install vendor-reset again

Thank you - I will give an update
 
sutlnet said:
Still getting this message
modprobe: FATAL: Module vendor-reset not found in directory /lib/modules/6.1.6-1-pve

I will now remove the module and upgrade - and then install vendor-reset again

Thank you - I will give an update
Click to expand...
Don't remove it! Just do the upgrade steps: apt update, apt install pve-headers-6.1 and apt dist-upgrade. You can remove all the 6.1.6 stuff later.
 
leesteken said:
Don't remove it! Just do the upgrade steps: apt update, apt install pve-headers-6.1 and apt dist-upgrade. You can remove all the 6.1.6 stuff later.
Click to expand...

Updated and started Ubuntu VM, but I am getting this message

kvm: vfio: Cannot reset device 0000:02:00.1, no available reset mechanism. kvm: vfio: Cannot reset device 0000:02:00.1, no available reset mechanism. TASK OK

I assume that vendor reset is not active. I am struggling with vendor reset. I followed all the steps from Nick Sherlock's page. Do you have any idea?
 
sutlnet said:
Updated and started Ubuntu VM, but I am getting this message

kvm: vfio: Cannot reset device 0000:02:00.1, no available reset mechanism. kvm: vfio: Cannot reset device 0000:02:00.1, no available reset mechanism. TASK OK

I assume that vendor reset is not active. I am struggling with vendor reset. I followed all the steps from Nick Sherlock's page. Do you have any idea?
Click to expand...
It is not related to vendor-reset. It's the second function of the GPU: the audio device, which used the snd_hda_inte driver. You might want to bind that to vfio-pci (like you did for the audio device of the GT710). It should not be a problem, unless you use the audio of the GPU and experience problems with it.
Did you get vendor-reset working for the AMD GPU (VGA function)? Do you see output on a physical display connected to the RX570 from the Ubuntu VM?
 
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom
Back