Update an error wit GPU passthrought

[MadMonkey]

Hello everybody

Yesterday all was well, my vm worked perfectly with my GPU (RX580) but i did a update upgrade and turn off. Today i tried to turn on a vm and not works, give me the error

unable to read tail (got 0 bytes)

I tried to solve with this
https://wiki.archlinux.org/title/PCI_passthrough_via_OVMF#"BAR_3:_cannot_reserve_[mem]"_error_in_dmesg_after_starting_virtual_machine

but when i try to do this, the terminal freeze and not works, also i can't reboot the server i have to press the turn on button
# echo 1 > /sys/bus/pci/devices/mipcidevice/remove
# echo 1 > /sys/bus/pci/rescan

My kernel version is
5.15.60-1-pve

And my pve version is Virtual Environment 7.2-11

Anyone know how can i solve this problem?

Thanks in advance

 

[leesteken]

Go back to the earlier kernel (by selecting it in the boot menu) and see if it works.

Give some information about your hardware. Is the RX580 used during the boot of the system and the Proxmox host console? If yes, Then the new kernel version or Debian 11.5 broke amdgpu unbind/unload and you need to early bind the GPU to vfio-pci and maybe the initcall_blacklist=sysfb_init work-around. Or use the new 5.19 kernel.

PS: Please don't private message about your thread and let met discover it in my own time, which I 'm spending here freely and for free.

EDIT: I asked if it was the only GPU but I should have asked whether the GPU is used during boot and by the host console.

 

[MadMonkey]

Hi, first sorry for bother you and sent you a private message.
About the problem - the GPU is used during the boot and stop when the vm is turnon, i remember i did in other PC and th card not was used during the boot, but i think that i did the same steps.

about the config


in /etc/default/grub


i did a vendor-reset to work, also i have this in the modrpobe.d


About hardware i have a
AMD Ryzen 9 5900X
32GB ram
Graphic card RX580

Thanks for support, and sorry for bother you again

Edit: i've checked and if i use kernel 5.13.19-6-pve works perfectly

 

[leesteken]

Then my expectation was correct and also the work-arounds. I cannot check if 1022:1480 is correct but you might also need to use the numeric ID for the audio device of the GPU. And you don't need to do it twice as you are doing now.
video=vesafbL:off video=efifb:off does nothing anymore, just remove them.
Use initcall_blacklist=sysfb_init instead.

EDIT: I'm not sure if upgrading to kernel 5.19 will fix your issue when using early binding or blacklisting of amdgpu. The simplest change is probably initcall_blacklist=sysfb_init as discovered in this thread.

 

[MadMonkey]

Hi, thanks for answer, i suposes that i did something wrong, i i Use this initcall_blacklist=sysfb_init only doesn't work, if i add amdgpu to the blacklist, i can turnon the VM but onliy appera a lot of strange characters in the screen

about the ids i change to test i show you all


Now i have, i supose they are the correct one.

About upgrade the kernel i didn't try it

 

[leesteken]

Do you have vendor-reset installed? I assumed you have in order to make the RX580 reset properly, but I followed this guide. With kernel 5.15, you also need to use device_specific reset: echo device_specific >/sys/bus/pci/devices/0000:2d:00.0/reset_method before starting the VM (after every reboot of the Proxmox host).

Note that you need to run update-initramfs when making changes to files in /etc/modprobe.d/ and maybe update-grub or proxmox-boot-tool refresh, which is also need when making changes to /etc/default/grub or /etc/kernel/cmdline respectively. It all depends on your Proxmox host bootloader. You probably know all this, but people sometimes forget to do it.

Could you maybe paste the text from the console as text (using code or inline-code)? The screenshot fragments are hard to read and sometimes appear to leave out a character or overlap.

 

[MadMonkey]

Yep, i have the vendor-reset installed and works ok with the lasts kernel versions, also i had a script to do de device_specific reset, you told me in other post . Also Thanks about the update-initramfs and update-grub in this case i didn't forget. but continue without works
My cards id

2d:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Ellesmere [Radeon RX 470/480/570/570X/580/580X/590] [1002:67df] (rev e7)
2d:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Ellesmere HDMI Audio [Radeon RX 470/480 / 570/580/590] [1002:aaf0]
2e:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Function [1022:148a]
2f:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Reserved SPP [1022:1485]

My /etc/default/grub

GRUB_DEFAULT=0
GRUB_TIMEOUT=5
GRUB_DISTRIBUTOR=`lsb_release -i -s 2> /dev/null || echo Debian`
#GRUB_CMDLINE_LINUX_DEFAULT="quiet"
#GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on iommu=pt video=vesafb:off video=efifb:off"
GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on iommu=pt initcall_blacklist=sysfb_init"
GRUB_CMDLINE_LINUX=""

my /etc/modprobe.d/vfio

#options vfio-pci ids=1002:67df,1002:aaf
options vfio-pci ids=1022:148a,1022:1485

I don't know what can i do, any idea?

Thanks for expend your time with my problems

 

[leesteken]

You can check if the kernel parameters are passed correctly by looking at the output of cat /proc/cmdline

You need to rename /etc/modprobe.d/vfio to /etc/modprobe.d/vfio.conf because only files with names that end with .conf are used. Therfore you are currently not early binding the GPU because your file is ignored.

Make sure that vendor-reset and vfio-pci are loaded before amdgpu and snd_hda_intel by adding this to /etc/modprobe.d/vfio.conf:
softdep vfio_pci pre: vendor_reset softdep amdgpu pre: vfio_pci softdep snd_hda_intel pre: vfio_pci
Don't forget to run update-initramfs -u to update it for your currently running kernel and reboot. You can check with lspci -nnks 2d:00 before starting the VM.

 

[MadMonkey]

Ok i have this now

works but no too well

cat /proc/cmdline
initrd=\EFI\proxmox\5.15.60-1-pve\initrd.img-5.15.60-1-pve root=ZFS=rpool/ROOT/pve-1 boot=zfs

cat /etc/modprobe.d/vfio.conf
softdep vfio_pci pre: vendor_reset softdep amdgpu pre: vfio vfio_pci softdep snd_hda_intel pre: vfio_pci options vfio-pci ids=1002:67df,1002:aaf #options vfio-pci ids=1022:148a,1022:1485

lspci -nnks 2d:00
2d:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Ellesmere [Radeon RX 470/480/570/570X/580/580X/590] [1002:67df] (rev e7) Subsystem: Sapphire Technology Limited Nitro+ Radeon RX 570/580/590 [1da2:e366] Kernel driver in use: vfio-pci Kernel modules: amdgpu 2d:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Ellesmere HDMI Audio [Radeon RX 470/480 / 570/580/590] [1002:aaf0] Subsystem: Sapphire Technology Limited Ellesmere HDMI Audio [Radeon RX 470/480 / 570/580/590] [1da2:aaf0] Kernel driver in use: vfio-pci Kernel modules: snd_hda_intel

anythinh wrong?

 

[leesteken]

cat /proc/cmdline
initrd=\EFI\proxmox\5.15.60-1-pve\initrd.img-5.15.60-1-pve root=ZFS=rpool/ROOT/pve-1 boot=zfs

It appears you have Proxmox on ZFS and GRUB is not used. Your changes to /etc/default/grub never had any effect and you need to edit /etc/kernel/cmdline as described in the manual (and to which I referred before). Just add initcall_blacklist=sysfb_init to it. No need for intel_iommu=ona because you don't have an Intel system and no need foriommu=pt because that does not influence passthrough.
Make sure to put everything on one line (it's not the same as GRUB) and run proxmox-boot-tool refresh and reboot to active the changes. Note that you will not see Proxmox boot messages and won't have a host console anymore, and you need to do everything via ssh from another computer.

 

[MadMonkey]

You are right as always, thanks for support, i never thought in this option. You rock.

Thanks for your time and patience.