[SOLVED] dmesg | grep ecap does not return anything

[2Stoned]

Hi there

I'm trying to set-up GPU pass-thorugh, but dmesg | grep ecap does not return anything. Not even an error. It is a pretty fresh proxmox install, where I followed the tutorial in the forum and set up a first Windows10 VM. I did not use ZFS for the root folder, but created a ZFS mirror for the VMs.
First I added amd_iommu=on as well as iommu=pt to /etc/default/grub, gave it a update-grub and rebootet.

root@pvehost:~# dmesg | grep -e DMAR -e IOMMU
[    0.610867] pci 0000:00:00.2: AMD-Vi: IOMMU performance counters supported
[    0.613588] pci 0000:00:00.2: AMD-Vi: Found IOMMU cap 0x40
[    0.615034] perf/amd_iommu: Detected AMD IOMMU #0 (2 banks, 4 counters/bank).
[    4.132699] AMD-Vi: AMD IOMMUv2 driver by Joerg Roedel <jroedel@suse.de>

I also added the lines to /etc/modules.

As dmesg | grep ecap didn't show anything, even after two reboots, I also created the /etc/kernel/cmdline file with a single line of text:

amd_iommu=on iommu=pt

I gave it a pve-efiboot-tool refresh and in the meantime also a update-grub and rebootet.

root@pvehost:~# update-grub
Generating grub configuration file ...
Found linux image: /boot/vmlinuz-5.3.18-3-pve
Found initrd image: /boot/initrd.img-5.3.18-3-pve
Found linux image: /boot/vmlinuz-5.3.18-2-pve
Found initrd image: /boot/initrd.img-5.3.18-2-pve
Found memtest86+ image: /boot/memtest86+.bin
Found memtest86+ multiboot image: /boot/memtest86+_multiboot.bin
Adding boot menu entry for EFI firmware configuration
done
root@pvehost:~# pve-efiboot-tool refresh
Running hook script 'pve-auto-removal'..
Running hook script 'zz-pve-efiboot'..
Re-executing '/etc/kernel/postinst.d/zz-pve-efiboot' in new private mount namespace..
No /etc/kernel/pve-efiboot-uuids found, skipping ESP sync.
root@pvehost:~#

Still no return. Any ideas?

CPU: AMD Ryzen 3950X, Mainboard: Gigabyte Aurous Ultra, Virtualisation-related things (virtualisation, IOMMU, ARI, ACS) are turned on in the BIOS.

Thank you very much for your reply.

 

[Stefan_R]

Why do you expect 'dmesg' to show messages about ecap? It doesn't show anything on my workstation (also AMD), and passthrough works fine...

Just try the passthrough and see if it works, leave the troubleshooting for the case where it doesn't

PS: you can check /proc/cmdline to see the currently booted commandline, so you can verify your settings did apply

 

[2Stoned]

Why do you expect 'dmesg' to show messages about ecap? It doesn't show anything on my workstation (also AMD), and passthrough works fine...

Well, mainly because the Wiki says so.

And I got the following messages in the host's console, when I try to add a Radeon RX580 to an Arch Linux VM (OVMF BIOS, q35) which only displays "Guest has not initialized the display (yet)" in noVNC.

Apr 23 16:35:06 pvehost kernel: vfio-pci 0000:0a:00.0: vfio_ecap_init: hiding ecap 0x19@0x270
Apr 23 16:35:06 pvehost kernel: vfio-pci 0000:0a:00.0: vfio_ecap_init: hiding ecap 0x1b@0x2d0
Apr 23 16:35:06 pvehost kernel: vfio-pci 0000:0a:00.0: vfio_ecap_init: hiding ecap 0x1e@0x370
Apr 23 16:35:08 pvehost kernel: pcieport 0000:00:03.1: AER: Uncorrected (Non-Fatal) error received: 0000:00:03.1
Apr 23 16:35:08 pvehost kernel: pcieport 0000:00:03.1: AER: PCIe Bus Error: severity=Uncorrected (Non-Fatal), type=Transaction Layer, (Requester ID)
Apr 23 16:35:08 pvehost kernel: pcieport 0000:00:03.1: AER:   device [1022:1483] error status/mask=00100000/04400000
Apr 23 16:35:08 pvehost kernel: pcieport 0000:00:03.1: AER:    [20] UnsupReq               (First)
Apr 23 16:35:08 pvehost kernel: pcieport 0000:00:03.1: AER:   TLP Header: 34000000 0a000010 00000000 80008000
Apr 23 16:35:08 pvehost kernel: pcieport 0000:00:03.1: AER: Device recovery successful

I set up this VM yesterday and it "works" without the GPU added (but it is unbearably slow, even typing in the Arch Terminal (under KDE) takes ~4 seconds to display a character I typed...).

 

[Stefan_R]

Well, mainly because the Wiki says so.

Ah I see, that part is indeed a bit confusing. It was originally intended to only describe Intel CPUs, that's why the 'dmesg' didn't show anything for you. I've updated the part in question to be more clear

I set up this VM yesterday and it "works" without the GPU added (but it is unbearably slow, even typing in the Arch Terminal (under KDE) takes ~4 seconds to display a character I typed...).

Hm, that's not entirely normal either... 4 seconds delay is *very* high, even for non-accelerated graphics (as long as you're not trying to do 4k resolution or something like that).

As to the error message: The 'hiding ecap' is fine, that one always appears. The actual error is the 'AER' part, which usually indicates either incompatible or broken hardware. Note that 'hardware' does not necessarily mean the GPU here, some mainboards can also cause that. Try updating the BIOS, putting the card into a different slot or passing through a different device, just to test if it's related to the GPU.

 

[2Stoned]

Thank you for updating it!

I'm not even doing FullHD over noVNC, so that should not cause the issue. But when installing arch (pure CLI) it was snappy, also downloading packages and installing them went quickly.

I successfully passed through a 10GBit network card (Aquantia AQC107 Chip). I did not test 10Gbit, but 1Gbit worked right away.

I updated the proxmox kernel to 5.4 but it did not change anything. It was just a wild guess.

My initial problem was solved with your edit (@Stefan_R ) - I actually had no problem. To resolve my GPU-passthrough problem opened a distinct thread for it to keep things tidy and separated. Thank you very much for your advice.