ADM GPU passthrough problem to LVC

A

aldariz

Member
Apr 23, 2022
21
4
8
Hi,

I have a AMD RX580 install on my proxmox server, however I need some assistance with what I believe is the last part in getting it to correctly register and work. In the LVC the card is visable however has the a lspci: Unable to load libkmod resources: error -2 when lspci -nnv -s 01:00 is run the the LVC. Also I'm not sure why its registering it as RX570 4gigs when its a RX580 8 gigs.

dmesg attached

I have also done the following:
  • Grub command line, added: intel_iommu=on
  • Kernel modules loaded: vfio, vfio_iommu_type1, vfio_pci, vfio_virqfd
  • Kernel module options:
    1. vfio_iommu_type1 allow_unsafe_interrupts=1
    2. vfio-pci ids=1002:67df,1002:aaf0 disable_vga=1
      kvm ignore_msrs=1
  • Blacklisted video drivers: amdgpu, radeon, nouveau, nvidia
  • Install the amdgpu-install using the https://github.com/RadeonOpenCompute/ROCm/issues/1713, workaround updating the rocm-llvm deb installer control file to include Depends: python3, libc6, libstdc++6|libstdc++8, libstdc++-5-dev|libstdc++-7-dev|libstdc++-10-dev, libgcc-5-dev|libgcc-7-dev|libgcc-10-dev, rocm-cor

pveversion -v
Code: 
proxmox-ve: 7.3-1 (running kernel: 5.15.74-1-pve)
pve-manager: 7.3-3 (running version: 7.3-3/c3928077)
pve-kernel-5.15: 7.2-14
pve-kernel-helper: 7.2-14
pve-kernel-5.13: 7.1-9
pve-kernel-5.15.74-1-pve: 5.15.74-1
pve-kernel-5.15.64-1-pve: 5.15.64-1
pve-kernel-5.13.19-6-pve: 5.13.19-15
pve-kernel-5.13.19-2-pve: 5.13.19-4
ceph-fuse: 15.2.15-pve1
corosync: 3.1.7-pve1
criu: 3.15-1+pve-1
glusterfs-client: 9.2-1
ifupdown2: 3.1.0-1+pmx3
ksm-control-daemon: 1.4-1
libjs-extjs: 7.0.0-1
libknet1: 1.24-pve2
libproxmox-acme-perl: 1.4.2
libproxmox-backup-qemu0: 1.3.1-1
libpve-access-control: 7.2-5
libpve-apiclient-perl: 3.2-1
libpve-common-perl: 7.2-8
libpve-guest-common-perl: 4.2-3
libpve-http-server-perl: 4.1-5
libpve-storage-perl: 7.2-12
libspice-server1: 0.14.3-2.1
lvm2: 2.03.11-2.1
lxc-pve: 5.0.0-3
lxcfs: 4.0.12-pve1
novnc-pve: 1.3.0-3
proxmox-backup-client: 2.2.7-1
proxmox-backup-file-restore: 2.2.7-1
proxmox-mini-journalreader: 1.3-1
proxmox-offline-mirror-helper: 0.5.0-1
proxmox-widget-toolkit: 3.5.3
pve-cluster: 7.3-1
pve-container: 4.4-2
pve-docs: 7.3-1
pve-edk2-firmware: 3.20220526-1
pve-firewall: 4.2-7
pve-firmware: 3.5-6
pve-ha-manager: 3.5.1
pve-i18n: 2.8-1
pve-qemu-kvm: 7.1.0-4
pve-xtermjs: 4.16.0-1
qemu-server: 7.3-1
smartmontools: 7.2-pve3
spiceterm: 3.2-2
swtpm: 0.8.0~bpo11+2
vncterm: 1.7-1
zfsutils-linux: 2.1.6-pve1

LXC - 111 Config
Code: 
GNU nano 5.4                                                                                  /var/lib/lxc/111/config                                                                                           
lxc.cgroup2.memory.max = 8589934592
lxc.cgroup2.memory.swap.max = 8589934592
lxc.rootfs.path = /var/lib/lxc/111/rootfs
lxc.net.0.type = veth
lxc.net.0.veth.pair = veth111i0
lxc.net.0.hwaddr = B6:D8:BC:35:EB:EC
lxc.net.0.name = eth0
lxc.net.0.script.up = /usr/share/lxc/lxcnetaddbr
lxc.idmap = u 0 100000 999
lxc.idmap = g 0 100000 999
lxc.idmap = u 999 999 1
lxc.idmap = g 999 999 1
lxc.idmap = u 1000 101000 64536
lxc.idmap = g 1000 101000 64536
lxc.cgroup2.devices.allow = c 226:0 rwm
lxc.cgroup2.devices.allow = c 226:128 rwm
lxc.mount.entry = /dev/dri/card0 dev/dri/card0 none bind,optional,create=file
lxc.mount.entry = /dev/dri/renderD128 dev/dri/renderD128 none bind,optional,create=file
lxc.cgroup2.cpuset.cpus = 0-7


Ispci on PVE - lspci -nnv -s 01:00
Code: 
01:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Ellesmere [Radeon RX 470/480/570/570X/580/580X/590] [1002:67df] (rev e7) (prog-if 00 [VGA controller])
        Subsystem: Sapphire Technology Limited Radeon RX 570 Pulse 4GB [1da2:e353]
        Flags: bus master, fast devsel, latency 0, IRQ 11
        Memory at e0000000 (64-bit, prefetchable) [size=256M]
        Memory at f0000000 (64-bit, prefetchable) [size=2M]
        I/O ports at e000 [size=256]
        Memory at f7e00000 (32-bit, non-prefetchable) [size=256K]
        Expansion ROM at 000c0000 [disabled] [size=128K]
        Capabilities: [48] Vendor Specific Information: Len=08 <?>
        Capabilities: [50] Power Management version 3
        Capabilities: [58] Express Legacy Endpoint, MSI 00
        Capabilities: [a0] MSI: Enable- Count=1/1 Maskable- 64bit+
        Capabilities: [100] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
        Capabilities: [150] Advanced Error Reporting
        Capabilities: [200] Physical Resizable BAR
        Capabilities: [270] Secondary PCI Express
        Capabilities: [2b0] Address Translation Service (ATS)
        Capabilities: [2c0] Page Request Interface (PRI)
        Capabilities: [2d0] Process Address Space ID (PASID)
        Capabilities: [320] Latency Tolerance Reporting
        Capabilities: [328] Alternative Routing-ID Interpretation (ARI)
        Capabilities: [370] L1 PM Substates
        Kernel modules: amdgpu

01:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Ellesmere HDMI Audio [Radeon RX 470/480 / 570/580/590] [1002:aaf0]
        Subsystem: Sapphire Technology Limited Ellesmere HDMI Audio [Radeon RX 470/480 / 570/580/590] [1da2:aaf0]
        Flags: bus master, fast devsel, latency 0, IRQ 49
        Memory at f7e60000 (64-bit, non-prefetchable) [size=16K]
        Capabilities: [48] Vendor Specific Information: Len=08 <?>
        Capabilities: [50] Power Management version 3
        Capabilities: [58] Express Legacy Endpoint, MSI 00
        Capabilities: [a0] MSI: Enable+ Count=1/1 Maskable- 64bit+
        Capabilities: [100] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
        Capabilities: [150] Advanced Error Reporting
        Capabilities: [328] Alternative Routing-ID Interpretation (ARI)
        Kernel driver in use: snd_hda_intel
        Kernel modules: snd_hda_intel

Ispci in LVC 111 - lspci -nnv -s 01:00
Code: 
01:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Ellesmere [Radeon RX 470/480/570/570X/580/580X/590] [1002:67df] (rev e7) (prog-if 00 [VGA controller])
        Subsystem: Sapphire Technology Limited Radeon RX 570 Pulse 4GB [1da2:e353]
        Flags: bus master, fast devsel, latency 0, IRQ 11
        Memory at e0000000 (64-bit, prefetchable) [size=256M]
        Memory at f0000000 (64-bit, prefetchable) [size=2M]
        I/O ports at e000 [size=256]
        Memory at f7e00000 (32-bit, non-prefetchable) [size=256K]
        Expansion ROM at 000c0000 [disabled] [size=128K]
        Capabilities: <access denied>
lspci: Unable to load libkmod resources: error -2

01:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Ellesmere HDMI Audio [Radeon RX 470/480 / 570/580/590] [1002:aaf0]
        Subsystem: Sapphire Technology Limited Ellesmere HDMI Audio [Radeon RX 470/480 / 570/580/590] [1da2:aaf0]
        Flags: bus master, fast devsel, latency 0, IRQ 49
        Memory at f7e60000 (64-bit, non-prefetchable) [size=16K]
        Capabilities: <access denied>
        Kernel driver in use: snd_hda_intel
 

Attachments

Hello,

did you fix the problem?
I cannot get the RX 580 to pci passthrough for my macos vm. Any suggetions?
thx
 
biskero said:
Hello,

did you fix the problem?
I cannot get the RX 580 to pci passthrough for my macos vm. Any suggetions?
thx
Click to expand...
Hi,
Yes, for a VM it should work easier as it can be undertkaen via the GUI. As I've gotten both a RX580 and RX6800 to work on MacOS Monterey/Windows 11.
 
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom