AMD RX 6600 not working in windows VM

E

Enspist

New Member
Jul 16, 2023
12
1
3
Hi all,

Sorry if this is not the right place to post this. It's my first time asking here.

I currently have a Proxmox VE server on some "old" hardware. It's a Radeon 5600, RX 6600, 32 GB ram, and all on an Asus b550 micro-atx mobo.

I am trying to build a Windows VM using this guide here. I have gotten all the way to the end of it; however, the VM doesn't fully see the GPU. I see it in Device Manager with the orange triangle; however, I don't see it in Task Manager. I also see it in GPU-Z and HWInfo. It doesn't have clock speeds, though, so I don't think Windows is using it. I have tried googling for AMD GPU issues with proxmox passthrough, but everything I find is just another simple guide on doing passthrough—nothing for when it doesn't work right. I tried running the AMD drivers from their website. However, they say they cannot detect the GPU, which is weird considering other programs can. I also ran driver easy, which did see the GPU and updated the drivers; however, that did not fix the issue.

Any Help is greatly appreciated. I would like to get this working fully so that I can remote into the machine whenever I need.
 
What would help is actually the output of:

1: lspci -k | grep VGA -A 3
2: dmesg | grep -e DMAR -e IOMMU -e VT-d
3: Make a bash script "iommu.sh" make it executable: chmod +x iommu.sh and post the output.
Code: 
#!/bin/bash
shopt -s nullglob
for g in `find /sys/kernel/iommu_groups/* -maxdepth 0 -type d | sort -V`; do
    echo "IOMMU Group ${g##*/}:"
    for d in $g/devices/*; do
        echo -e "\t$(lspci -nns ${d##*/})"
    done;
done;
4. Your VM Config cat /etc/pve/qemu-server/1XX.conf

Otherwise no one will be able to help with no info provided other as "it doesn't work" xD
 
Last edited:
Since the GPU is visible inside the VM, IOMMU is probably setup right but the GPU might not reset properly. 6800 and 69x0 almost always work fine, but the other 6000-series GPU do not always and are not supported by vendor-reset. Make sure to disable Resizable BAR in the motherboard BIOS, since it is not supported (yet) by VFIO. Make sure to boot your Proxmox with another GPU and to early bind the 6600 to vfio-pci.
 
Ramalama said:
What would help is actually the output of:

1: lspci -k | grep VGA -A 3
2: dmesg | grep -e DMAR -e IOMMU -e VT-d
3: Make a bash script "iommu.sh" make it executable: chmod +x iommu.sh and post the output.
Code: 
#!/bin/bash
shopt -s nullglob
for g in `find /sys/kernel/iommu_groups/* -maxdepth 0 -type d | sort -V`; do
    echo "IOMMU Group ${g##*/}:"
    for d in $g/devices/*; do
        echo -e "\t$(lspci -nns ${d##*/})"
    done;
done;
4. Your VM Config cat /etc/pve/qemu-server/1XX.conf

Otherwise no one will be able to help with no info provided other as "it doesn't work" xD
Click to expand...
sorry I just got around to this. I have been super busy the past month or so


1)
Code: 
0d:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Navi 23 (rev c7)
        Subsystem: Tul Corporation / PowerColor Navi 23
        Kernel driver in use: vfio-pci
        Kernel modules: amdgpu

2) I got no output when running this one

3)
Code: 
IOMMU Group 0:
        00:01.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
IOMMU Group 1:
        00:01.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse GPP Bridge [1022:1483]
IOMMU Group 2:
        00:01.2 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse GPP Bridge [1022:1483]
IOMMU Group 3:
        00:02.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
IOMMU Group 4:
        00:03.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
IOMMU Group 5:
        00:03.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse GPP Bridge [1022:1483]
IOMMU Group 6:
        00:04.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
IOMMU Group 7:
        00:05.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
IOMMU Group 8:
        00:07.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
IOMMU Group 9:
        00:07.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Internal PCIe GPP Bridge 0 to bus[E:B] [1022:1484]
IOMMU Group 10:
        00:08.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
IOMMU Group 11:
        00:08.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Internal PCIe GPP Bridge 0 to bus[E:B] [1022:1484]
IOMMU Group 12:
        00:14.0 SMBus [0c05]: Advanced Micro Devices, Inc. [AMD] FCH SMBus Controller [1022:790b] (rev 61)
        00:14.3 ISA bridge [0601]: Advanced Micro Devices, Inc. [AMD] FCH LPC Bridge [1022:790e] (rev 51)
IOMMU Group 13:
        00:18.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Matisse Device 24: Function 0 [1022:1440]
        00:18.1 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Matisse Device 24: Function 1 [1022:1441]
        00:18.2 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Matisse Device 24: Function 2 [1022:1442]
        00:18.3 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Matisse Device 24: Function 3 [1022:1443]
        00:18.4 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Matisse Device 24: Function 4 [1022:1444]
        00:18.5 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Matisse Device 24: Function 5 [1022:1445]
        00:18.6 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Matisse Device 24: Function 6 [1022:1446]
        00:18.7 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Matisse Device 24: Function 7 [1022:1447]
IOMMU Group 14:
        01:00.0 Non-Volatile memory controller [0108]: Samsung Electronics Co Ltd NVMe SSD Controller PM9A1/980PRO [144d:a80a]
IOMMU Group 15:
        02:00.0 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] Device [1022:43ee]
IOMMU Group 16:
        02:00.1 SATA controller [0106]: Advanced Micro Devices, Inc. [AMD] Device [1022:43eb]
IOMMU Group 17:
        02:00.2 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:43e9]
IOMMU Group 18:
        03:00.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:43ea]
IOMMU Group 19:
        03:01.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:43ea]
IOMMU Group 20:
        03:02.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:43ea]
IOMMU Group 21:
        03:03.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:43ea]
IOMMU Group 22:
        03:04.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:43ea]
IOMMU Group 23:
        03:08.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:43ea]
IOMMU Group 24:
        03:09.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:43ea]
IOMMU Group 25:
        07:00.0 Ethernet controller [0200]: Realtek Semiconductor Co., Ltd. RTL8125 2.5GbE Controller [10ec:8125] (rev 05)
IOMMU Group 26:
        08:00.0 Non-Volatile memory controller [0108]: Phison Electronics Corporation Device [1987:5015] (rev 01)
IOMMU Group 27:
        09:00.0 Network controller [0280]: Realtek Semiconductor Co., Ltd. RTL8821CE 802.11ac PCIe Wireless Network Adapter [10ec:c821]
IOMMU Group 28:
        0a:00.0 Ethernet controller [0200]: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller [10ec:8168] (rev 15)
IOMMU Group 29:
        0b:00.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD/ATI] Navi 10 XL Upstream Port of PCI Express Switch [1002:1478] (rev c7)
IOMMU Group 30:
        0c:00.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD/ATI] Navi 10 XL Downstream Port of PCI Express Switch [1002:1479]
IOMMU Group 31:
        0d:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Navi 23 [1002:73ff] (rev c7)
IOMMU Group 32:
        0d:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Device [1002:ab28]
IOMMU Group 33:
        0e:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Function [1022:148a]
IOMMU Group 34:
        0f:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Reserved SPP [1022:1485]
IOMMU Group 35:
        0f:00.1 Encryption controller [1080]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Cryptographic Coprocessor PSPCPP [1022:1486]
IOMMU Group 36:
        0f:00.3 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] Matisse USB 3.0 Host Controller [1022:149c]
IOMMU Group 37:
        0f:00.4 Audio device [0403]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse HD Audio Controller [1022:1487]

4)
Code: 
args: -cpu 'host,+kvm_pv_unhalt,+kvm_pv_eoi,hv_vendor_id=NV43FIX,kvm=off'
bios: ovmf
boot: order=scsi0;net0
cores: 4
cpu: host,hidden=1,flags=+pcid
efidisk0: vmstorage:103/vm-103-disk-4.raw,efitype=4m,pre-enrolled-keys=1,size=528K
hostpci0: 0000:0d:00,pcie=1,romfile=Powercolor.RX6600.8192.210830.rom
machine: q35
memory: 16384
meta: creation-qemu=7.1.0,ctime=1689445614
name: win10
net0: virtio=E2:4F:25:B3:BC:B2,bridge=vmbr0,firewall=1
numa: 0
ostype: win10
scsi0: local-lvm:vm-103-disk-2,iothread=1,size=256G
scsihw: virtio-scsi-single
smbios1: uuid=92c69b43-b88e-4b4d-9161-964f92c53ee2
sockets: 1
tpmstate0: vmstorage:103/vm-103-disk-5.raw,size=4M,version=v2.0
vmgenid: d40a80d3-2f02-4831-ba76-382d7e2081f5
 
Last edited:
leesteken said:
Since the GPU is visible inside the VM, IOMMU is probably setup right but the GPU might not reset properly. 6800 and 69x0 almost always work fine, but the other 6000-series GPU do not always and are not supported by vendor-reset. Make sure to disable Resizable BAR in the motherboard BIOS, since it is not supported (yet) by VFIO. Make sure to boot your Proxmox with another GPU and to early bind the 6600 to vfio-pci.
Click to expand...
Resize Bar is off, if I add a second GPU, how do I make sure that proxmox is using that one to boot and not the 660? also how do I add the 6600 to vfi-pci? I followed a guide and I am not sure if I already did that step or not
 
Enspist said:
Resize Bar is off, if I add a second GPU, how do I make sure that proxmox is using that one to boot and not the 660?
Click to expand...
That depends on the motherboard. Very few allow you to choose (in the BIOS), sometimes switching CSM on or off (in the BIOS) switches the boot GPU. Otherwise put the GPU you want to use for boot in the first/best PCIe slot. Make sure that a working display is connected to the boot GPU, as some will switch to the other GPU that has a display.
Enspist said:
also how do I add the 6600 to vfi-pci? I followed a guide and I am not sure if I already did that step or not
Click to expand...
Use options vfio-pci ids= as shown in the manual. You should no longer have a Proxmox host console and no output on the GPU, except for the BIOS POST and the bootloader (which you can fix with a second GPU, see above).

Preparing your system will give the GPU the best chance to work inside a VM (maybe only once, after which you need to reboot the Proxmox host). It's quite possible that the RX6600 (in general or your specific make and model) does not work with PCI(e) passthrough and maybe there is no work-around yet.
Please let me know if someone on the internet has been successful in passthrough of a RX6600 (or lower). The 6700's have been hit or miss and only 6800+ worked reliable (or older generations supported by vendor-reset).
 
1
Enspist said:
sorry I just got around to this. I have been super busy the past month or so


1)
Code: 
0d:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Navi 23 (rev c7)
        Subsystem: Tul Corporation / PowerColor Navi 23
        Kernel driver in use: vfio-pci
        Kernel modules: amdgpu

2) I got no output when running this one

3)
Code: 
IOMMU Group 0:
        00:01.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
IOMMU Group 1:
        00:01.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse GPP Bridge [1022:1483]
IOMMU Group 2:
        00:01.2 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse GPP Bridge [1022:1483]
IOMMU Group 3:
        00:02.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
IOMMU Group 4:
        00:03.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
IOMMU Group 5:
        00:03.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse GPP Bridge [1022:1483]
IOMMU Group 6:
        00:04.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
IOMMU Group 7:
        00:05.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
IOMMU Group 8:
        00:07.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
IOMMU Group 9:
        00:07.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Internal PCIe GPP Bridge 0 to bus[E:B] [1022:1484]
IOMMU Group 10:
        00:08.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
IOMMU Group 11:
        00:08.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Internal PCIe GPP Bridge 0 to bus[E:B] [1022:1484]
IOMMU Group 12:
        00:14.0 SMBus [0c05]: Advanced Micro Devices, Inc. [AMD] FCH SMBus Controller [1022:790b] (rev 61)
        00:14.3 ISA bridge [0601]: Advanced Micro Devices, Inc. [AMD] FCH LPC Bridge [1022:790e] (rev 51)
IOMMU Group 13:
        00:18.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Matisse Device 24: Function 0 [1022:1440]
        00:18.1 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Matisse Device 24: Function 1 [1022:1441]
        00:18.2 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Matisse Device 24: Function 2 [1022:1442]
        00:18.3 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Matisse Device 24: Function 3 [1022:1443]
        00:18.4 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Matisse Device 24: Function 4 [1022:1444]
        00:18.5 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Matisse Device 24: Function 5 [1022:1445]
        00:18.6 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Matisse Device 24: Function 6 [1022:1446]
        00:18.7 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Matisse Device 24: Function 7 [1022:1447]
IOMMU Group 14:
        01:00.0 Non-Volatile memory controller [0108]: Samsung Electronics Co Ltd NVMe SSD Controller PM9A1/980PRO [144d:a80a]
IOMMU Group 15:
        02:00.0 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] Device [1022:43ee]
IOMMU Group 16:
        02:00.1 SATA controller [0106]: Advanced Micro Devices, Inc. [AMD] Device [1022:43eb]
IOMMU Group 17:
        02:00.2 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:43e9]
IOMMU Group 18:
        03:00.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:43ea]
IOMMU Group 19:
        03:01.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:43ea]
IOMMU Group 20:
        03:02.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:43ea]
IOMMU Group 21:
        03:03.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:43ea]
IOMMU Group 22:
        03:04.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:43ea]
IOMMU Group 23:
        03:08.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:43ea]
IOMMU Group 24:
        03:09.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:43ea]
IOMMU Group 25:
        07:00.0 Ethernet controller [0200]: Realtek Semiconductor Co., Ltd. RTL8125 2.5GbE Controller [10ec:8125] (rev 05)
IOMMU Group 26:
        08:00.0 Non-Volatile memory controller [0108]: Phison Electronics Corporation Device [1987:5015] (rev 01)
IOMMU Group 27:
        09:00.0 Network controller [0280]: Realtek Semiconductor Co., Ltd. RTL8821CE 802.11ac PCIe Wireless Network Adapter [10ec:c821]
IOMMU Group 28:
        0a:00.0 Ethernet controller [0200]: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller [10ec:8168] (rev 15)
IOMMU Group 29:
        0b:00.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD/ATI] Navi 10 XL Upstream Port of PCI Express Switch [1002:1478] (rev c7)
IOMMU Group 30:
        0c:00.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD/ATI] Navi 10 XL Downstream Port of PCI Express Switch [1002:1479]
IOMMU Group 31:
        0d:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Navi 23 [1002:73ff] (rev c7)
IOMMU Group 32:
        0d:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Device [1002:ab28]
IOMMU Group 33:
        0e:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Function [1022:148a]
IOMMU Group 34:
        0f:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Reserved SPP [1022:1485]
IOMMU Group 35:
        0f:00.1 Encryption controller [1080]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Cryptographic Coprocessor PSPCPP [1022:1486]
IOMMU Group 36:
        0f:00.3 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] Matisse USB 3.0 Host Controller [1022:149c]
IOMMU Group 37:
        0f:00.4 Audio device [0403]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse HD Audio Controller [1022:1487]

4)
Code: 
args: -cpu 'host,+kvm_pv_unhalt,+kvm_pv_eoi,hv_vendor_id=NV43FIX,kvm=off'
bios: ovmf
boot: order=scsi0;net0
cores: 4
cpu: host,hidden=1,flags=+pcid
efidisk0: vmstorage:103/vm-103-disk-4.raw,efitype=4m,pre-enrolled-keys=1,size=528K
hostpci0: 0000:0d:00,pcie=1,romfile=Powercolor.RX6600.8192.210830.rom
machine: q35
memory: 16384
meta: creation-qemu=7.1.0,ctime=1689445614
name: win10
net0: virtio=E2:4F:25:B3:BC:B2,bridge=vmbr0,firewall=1
numa: 0
ostype: win10
scsi0: local-lvm:vm-103-disk-2,iothread=1,size=256G
scsihw: virtio-scsi-single
smbios1: uuid=92c69b43-b88e-4b4d-9161-964f92c53ee2
sockets: 1
tpmstate0: vmstorage:103/vm-103-disk-5.raw,size=4M,version=v2.0
vmgenid: d40a80d3-2f02-4831-ba76-382d7e2081f5
Click to expand...
1: is amazing, you setted it up correctly, using vfio-pcie driver already.

2: is very weird, it should show some output, i don't get it because you have accordingly to 3 iommu groups.

3. I don't see your GPU in iommu groups, this is probably the root cause why it doesn't work.
If your gpu isn't in a iommu group, you cannot passthrough it, that's an absolutely normal restriction.

4. Looks correct to me!


However, it looks like you did the Proxmox part right! Means setup Linux for passthough.

But something isn't right on the bios side, i cannot say if you didn't enabled iommu support (because point 2 has no output),
Or if you're restricted because your mainboard/cpu doesn't support passing through gpu, because it's not in any group (point 3 iommu groups)

I think actually that you didn't enabled iommu support in your bios, maybe there is no option even, or there is one that simply does nothing.

Can you check in your bios again?
 
Ramalama said:
3. I don't see your GPU in iommu groups, this is probably the root cause why it doesn't work.
If your gpu isn't in a iommu group, you cannot passthrough it, that's an absolutely normal restriction.

But something isn't right on the bios side, i cannot say if you didn't enabled iommu support (because point 2 has no output),
Or if you're restricted because your mainboard/cpu doesn't support passing through gpu, because it's not in any group (point 3 iommu groups)
Click to expand...
See group 31 and 32 where device 0d:00 is present.
Ramalama said:
I think actually that you didn't enabled iommu support in your bios, maybe there is no option even, or there is one that simply does nothing.
Click to expand...
There are separate IOMMU groups; therefore IOMMU is enabled. The VM starts withour an error about IOMMU not being enabled; therefore IOMMU is enabled. The VM sees the GPU (but it does not work inside the VM); therefore IOMMU is enabled. Maybe you meant some other BIOS setting? Can you explain a bit more please?
 
leesteken said:
That depends on the motherboard. Very few allow you to choose (in the BIOS), sometimes switching CSM on or off (in the BIOS) switches the boot GPU. Otherwise put the GPU you want to use for boot in the first/best PCIe slot. Make sure that a working display is connected to the boot GPU, as some will switch to the other GPU that has a display.
Click to expand...
so my only issue here would be that I have a micro-atx motherboard that I am using. It is what I had laying around when I decided to start my server. I managed to find an Nvidia quadro at work that has a x4 connector so that I can add it to the motherboard, however, I would have no way to put it in the top slot to boot with and then add the 6600 in a lower slot. I can see what would happen if I put it in the bottom slot and adjusting csm, maybe that will do something.
 
also thanks for the help guys. I know I am probably making this more difficult than it needs to be by using a non-traditional motherboard, an AMD CPU, and GPU, and not exactly understanding what I am doing. You are both super helpful.
 
Ramalama said:
2: is very weird, it should show some output, i don't get it because you have accordingly to 3 iommu groups.
Click to expand...
idk why it didn't output the first time, I just ran it again and got

Code: 
[    0.000000] Warning: PCIe ACS overrides enabled; This may allow non-IOMMU protected peer-to-peer DMA
[    0.644293] pci 0000:00:00.2: AMD-Vi: IOMMU performance counters supported
[    0.645093] pci 0000:00:00.2: AMD-Vi: Found IOMMU cap 0x40
[    0.652971] perf/amd_iommu: Detected AMD IOMMU #0 (2 banks, 4 counters/bank).
[    5.866082] AMD-Vi: AMD IOMMUv2 loaded and initialized
 
Enspist said:
so my only issue here would be that I have a micro-atx motherboard that I am using. It is what I had laying around when I decided to start my server. I managed to find an Nvidia quadro at work that has a x4 connector so that I can add it to the motherboard, however, I would have no way to put it in the top slot to boot with and then add the 6600 in a lower slot. I can see what would happen if I put it in the bottom slot and adjusting csm, maybe that will do something.
Click to expand...
Enspist said:
also thanks for the help guys. I know I am probably making this more difficult than it needs to be by using a non-traditional motherboard, an AMD CPU, and GPU, and not exactly understanding what I am doing. You are both super helpful.
Click to expand...
AMD Ryzen usually works fine but except for the X570 chipset, you have limited passthrough choices. I don't know which motherboard you have, but passthrough is probably limited to the first/only PCIe x16 slot and the first M.2 slot (4 PCIe lanes). Everything else will probably end up in the big chipset IOMMU group.
Are you using the insecure pcie_acs_override (cat /proc/cmdline) to make Proxmox ignore the actual IOMMU groups?

A 5700 (about as fast as a 6600) will most likely work with passthrough (with the help of vendor-reset) and it looks like at least one 6600 works.

You could try booting your VM (with Display set to None) with the latest Ubuntu installer ISO (without installing it) to see if you get output from the RX6600 on a physical display. That way you can test passthrough (and hard-reset the VM) without cause problems for, or having interference from, Windows.
 
leesteken said:
AMD Ryzen usually works fine but except for the X570 chipset, you have limited passthrough choices. I don't know which motherboard you have, but passthrough is probably limited to the first/only PCIe x16 slot and the first M.2 slot (4 PCIe lanes). Everything else will probably end up in the big chipset IOMMU group.
Click to expand...
The motherboard I am running is an Asus b550m-a ac, I am not sure if that affects if the pcie slots are running through the cpu or chipset.

leesteken said:
Are you using the insecure pcie_acs_override (cat /proc/cmdline) to make Proxmox ignore the actual IOMMU groups?
Click to expand...
Here is the output running that command:

Code: 
BOOT_IMAGE=/boot/vmlinuz-5.15.74-1-pve root=/dev/mapper/pve-root ro quiet amd_iommu=on iommu=pt pcie_acs_override=downstream,multifunction nofb nomodeset iommu=1 video=vesafb:off,efifb:off disable_idle_d3=1

leesteken said:
You could try booting your VM (with Display set to None) with the latest Ubuntu installer ISO (without installing it) to see if you get output from the RX6600 on a physical display. That way you can test passthrough (and hard-reset the VM) without cause problems for, or having interference from, Windows.
Click to expand...
So I have tried booting with the display set to none and the gpu set to primary gpu, and I will sometimes get an output when I connect to the vm, but not always.

when you say boot with the ubuntu iso, how does that work?
 
Enspist said:
The motherboard I am running is an Asus b550m-a ac, I am not sure if that affects if the pcie slots are running through the cpu or chipset.
Click to expand...
No, all AM4 chipsets are the same except for X570 and X570S.
Enspist said:
Here is the output running that command:

BOOT_IMAGE=/boot/vmlinuz-5.15.74-1-pve root=/dev/mapper/pve-root ro quiet amd_iommu=on iommu=pt pcie_acs_override=downstream,multifunction nofb nomodeset iommu=1 video=vesafb:off,efifb:off disable_idle_d3=1
Click to expand...
amd_iommu=on does nothing because it is on by default on AMD. iommu=pt probably does nothing on your system and is for non-passed through devices. pcie_acs_override=downstream,multifunction is insecure and a last resort and you don't need it for the x16 PCIe slot. iommu=1/[ICODE] does nothing. [ICODE]video=vesafb:off,efifb:off is invalid nowadays and even does nothing when you write it correctly, and just like nofb nomodeset, it does not work since Proxmox 7.2. disable_idle_d3=1 might be (part of) a work-around but I would first try without it.

What you need for passthrough of the boot (or only) GPU is this work-around. I'm sorry for not asking this earlier, as it would be the first thing to advise to your situation.

Enspist said:
So I have tried booting with the display set to none and the gpu set to primary gpu, and I will sometimes get an output when I connect to the vm, but not always.
Click to expand...
Don't use the Primary GPU setting as it is intended for NVidia GPUs (which is not clear at all from the name).
Enspist said:
when you say boot with the ubuntu iso, how does that work?
Click to expand...
Download the ISO and insert it into the virtual CD-ROM. Then boot the VM from the virtual CD-ROM (via Option, enable the virtual optical drive and make it the first one).
 
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom
Back