Problem Error 43 on AMD RX 6750XT FE

[ZoThyx]

Hi,
I need help for fixed error 43 on Windows 11 with my RX 6750XT.
Linux kernel :
Linux 6.2.16-3-pve x86_64

/etc/default/grub :

GRUB_CMDLINE_LINUX_DEFAULT="quiet amd_iommu=on initcall_blacklist=sysfb_init iommu=pt video=vesafb:off video=efifb:off"

dmesg | grep -e IOMMU :

[    0.022685] DMAR: IOMMU enabled
[    0.294304] pci 0000:00:00.2: AMD-Vi: IOMMU performance counters supported
[    0.294946] pci 0000:00:00.2: AMD-Vi: Found IOMMU cap 0x40
[    0.358187] perf/amd_iommu: Detected AMD IOMMU #0 (2 banks, 4 counters/bank).

/etc/modules :

# /etc/modules: kernel modules to load at boot time.
#
# This file contains the names of kernel modules that should be loaded
# at boot time, one per line. Lines beginning with "#" are ignored.
# Parameters can be specified after the module name.

vfio
vfio_iommu_type1
vfio_pci

dmesg | grep -i vfio :
[ 4.456292] VFIO - User Level meta-driver version: 0.3

dmesg | grep 'remapping' :
[ 0.294952] AMD-Vi: Interrupt remapping enabled

echo "options vfio_iommu_type1 allow_unsafe_interrupts=1" > /etc/modprobe.d/iommu_unsafe_interrupts.conf

lspci -nn | grep 'AMD' :

03:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Navi 22 [Radeon RX 6700/6700 XT/6750 XT / 6800M/6850M XT] [1002:73df] (rev c0)
03:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Navi 21/23 HDMI/DP Audio Controller [1002:ab28]

Blacklisting drivers :
echo "options vfio-pci ids=1002:67df,1002:aaf0" > /etc/modprobe.d/vfio.conf

# AMD drivers
echo "blacklist radeon" >> /etc/modprobe.d/blacklist.conf
echo "blacklist amdgpu" >> /etc/modprobe.d/blacklist.conf
# NVIDIA drivers
echo "blacklist nouveau" >> /etc/modprobe.d/blacklist.conf
echo "blacklist nvidia" >> /etc/modprobe.d/blacklist.conf
echo "blacklist nvidiafb" >> /etc/modprobe.d/blacklist.conf
echo "blacklist nvidia_drm" >> /etc/modprobe.d/blacklist.conf
# Intel drivers
echo "snd_hda_intel" >> /etc/modprobe.d/blacklist.conf
echo "snd_hda_codec_hdmi" >> /etc/modprobe.d/blacklist.conf
echo "i915" >> /etc/modprobe.d/blacklist.conf

And then, I rebooted. After that, I set up a Windows 11 VM installation and installed the driver, but I am encountering error code 43. Can someone help me resolve this issue ?

 

[leesteken]

Linux kernel :
Linux 6.2.16-3-pve x86_64

Stop using kernel version 6.2 as it does not get updates/fixes. What version of PVE are you using? I suggest moving to kernel version 6.5.

/etc/default/grub :

GRUB_CMDLINE_LINUX_DEFAULT="quiet amd_iommu=on initcall_blacklist=sysfb_init iommu=pt video=vesafb:off video=efifb:off"

amd_iommu=on video=vesafb:off video=efifb:off all do nothing. iommu=pt does mostly nothing, but initcall_blacklist=sysfb_init is necessary if you boot Proxmox with the 6750XT.
What is the output of cat /proc/cmdline (to check the actual active kernel parameters)?

/etc/modules :

vfio
vfio_iommu_type1
vfio_pci

This is correct if you're on PVE 8.x but then why are you not on kernel verion 6.5?

echo "options vfio_iommu_type1 allow_unsafe_interrupts=1" > /etc/modprobe.d/iommu_unsafe_interrupts.conf

Why do you need unsafe interrupts?

lspci -nn | grep 'AMD' :

03:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Navi 22 [Radeon RX 6700/6700 XT/6750 XT / 6800M/6850M XT] [1002:73df] (rev c0)
03:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Navi 21/23 HDMI/DP Audio Controller [1002:ab28]

What is the output of lspci -nnks 03:00 (to make sure there are no other functions on the device)?

Blacklisting drivers :
echo "options vfio-pci ids=1002:67df,1002:aaf0" > /etc/modprobe.d/vfio.conf

This might not be necessary but should not do harm. I would add a softdep amdgpu pre: vfio-pci and softdep snd_hda_intel pre: vfio-pci to make sure vfio-pci is loaded first.

# AMD drivers
echo "blacklist radeon" >> /etc/modprobe.d/blacklist.conf
echo "blacklist amdgpu" >> /etc/modprobe.d/blacklist.conf
# NVIDIA drivers
echo "blacklist nouveau" >> /etc/modprobe.d/blacklist.conf
echo "blacklist nvidia" >> /etc/modprobe.d/blacklist.conf
echo "blacklist nvidiafb" >> /etc/modprobe.d/blacklist.conf
echo "blacklist nvidia_drm" >> /etc/modprobe.d/blacklist.conf
# Intel drivers
echo "snd_hda_intel" >> /etc/modprobe.d/blacklist.conf
echo "snd_hda_codec_hdmi" >> /etc/modprobe.d/blacklist.conf
echo "i915" >> /etc/modprobe.d/blacklist.conf

Most of this blacklisting is complete nonsense as your 6750XT only uses amdgpu. Note that is also uses snd_hda_intel and maybe some other drivers. Best not to blacklist but use a softdep to make sure vfio-pci loads before the actual drivers.

And then, I rebooted. After that, I set up a Windows 11 VM installation and installed the driver, but I am encountering error code 43. Can someone help me resolve this issue ?

Please share the VM configuration file. Maybe you enabled all kind of NVidia work-arounds that onyl hinder AMD GPUY passthrough.
Maybe try the latest Ubuntu Live installer ISO and see if that gives output on the GPU (without installing it).

 

[ZoThyx]

Thanks for your reply.

Stop using kernel version 6.2 as it does not get updates/fixes. What version of PVE are you using? I suggest moving to kernel version 6.5.

I use kernel 6.2 because I don't update it (and don't know why I didn't do it).

What is the output of cat /proc/cmdline (to check the actual active kernel parameters)?

BOOT_IMAGE=/boot/vmlinuz-6.5.11-7-pve root=/dev/mapper/pve-root ro quiet amd_iommu=on initcall_blacklist=sysfb_init iommu=pt

Why do you need unsafe interrupts?

I need to use unsafe interrupts because otherwise my system doesn't support interrupt remapping.

What is the output of lspci -nnks 03:00 (to make sure there are no other functions on the device)?

03:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Navi 22 [Radeon RX 6700/6700 XT/6750 XT / 6800M/6850M XT] [1002:73df] (rev c0)
        Subsystem: Advanced Micro Devices, Inc. [AMD/ATI] Navi 22 [Radeon RX 6700/6700 XT/6750 XT / 6800M/6850M XT] [1002:0636]
        Kernel driver in use: vfio-pci
        Kernel modules: amdgpu
03:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Navi 21/23 HDMI/DP Audio Controller [1002:ab28]
        Subsystem: Advanced Micro Devices, Inc. [AMD/ATI] Navi 21/23 HDMI/DP Audio Controller [1002:ab28]
        Kernel driver in use: vfio-pci
        Kernel modules: snd_hda_intel

/etc/modprobe.d/vfio.conf :

softdep amdgpu pre: vfio-pci
options vfio-pci ids=1002:73df,1002:ab28
echo "softdep radeon pre: vfio-pci" >> /etc/modprobe.d/vfio.conf
echo "softdep amdgpu pre: vfio-pci" >> /etc/modprobe.d/vfio.conf
echo "softdep snd_hda_intel pre: vfio-pci" >> /etc/modprobe.d/vfio.conf
echo "softdep snd_hda_codec_hdmi pre: vfio-pci" >> /etc/modprobe.d/vfio.conf

I took out everything you said was unnecessary. I also added what was needed. But one thing I don't understand is why I have to put this when I don't have an Intel, I have a Ryzen 5 5600G...

echo "softdep snd_hda_intel pre: vfio-pci" >> /etc/modprobe.d/vfio.conf
echo "softdep snd_hda_codec_hdmi pre: vfio-pci" >> /etc/modprobe.d/vfio.conf

This is what the VM configuration looks like :

bios: ovmf
boot: order=ide0;ide2;net0
cores: 10
cpu: host
efidisk0: server:100/vm-100-disk-0.qcow2,efitype=4m,pre-enrolled-keys=1,size=528K
hostpci0: 0000:03:00,pcie=1
ide0: server:100/vm-100-disk-1.qcow2,size=150G
ide2: local:iso/Win11_23H2_French_x64v2.iso,media=cdrom,size=6648108K
machine: pc-q35-8.1
memory: 8192
meta: creation-qemu=8.1.2,ctime=1704227352
name: Gaming
net0: e1000=BC:24:11:89:41:A4,bridge=vmbr0,firewall=1
numa: 0
ostype: win11
scsihw: virtio-scsi-single
smbios1: uuid=f91f4eec-59b0-4cbd-85c4-a3e99090afec
sockets: 1
tpmstate0: server:100/vm-100-disk-2.raw,size=4M,version=v2.0
vga: none
vmgenid: 31d51710-e420-4135-aba7-7cf6f7899600

$ pveversion -v :

proxmox-ve: 8.1.0 (running kernel: 6.5.11-7-pve)
pve-manager: 8.1.3 (running version: 8.1.3/b46aac3b42da5d15)
proxmox-kernel-helper: 8.1.0
pve-kernel-6.2: 8.0.5
proxmox-kernel-6.5: 6.5.11-7
proxmox-kernel-6.5.11-7-pve-signed: 6.5.11-7
proxmox-kernel-6.2.16-20-pve: 6.2.16-20
proxmox-kernel-6.2: 6.2.16-20
pve-kernel-6.2.16-3-pve: 6.2.16-3
ceph-fuse: 17.2.6-pve1+3
corosync: 3.1.7-pve3
criu: 3.17.1-2
glusterfs-client: 10.3-5
ifupdown2: 3.2.0-1+pmx7
ksm-control-daemon: 1.4-1
libjs-extjs: 7.0.0-4
libknet1: 1.28-pve1
libproxmox-acme-perl: 1.5.0
libproxmox-backup-qemu0: 1.4.1
libproxmox-rs-perl: 0.3.3
libpve-access-control: 8.0.7
libpve-apiclient-perl: 3.3.1
libpve-common-perl: 8.1.0
libpve-guest-common-perl: 5.0.6
libpve-http-server-perl: 5.0.5
libpve-network-perl: 0.9.5
libpve-rs-perl: 0.8.7
libpve-storage-perl: 8.0.5
libspice-server1: 0.15.1-1
lvm2: 2.03.16-2
lxc-pve: 5.0.2-4
lxcfs: 5.0.3-pve4
novnc-pve: 1.4.0-3
proxmox-backup-client: 3.1.2-1
proxmox-backup-file-restore: 3.1.2-1
proxmox-kernel-helper: 8.1.0
proxmox-mail-forward: 0.2.2
proxmox-mini-journalreader: 1.4.0
proxmox-widget-toolkit: 4.1.3
pve-cluster: 8.0.5
pve-container: 5.0.8
pve-docs: 8.1.3
pve-edk2-firmware: 4.2023.08-2
pve-firewall: 5.0.3
pve-firmware: 3.9-1
pve-ha-manager: 4.0.3
pve-i18n: 3.1.5
pve-qemu-kvm: 8.1.2-6
pve-xtermjs: 5.3.0-3
qemu-server: 8.0.10
smartmontools: 7.3-pve1
spiceterm: 3.3.0
swtpm: 0.8.0+pve1
vncterm: 1.8.0
zfsutils-linux: 2.2.2-pve1

Another major problem is that I can't connect a monitor directly to the graphics card (I've run out...). I'll try to get one within the week. But even after all the modifications you told me to make, I still get the same error.

 

[leesteken]

BOOT_IMAGE=/boot/vmlinuz-6.5.11-7-pve root=/dev/mapper/pve-root ro quiet amd_iommu=on initcall_blacklist=sysfb_init iommu=pt

amd_iommu=on is still invalid and does nothing.

I need to use unsafe interrupts because otherwise my system doesn't support interrupt remapping.

That's surprising. What error did you get that you fixed with this and what motherboard are you using.

/etc/modprobe.d/vfio.conf :

softdep amdgpu pre: vfio-pci
options vfio-pci ids=1002:73df,1002:ab28
echo "softdep radeon pre: vfio-pci" >> /etc/modprobe.d/vfio.conf
echo "softdep amdgpu pre: vfio-pci" >> /etc/modprobe.d/vfio.conf
echo "softdep snd_hda_intel pre: vfio-pci" >> /etc/modprobe.d/vfio.conf
echo "softdep snd_hda_codec_hdmi pre: vfio-pci" >> /etc/modprobe.d/vfio.conf

I'm not sure what I'm looking at. Do you keep adding lines with echo "..." >> ... (which will not give the right resutls) or are there lines in the file that start with echo (which is wrong)?

I took out everything you said was unnecessary. I also added what was needed. But one thing I don't understand is why I have to put this when I don't have an Intel, I have a Ryzen 5 5600G...

echo "softdep snd_hda_intel pre: vfio-pci" >> /etc/modprobe.d/vfio.conf
echo "softdep snd_hda_codec_hdmi pre: vfio-pci" >> /etc/modprobe.d/vfio.conf

No need to to this multiple times (and see remark above). What is the output of cat /etc/modprobe.d/vfio.conf?

This is what the VM configuration looks like :

bios: ovmf
boot: order=ide0;ide2;net0
cores: 10
cpu: host
efidisk0: server:100/vm-100-disk-0.qcow2,efitype=4m,pre-enrolled-keys=1,size=528K
hostpci0: 0000:03:00,pcie=1
ide0: server:100/vm-100-disk-1.qcow2,size=150G
ide2: local:iso/Win11_23H2_French_x64v2.iso,media=cdrom,size=6648108K
machine: pc-q35-8.1
memory: 8192
meta: creation-qemu=8.1.2,ctime=1704227352
name: Gaming
net0: e1000=BC:24:11:89:41:A4,bridge=vmbr0,firewall=1
numa: 0
ostype: win11
scsihw: virtio-scsi-single
smbios1: uuid=f91f4eec-59b0-4cbd-85c4-a3e99090afec
sockets: 1
tpmstate0: server:100/vm-100-disk-2.raw,size=4M,version=v2.0
vga: none
vmgenid: 31d51710-e420-4135-aba7-7cf6f7899600

Look fine but AMD drivers might have problems with machine version 8.1. But I'm not sure if you can change that without reinstalling Windows.

Another major problem is that I can't connect a monitor directly to the graphics card (I've run out...). I'll try to get one within the week. But even after all the modifications you told me to make, I still get the same error.

I'm unsure about your vfio.conf at this time. Did you make sure to disable Resizable BAR (Smart/Clever Memory Access or SMA or CMA) because QEMU does not support that?
Please try an Ubuntu Live installer ISO with a physical display, when you can.

 

[ZoThyx]

Thank you !

amd_iommu=on is still invalid and does nothing.

I'm not sure what I'm looking at. Do you keep adding lines with echo "..." >> ... (which will not give the right resutls) or are there lines in the file that start with echo (which is wrong)?

No need to to this multiple times (and see remark above). What is the output of cat /etc/modprobe.d/vfio.conf?

Yeah I see this mistake after I post the reply, I fixed them (I was tired). So cat /etc/modprobe.d/vfio.conf give me :

options vfio-pci ids=1002:73df,1002:ab28
softdep radeon pre: vfio-pci
softdep amdgpu pre: vfio-pci
softdep snd_hda_intel pre: vfio-pci
softdep snd_hda_codec_hdmi pre: vfio-pci

Look fine but AMD drivers might have problems with machine version 8.1. But I'm not sure if you can change that without reinstalling Windows.

If needed, I can reinstall the Windows VM after trying with Ubuntu Live, but I'm not sure how to put version 8.0.

Did you make sure to disable Resizable BAR (Smart/Clever Memory Access or SMA or CMA) because QEMU does not support that?

Normally I think I've disable them, but maybe I've forgotten one of them...
I will try soon with a display, and keep you on about result.

 

[ZoThyx]

what motherboard are you using

I'm using a HP Omen motherboard. Yes I know, it's not ideal at all, I don't know the exact model, but it's as if someone had given me the pc so I'm not going to complain
I don't have any error when I enable unsafe interrups, but I didn't get these line :
[ 0.294952] AMD-Vi: Interrupt remapping enabled
And i read on many site, that I need to get these line

 

[ZoThyx]

Please try an Ubuntu Live installer ISO with a physical display, when you can.

I just tried it with Ubuntu Live on my TV and this is what I get when I do the command lspci -nnks 01:00 :




After this I try on a Windows 11 VM with this config but when I install the AMD drivers, I got error 43 in the device manager. I don't what to do more.

/etc/default/grub:

GRUB_DEFAULT=0
GRUB_TIMEOUT=5
GRUB_DISTRIBUTOR=`lsb_release -i -s 2> /dev/null || echo Debian`
GRUB_CMDLINE_LINUX_DEFAULT="quiet initcall_blacklist=sysfb_init iommu=pt"
GRUB_CMDLINE_LINUX=""

/etc/modprobe.d/vfio.conf:
options vfio-pci ids=1002:73df,1002:ab28
softdep radeon pre: vfio-pci
softdep amdgpu pre: vfio-pci
softdep snd_hda_intel pre: vfio-pci
softdep snd_hda_codec_hdmi pre: vfio-pci

/etc/modprobe.d/iommu_unsafe_interrupts.conf :

options vfio_iommu_type1 allow_unsafe_interrupts=1

/etc/modprobe.d/kvm.conf :
options kvm ignore_msrs=1

dmesg | grep -e DMAR -e IOMMU :
[ 0.358726] pci 0000:00:00.2: AMD-Vi: IOMMU performance counters supported
[ 0.359752] pci 0000:00:00.2: AMD-Vi: Found IOMMU cap 0x40
[ 0.371955] perf/amd_iommu: Detected AMD IOMMU #0 (2 banks, 4 counters/bank).
[ 5.111203] AMD-Vi: AMD IOMMUv2 loaded and initialized

I keep /etc/module unchanged than the first post.
cat /proc/cmdline :
BOOT_IMAGE=/boot/vmlinuz-6.5.11-7-pve root=/dev/mapper/pve-root ro quiet initcall_blacklist=sysfb_init iommu=pt

lspci -nnks 03:00 :

03:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Navi 22 [Radeon RX 6700/6700 XT/6750 XT / 6800M/6850M XT] [1002:73df] (rev c0)
        Subsystem: Advanced Micro Devices, Inc. [AMD/ATI] Navi 22 [Radeon RX 6700/6700 XT/6750 XT / 6800M/6850M XT] [1002:0636]
        Kernel driver in use: vfio-pci
        Kernel modules: amdgpu
03:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Navi 21/23 HDMI/DP Audio Controller [1002:ab28]
        Subsystem: Advanced Micro Devices, Inc. [AMD/ATI] Navi 21/23 HDMI/DP Audio Controller [1002:ab28]
        Kernel driver in use: vfio-pci
        Kernel modules: snd_hda_intel

/etc/pve/qemu_server/100.conf :

balloon: 0
bios: ovmf
boot: order=ide0;ide2;net0
cores: 10
cpu: host
efidisk0: server:100/vm-100-disk-0.qcow2,efitype=4m,pre-enrolled-keys=1,size=528K
ide0: server:100/vm-100-disk-1.qcow2,size=150G
ide2: local:iso/Win11_23H2_French_x64v2.iso,media=cdrom,size=6648108K
machine: pc-q35-8.0
memory: 10737
meta: creation-qemu=8.1.2,ctime=1704306401
name: 100
net0: e1000=BC:24:11:0E:B7:C6,bridge=vmbr0,firewall=1
numa: 0
ostype: win11
scsihw: virtio-scsi-single
smbios1: uuid=0f7a410d-caa7-46ad-952b-479ef01da725
sockets: 1
tpmstate0: server:100/vm-100-disk-2.raw,size=4M,version=v2.0
vga: none
vmgenid: ad82baad-a3fa-4660-8762-f5a41c9a2527

I try with 8.0 instead of 8.1 as you suggest.

When I make my VM, I start a first time to install Win11 and download the driver. After this, I shutdown VM add PCI Device and set display as none. Then I restart the VM and install the driver. And finally got the error 43 :/

I take note of all the advice. Thanks in advance

 

[leesteken]

I just tried it with Ubuntu Live on my TV and this is what I get when I do the command lspci -nnks 01:00 :

View attachment 60743

After this I try on a Windows 11 VM with this config but when I install the AMD drivers, I got error 43 in the device manager. I don't what to do more.

Looks like passthrough works fine (with Ubuntu), so it's probably a Windows driver issue. You've tried machine version 8.0 but maybe try machine version 7.2. I have a 6950XT on WIndows 10 with pc-q35-7.2 running fine with AMD drivers.

 

[ZoThyx]

Looks like passthrough works fine (with Ubuntu), so it's probably a Windows driver issue. You've tried machine version 8.0 but maybe try machine version 7.2. I have a 6950XT on WIndows 10 with pc-q35-7.2 running fine with AMD drivers.

Okay, I'll try with version 7.2, just in case what version of Windows 10 are you using?

 

[leesteken]

what version of Windows 10 are you using?

I hardly use Windows anymore. It's the most simple one you can download from Microsoft and keep trying out indefinitely but it's updated until a month ago or so.

 

[ZoThyx]

I hardly use Windows anymore. It's the most simple one you can download from Microsoft and keep trying out indefinitely but it's updated until a month ago or so.

Thank you for your help, but unfortunately, even trying different machine versions such as 7.2, 6.2, it's still no better, I also tried with Windows 10 as well as 11, but I always end up with error code 43... Do you have any other ideas? For my part, I'm going to keep on looking in the hope of solving this dreadful problem!

 

[leesteken]

Do you have any other ideas?

From searching this forum for 6750XT and 6700XT (almost the same): try disabling Above 4G Decoding in the motherboard BIOS. This is probably not Proxmox specific, so other websites might have clues for working around this Windows or AMD issue.

 

[ZoThyx]

From searching this forum for 6750XT and 6700XT (almost the same): try disabling Above 4G Decoding in the motherboard BIOS. This is probably not Proxmox specific, so other websites might have clues for working around this Windows or AMD issue.

If these are options to be enabled/disabled on the motherboard, then there's nothing I can do about it. It's an HP Omen board, so you can't do anything on it, just enable virtualization. I hope I'll find the solution somewhere, otherwise I'll change my motherboard as a last resort.