Proxmox VE SATA Passthrough not working

[TinyTuna]

Hey, so I've been trying to use the onboard SATA passthrough on my AOOSTAR WTR PRO Ryzen 7 5825u CPU to no avail. Either I just get a connection error when I start my TrueNAS VM with the PCIE device added and I have to fully restart the server OR most of the time I can actually get into TrueNAS but the HDD's aren't there, and they disappear in Proxmox disks as well. I tried shutting down the TrueNAS VM and restarting but I get this error.

error writing '1' to '/sys/bus/pci/devices/0000:05:00.0/reset': Inappropriate ioctl for device
failed to reset PCI device '0000:05:00.0', but trying to continue as not all devices need a reset
error writing '1' to '/sys/bus/pci/devices/0000:05:00.1/reset': Inappropriate ioctl for device
failed to reset PCI device '0000:05:00.1', but trying to continue as not all devices need a reset
kvm: -device vfio-pci,host=0000:05:00.0,id=hostpci1.0,bus=ich9-pcie-port-2,addr=0x0.0,multifunction=on: vfio 0000:05:00.0: Failed to set up TRIGGER eventfd signaling for interrupt INTX-0: VFIO_DEVICE_SET_IRQS failure: No such device
TASK ERROR: start failed: QEMU exited with code 1

Here are my IOMMU groups as well:

IOMMU group 0 00:01.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Renoir PCIe Dummy Host Bridge [1022:1632]
IOMMU group 10 01:00.0 Non-Volatile memory controller [0108]: Kingston Technology Company, Inc. NV3 NVMe SSD SM2268XT2 (DRAM-less) [2646:5028] (rev 03)
IOMMU group 11 02:00.0 Ethernet controller [0200]: Intel Corporation Ethernet Controller I226-V [8086:125c] (rev 04)
IOMMU group 12 03:00.0 Ethernet controller [0200]: Intel Corporation Ethernet Controller I226-V [8086:125c] (rev 04)
IOMMU group 13 04:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Barcelo [1002:15e7] (rev c1)
IOMMU group 14 04:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Renoir Radeon High Definition Audio Controller [1002:1637]
IOMMU group 15 04:00.2 Encryption controller [1080]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 10h-1fh) Platform Security Processor [1022:15df]
IOMMU group 16 04:00.3 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] Renoir/Cezanne USB 3.1 [1022:1639]
IOMMU group 17 04:00.4 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] Renoir/Cezanne USB 3.1 [1022:1639]
IOMMU group 18 04:00.5 Multimedia controller [0480]: Advanced Micro Devices, Inc. [AMD] ACP/ACP3X/ACP6x Audio Coprocessor [1022:15e2] (rev 01)
IOMMU group 19 04:00.6 Audio device [0403]: Advanced Micro Devices, Inc. [AMD] Family 17h/19h HD Audio Controller [1022:15e3]
IOMMU group 1 00:01.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Renoir PCIe GPP Bridge [1022:1633]
IOMMU group 20 04:00.7 Signal processing controller [1180]: Advanced Micro Devices, Inc. [AMD] Sensor Fusion Hub [1022:15e4]
IOMMU group 21 05:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Zeppelin/Raven/Raven2 PCIe Dummy Function [1022:145a] (rev 81)
IOMMU group 22 05:00.1 SATA controller [0106]: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] [1022:7904] (rev 81)
IOMMU group 2 00:01.3 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Renoir/Cezanne PCIe GPP Bridge [1022:1634]
IOMMU group 3 00:02.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Renoir PCIe Dummy Host Bridge [1022:1632]
IOMMU group 4 00:02.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Renoir/Cezanne PCIe GPP Bridge [1022:1634]
IOMMU group 5 00:08.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Renoir PCIe Dummy Host Bridge [1022:1632]
IOMMU group 6 00:08.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Renoir Internal PCIe GPP Bridge to Bus [1022:1635]
IOMMU group 7 00:08.2 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Renoir Internal PCIe GPP Bridge to Bus [1022:1635]
IOMMU group 8 00:14.0 SMBus [0c05]: Advanced Micro Devices, Inc. [AMD] FCH SMBus Controller [1022:790b] (rev 51)
IOMMU group 8 00:14.3 ISA bridge [0601]: Advanced Micro Devices, Inc. [AMD] FCH LPC Bridge [1022:790e] (rev 51)
IOMMU group 9 00:18.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Cezanne Data Fabric; Function 0 [1022:166a]
IOMMU group 9 00:18.1 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Cezanne Data Fabric; Function 1 [1022:166b]
IOMMU group 9 00:18.2 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Cezanne Data Fabric; Function 2 [1022:166c]
IOMMU group 9 00:18.3 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Cezanne Data Fabric; Function 3 [1022:166d]
IOMMU group 9 00:18.4 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Cezanne Data Fabric; Function 4 [1022:166e]
IOMMU group 9 00:18.5 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Cezanne Data Fabric; Function 5 [1022:166f]
IOMMU group 9 00:18.6 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Cezanne Data Fabric; Function 6 [1022:1670]
IOMMU group 9 00:18.7 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Cezanne Data Fabric; Function 7 [1022:1671]

I've tried this in Proxmox VE 9 and 8.4 and neither work

 

[leesteken]

If you want to passthrough 05:00.1 then don't also passthrough 05:00.0. Maybe you enabled All Functions? That passes through every function of device 05:00, which is 05:00.0 and 05:00.1. Don't do that and maybe the passthrouhg of just 05:00.1 will work.

Your groups look way to good (many groups with few devices) for your platform. Are you using pcie_acs_override which breaks PCI(e) device isolation and to break up the IOMMU groups?
Passthrough always is trial and error and hit or miss and pcie_acs_override cannot fix problems only let's you ignore it as your own peril. Sometimes it just does not work. Sometimes you'll find work-arounds from people with the exact same hardware (which I do not have).

Please show the VM configuration (qm config VM_ID_NUMBER and cat /proc/cmdline) after doing the first change, if you still have issues.

 

[TinyTuna]

If you want to passthrough 05:00.1 then don't also passthrough 05:00.0. Maybe you enabled All Functions? That passes through every function of device 05:00, which is 05:00.0 and 05:00.1. Don't do that and maybe the passthrouhg of just 05:00.1 will work.

Your groups look way to good (many groups with few devices) for your platform. Are you using pcie_acs_override which breaks PCI(e) device isolation and to break up the IOMMU groups?
Passthrough always is trial and error and hit or miss and pcie_acs_override cannot fix problems only let's you ignore it as your own peril. Sometimes it just does not work. Sometimes you'll find work-arounds from people with the exact same hardware (which I do not have).

Please show the VM configuration (qm config VM_ID_NUMBER and cat /proc/cmdline) after doing the first change, if you still have issues.

Oh my god, thank you SO MUCH. I had to disable All Functions, didn't know that it would use the entire group. It's because the AOOSTAR WTR PRO has problems with using both SATA controllers at the same time (one is used for CPU monitoring) and it just stops working, but I thought that they divided it in the new BIOS update and had the 2nd SATA controller with the temp reads at:
"IOMMU group 20 04:00.7 Signal processing controller [1180]: Advanced Micro Devices, Inc. [AMD] Sensor Fusion Hub [1022:15e4]"
But I guess not.

Once again thanks for the quick reply that actually fixed the issue!

 

[esvee]

Oh my god, thank you SO MUCH. I had to disable All Functions, didn't know that it would use the entire group. It's because the AOOSTAR WTR PRO has problems with using both SATA controllers at the same time (one is used for CPU monitoring) and it just stops working, but I thought that they divided it in the new BIOS update and had the 2nd SATA controller with the temp reads at:
"IOMMU group 20 04:00.7 Signal processing controller [1180]: Advanced Micro Devices, Inc. [AMD] Sensor Fusion Hub [1022:15e4]"
But I guess not.

Once again thanks for the quick reply that actually fixed the issue!

Hey there,
I have received the same system but im wary to unbox it in case I have an issue here.

Can you please confirm you managed to get all 4 sata drives to work in truenas AND cpu is still able to boost above 2ghz? Because I see many people a year ago having this issue and none found a solution.

 

[TinyTuna]

Hey there,
I have received the same system but im wary to unbox it in case I have an issue here.

Can you please confirm you managed to get all 4 sata drives to work in truenas AND cpu is still able to boost above 2ghz? Because I see many people a year ago having this issue and none found a solution.

Hey! Unfortunately I can't test all 4 drives, I've only got 2, but it's turboing and working with them. While I'm not ENTIRELY certain but I believe that there are only two onboard SATA passthroughs - 1 for all of the drives, and the second is for the CPU sensors and etc.

 

[iEatData]

Oh my god, thank you SO MUCH. I had to disable All Functions, didn't know that it would use the entire group. It's because the AOOSTAR WTR PRO has problems with using both SATA controllers at the same time (one is used for CPU monitoring) and it just stops working, but I thought that they divided it in the new BIOS update and had the 2nd SATA controller with the temp reads at:
"IOMMU group 20 04:00.7 Signal processing controller [1180]: Advanced Micro Devices, Inc. [AMD] Sensor Fusion Hub [1022:15e4]"
But I guess not.

Once again thanks for the quick reply that actually fixed the issue!

Hi, can you please tell me where you found that Bios update, how you installed it and what version works for you!? Thank you!

 

[iEatData]

If you want to passthrough 05:00.1 then don't also passthrough 05:00.0. Maybe you enabled All Functions? That passes through every function of device 05:00, which is 05:00.0 and 05:00.1. Don't do that and maybe the passthrouhg of just 05:00.1 will work.

Your groups look way to good (many groups with few devices) for your platform. Are you using pcie_acs_override which breaks PCI(e) device isolation and to break up the IOMMU groups?
Passthrough always is trial and error and hit or miss and pcie_acs_override cannot fix problems only let's you ignore it as your own peril. Sometimes it just does not work. Sometimes you'll find work-arounds from people with the exact same hardware (which I do not have).

Please show the VM configuration (qm config VM_ID_NUMBER and cat /proc/cmdline) after doing the first change, if you still have issues.

I wish that this fix would work for me too. As far, when i do it, only half my disks are shown inside the VM. But thank you for the general path i hope it will somehow workout for me too.

EDIT: OK, When i pass 06:00.0 and 06:00.1 it works again. Without all function. But i am sure this is not te solution because now theFan controller does not work again. (Only sound-check)

 

[Bypper]

I have the same hardware:
AOOSTAR WTR Ryzen.
And when I installed proxmox, I created 2 VM, the first with Fedora and I did passthru of Graphical card, and I can show this screen in my monitor, and another VM with Openmediavault and passthru of one Ethernet card and SATA disks.

All was OK, and bought an 1 Tb. NVME additional module, and crash!. Passthru options not working then.
I have extract this NVME module, but the problem is not solved.

I've tried with the options in /etc/default/grub that I've seen in forums:
amd_iommu=on, pci_acs_override=downstream,multifunction (I don't know if I have to probe with any more)

I charge vfio modules too...

The passthru process with Web is normal and all looks fine, but when I boot the VM I always the same result.
It looks as if the PCI buses turned crazy after NVME module installation, and it would need and reorder.
I hope you know I want to tell.

Do you have any idea?

Best Regards,

 

[uzumo]

The only required option is iommu=pt.

Recent kernels do not require amd_iommu=on.

Furthermore, if iommu is functioning correctly, pci_acs_override=downstream,multifunction should not be specified either.

You must provide specific details along with the log.

 

[Bypper]

Hi,
Thank you for your answer,

I can extract some logs from journal with related information:
...
oct 29 12:22:45 aoostar kernel: vfio-pci 0000:04:00.0: Unable to change power state from D3cold to D0, device inaccessible
oct 29 12:22:45 aoostar kernel: vfio-pci 0000:04:00.1: Unable to change power state from D3cold to D0, device inaccessible
oct 29 12:22:45 aoostar kernel: vfio-pci 0000:04:00.2: Unable to change power state from D3cold to D0, device inaccessible
oct 29 12:22:45 aoostar kernel: vfio-pci 0000:04:00.3: Unable to change power state from D3cold to D0, device inaccessible
oct 29 12:22:45 aoostar kernel: vfio-pci 0000:04:00.4: Unable to change power state from D3cold to D0, device inaccessible
oct 29 12:22:45 aoostar kernel: vfio-pci 0000:04:00.5: Unable to change power state from D3cold to D0, device inaccessible
oct 29 12:22:45 aoostar pvedaemon[6360]: start failed: QEMU exited with code 1
oct 29 12:22:45 aoostar pvedaemon[1367]: <root@pam> end task UPID:aoostar:000018D8:00029837:6901F8D4:qmstart:100:root@pam: start failed: QEMU exited with code 1
oct 29 12:23:15 aoostar kernel:
oct 29 12:23:15 aoostar kernel: No Local Variables are initialized for Method [A011]
oct 29 12:23:15 aoostar kernel:
oct 29 12:23:15 aoostar kernel: Initialized Arguments for Method [A011]: (2 arguments defined for method invocation)
oct 29 12:23:15 aoostar kernel: Arg0: 00000000b2744c60 <Obj> Integer 0000000000000059
oct 29 12:23:15 aoostar kernel: Arg1: 000000004b923f31 <Obj> Buffer(24) C8 00 00 00 00 00 00 00
oct 29 12:23:15 aoostar kernel:
oct 29 12:23:15 aoostar kernel: ACPI Error: Aborting method \_SB.A011 due to previous error (AE_AML_LOOP_TIMEOUT) (20240827/psparse-529)
oct 29 12:23:15 aoostar kernel: ACPI Error: Aborting method \_SB.ALIB due to previous error (AE_AML_LOOP_TIMEOUT) (20240827/psparse-529)
oct 29 12:23:15 aoostar kernel: ACPI Error: Aborting method \M276 due to previous error (AE_AML_LOOP_TIMEOUT) (20240827/psparse-529)
oct 29 12:23:15 aoostar kernel: ACPI Error: Aborting method \_SB.PCI0.GP17.AZAL._PS0 due to previous error (AE_AML_LOOP_TIMEOUT) (20240827/psparse-529)
oct 29 12:23:15 aoostar kernel: vfio-pci 0000:04:00.6: Unable to change power state from D3cold to D0, device inaccessible
oct 29 12:23:15 aoostar kernel: vfio-pci 0000:04:00.7: Unable to change power state from D3cold to D0, device inaccessible
oct 29 12:23:15 aoostar kernel: vfio-pci 0000:04:00.0: resetting
oct 29 12:23:15 aoostar kernel: vfio-pci 0000:04:00.0: Unable to change power state from D3cold to D0, device inaccessible
oct 29 12:23:15 aoostar kernel: vfio-pci 0000:04:00.1: resetting
oct 29 12:23:15 aoostar kernel: vfio-pci 0000:04:00.1: Unable to change power state from D3cold to D0, device inaccessible
oct 29 12:23:15 aoostar kernel: vfio-pci 0000:04:00.2: resetting
oct 29 12:23:15 aoostar kernel: vfio-pci 0000:04:00.2: Unable to change power state from D3cold to D0, device inaccessible
oct 29 12:23:15 aoostar kernel: vfio-pci 0000:04:00.3: resetting
oct 29 12:23:15 aoostar kernel: vfio-pci 0000:04:00.3: Unable to change power state from D3cold to D0, device inaccessible
oct 29 12:23:15 aoostar kernel: vfio-pci 0000:04:00.4: resetting
oct 29 12:23:15 aoostar kernel: vfio-pci 0000:04:00.4: Unable to change power state from D3cold to D0, device inaccessible
oct 29 12:23:15 aoostar kernel: vfio-pci 0000:04:00.5: resetting
oct 29 12:23:15 aoostar kernel: vfio-pci 0000:04:00.5: Unable to change power state from D3cold to D0, device inaccessible
oct 29 12:23:15 aoostar kernel: vfio-pci 0000:04:00.6: resetting
...
I'm surprised with the "device inaccessible" message, because is correct:

aoostar:~# lspci | grep -i vga
04:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Barcelo (rev c1)

And with the SATA Controller passthru which I'm doing in another VM, the same problem...

aoostar:~# lspci | grep -i sata
05:00.0 SATA controller: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] (rev 81)
05:00.1 SATA controller: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] (rev 81)

journal log:
...
oct 29 15:10:01 aoostar kernel: vfio-pci 0000:05:00.0: reset done
oct 29 15:10:01 aoostar kernel: vfio-pci 0000:05:00.1: reset done
oct 29 15:10:01 aoostar kernel: vfio-pci 0000:05:00.1: Unable to change power state from D3cold to D0, device inaccessible
oct 29 15:10:01 aoostar kernel: vfio-pci 0000:05:00.0: Unable to change power state from D3cold to D0, device inaccessible
...

Regards,

 

[uzumo]

Are you attempting to pass through 04:00.0 to 04:00.6?
If you're only passing through the GPU, it should be just 04:00.0. Why are you passing through others?

Please attach the results of the following command as a file. Pasting them into the body makes it hard to read.

dmesg | grep -e DMAR -e IOMMU -e AMD-Vi
for d in $(find /sys/kernel/iommu_groups/ -type l | sort -n -k5 -t/); do
    n=${d#*/iommu_groups/*}; n=${n%%/*}
    printf 'IOMMU Group %s ' "$n"
    lspci -nns "${d##*/}"
done;
qm showcmd VMID --pretty true

 

[Bypper]

Hi,
Thank you for your answer...

I'm only attempting to pass 04:00.0, but is marked "All Functions" checkbox.
Attach output of your code below:
[ 0.079896] AMD-Vi: Unknown option - 'on'
[ 0.207203] AMD-Vi: ivrs, add hid:AMDI0020, uid:\_SB.FUR0, rdevid:0xa0
[ 0.207206] AMD-Vi: ivrs, add hid:AMDI0020, uid:\_SB.FUR1, rdevid:0xa0
[ 0.207208] AMD-Vi: ivrs, add hid:AMDI0020, uid:\_SB.FUR2, rdevid:0xa0
[ 0.207209] AMD-Vi: ivrs, add hid:AMDI0020, uid:\_SB.FUR3, rdevid:0xa0
[ 0.207211] AMD-Vi: Using global IVHD EFR:0x206d73ef22254ade, EFR2:0x0
[ 0.468112] pci 0000:00:00.2: AMD-Vi: IOMMU performance counters supported
[ 0.471189] AMD-Vi: Extended features (0x206d73ef22254ade, 0x0): PPR X2APIC NX GT IA GA PC GA_vAPIC
[ 0.471204] AMD-Vi: Interrupt remapping enabled
[ 0.471207] AMD-Vi: X2APIC enabled
[ 0.719069] AMD-Vi: Virtual APIC enabled
[ 0.719438] perf/amd_iommu: Detected AMD IOMMU #0 (2 banks, 4 counters/bank).
IOMMU Group 0 00:00.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Renoir/Cezanne Root Complex [1022:1630]
IOMMU Group 1 00:01.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Renoir PCIe Dummy Host Bridge [1022:1632]
IOMMU Group 2 00:01.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Renoir PCIe GPP Bridge [1022:1633]
IOMMU Group 3 00:01.3 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Renoir/Cezanne PCIe GPP Bridge [1022:1634]
IOMMU Group 4 00:02.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Renoir PCIe Dummy Host Bridge [1022:1632]
IOMMU Group 5 00:02.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Renoir/Cezanne PCIe GPP Bridge [1022:1634]
IOMMU Group 6 00:08.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Renoir PCIe Dummy Host Bridge [1022:1632]
IOMMU Group 7 00:08.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Renoir Internal PCIe GPP Bridge to Bus [1022:1635]
IOMMU Group 8 00:08.2 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Renoir Internal PCIe GPP Bridge to Bus [1022:1635]
IOMMU Group 9 00:14.0 SMBus [0c05]: Advanced Micro Devices, Inc. [AMD] FCH SMBus Controller [1022:790b] (rev 51)
IOMMU Group 9 00:14.3 ISA bridge [0601]: Advanced Micro Devices, Inc. [AMD] FCH LPC Bridge [1022:790e] (rev 51)
IOMMU Group 10 00:18.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Cezanne Data Fabric; Function 0 [1022:166a]
IOMMU Group 10 00:18.1 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Cezanne Data Fabric; Function 1 [1022:166b]
IOMMU Group 10 00:18.2 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Cezanne Data Fabric; Function 2 [1022:166c]
IOMMU Group 10 00:18.3 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Cezanne Data Fabric; Function 3 [1022:166d]
IOMMU Group 10 00:18.4 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Cezanne Data Fabric; Function 4 [1022:166e]
IOMMU Group 10 00:18.5 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Cezanne Data Fabric; Function 5 [1022:166f]
IOMMU Group 10 00:18.6 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Cezanne Data Fabric; Function 6 [1022:1670]
IOMMU Group 10 00:18.7 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Cezanne Data Fabric; Function 7 [1022:1671]
IOMMU Group 11 01:00.0 Non-Volatile memory controller [0108]: MAXIO Technology (Hangzhou) Ltd. NVMe SSD Controller MAP1202 (DRAM-less) [1e4b:1202] (rev 01)
IOMMU Group 12 02:00.0 Ethernet controller [0200]: Intel Corporation Ethernet Controller I226-V [8086:125c] (rev 04)
IOMMU Group 13 03:00.0 Ethernet controller [0200]: Intel Corporation Ethernet Controller I226-V [8086:125c] (rev 04)
IOMMU Group 14 04:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Barcelo [1002:15e7] (rev c1)
IOMMU Group 15 04:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Renoir Radeon High Definition Audio Controller [1002:1637]
IOMMU Group 16 04:00.2 Encryption controller [1080]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 10h-1fh) Platform Security Processor [1022:15df]
IOMMU Group 17 04:00.3 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] Renoir/Cezanne USB 3.1 [1022:1639]
IOMMU Group 18 04:00.4 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] Renoir/Cezanne USB 3.1 [1022:1639]
IOMMU Group 19 04:00.5 Multimedia controller [0480]: Advanced Micro Devices, Inc. [AMD] Audio Coprocessor [1022:15e2] (rev 01)
IOMMU Group 20 04:00.6 Audio device [0403]: Advanced Micro Devices, Inc. [AMD] Family 17h/19h/1ah HD Audio Controller [1022:15e3]
IOMMU Group 21 04:00.7 Signal processing controller [1180]: Advanced Micro Devices, Inc. [AMD] Sensor Fusion Hub [1022:15e4]
IOMMU Group 22 05:00.0 SATA controller [0106]: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] [1022:7904] (rev 81)
IOMMU Group 23 05:00.1 SATA controller [0106]: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] [1022:7904] (rev 81)
/usr/bin/kvm \
-id 100 \
-name 'vFedora,debug-threads=on' \
-no-shutdown \
-chardev 'socket,id=qmp,path=/var/run/qemu-server/100.qmp,server=on,wait=off' \
-mon 'chardev=qmp,mode=control' \
-chardev 'socket,id=qmp-event,path=/var/run/qmeventd.sock,reconnect-ms=5000' \
-mon 'chardev=qmp-event,mode=control' \
-pidfile /var/run/qemu-server/100.pid \
-daemonize \
-smbios 'type=1,uuid=9e377cff-5cf7-419b-849d-44b44a69c63a' \
-smp '2,sockets=1,cores=2,maxcpus=2' \
-nodefaults \
-boot 'menu=on,strict=on,reboot-timeout=1000,splash=/usr/share/qemu-server/bootsplash.jpg' \
-vnc 'unix:/var/run/qemu-server/100.vnc,password=on' \
-cpu 'qemu64,+aes,enforce,kvm=off,+kvm_pv_eoi,+kvm_pv_unhalt,+pni,+popcnt,+sse4.1,+sse4.2,+ssse3' \
-m 8192 \
-object 'iothread,id=iothread-virtioscsi0' \
-object '{"id":"throttle-drive-scsi0","limits":{},"qom-type":"throttle-group"}' \
-global 'ICH9-LPC.disable_s3=1' \
-global 'ICH9-LPC.disable_s4=1' \
-readconfig /usr/share/qemu-server/pve-q35-4.0.cfg \
-device 'vmgenid,guid=ab605c1c-9f4e-4ba8-bbf4-f20ed294b8d8' \
-device 'usb-tablet,id=tablet,bus=ehci.0,port=1' \
-device 'vfio-pci,host=0000:04:00.0,id=hostpci0.0,bus=ich9-pcie-port-1,addr=0x0.0,rombar=0,x-vga=on,multifunction=on' \
-device 'vfio-pci,host=0000:04:00.1,id=hostpci0.1,bus=ich9-pcie-port-1,addr=0x0.1' \
-device 'vfio-pci,host=0000:04:00.2,id=hostpci0.2,bus=ich9-pcie-port-1,addr=0x0.2' \
-device 'vfio-pci,host=0000:04:00.3,id=hostpci0.3,bus=ich9-pcie-port-1,addr=0x0.3' \
-device 'vfio-pci,host=0000:04:00.4,id=hostpci0.4,bus=ich9-pcie-port-1,addr=0x0.4' \
-device 'vfio-pci,host=0000:04:00.5,id=hostpci0.5,bus=ich9-pcie-port-1,addr=0x0.5' \
-device 'vfio-pci,host=0000:04:00.6,id=hostpci0.6,bus=ich9-pcie-port-1,addr=0x0.6' \
-device 'vfio-pci,host=0000:04:00.7,id=hostpci0.7,bus=ich9-pcie-port-1,addr=0x0.7' \
-device 'virtio-vga,id=vga,bus=pcie.0,addr=0x1' \
-device 'virtio-serial,id=spice,bus=pci.0,addr=0x9' \
-chardev 'spicevmc,id=vdagent,name=vdagent' \
-device 'virtserialport,chardev=vdagent,name=com.redhat.spice.0' \
-spice 'tls-port=61000,addr=127.0.0.1,tls-ciphers=HIGH,seamless-migration=on' \
-device 'virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x3,free-page-reporting=on' \
-iscsi 'initiator-name=iqn.1993-08.org.debian:01:435d70edd8db' \
-device 'ide-cd,bus=ide.1,unit=0,id=ide2,bootindex=101' \
-device 'virtio-scsi-pci,id=virtioscsi0,bus=pci.3,addr=0x1,iothread=iothread-virtioscsi0' \
-blockdev '{"detect-zeroes":"on","discard":"ignore","driver":"throttle","file":{"cache":{"direct":true,"no-flush":false},"detect-zeroes":"on","discard":"ignore","driver":"raw","file":{"aio":"io_uring","cache":{"direct":true,"no-flush":false},"detect-zeroes":"on","discard":"ignore","driver":"host_device","filename":"/dev/pve/vm-100-disk-0","node-name":"e0bf565c58599febe8de731ee07d8b5","read-only":false},"node-name":"f0bf565c58599febe8de731ee07d8b5","read-only":false},"node-name":"drive-scsi0","read-only":false,"throttle-group":"throttle-drive-scsi0"}' \
-device 'scsi-hd,bus=virtioscsi0.0,channel=0,scsi-id=0,lun=0,drive=drive-scsi0,id=scsi0,device_id=drive-scsi0,bootindex=100,write-cache=on' \
-netdev 'type=tap,id=net0,ifname=tap100i0,script=/usr/libexec/qemu-server/pve-bridge,downscript=/usr/libexec/qemu-server/pve-bridgedown,vhost=on' \
-device 'virtio-net-pci,mac=BC:24:11:3E:EA:B6,netdev=net0,bus=pci.0,addr=0x12,id=net0,rx_queue_size=1024,tx_queue_size=256,bootindex=102,host_mtu=1500' \
-device virtio-iommu-pci \
-machine 'type=q35+pve1'

 

[Bypper]

Hi,
I've deactivated the "amd_iommu=on" option because your older answer and I've saw this in log:
...
oct 29 17:30:40 aoostar kernel: ------------[ cut here ]------------
oct 29 17:30:40 aoostar kernel: WARNING: CPU: 15 PID: 1525 at drivers/pci/pcie/bwctrl.c:168 pcie_set_target_speed+0x1ad/0x280
oct 29 17:30:40 aoostar kernel: Modules linked in: veth ebtable_filter ebtables ip_set ip6table_raw iptable_raw ip6table_filter ip6_tables iptable_filter nf_tables bonding tls softdog sunrpc binfmt_misc nfnet>
oct 29 17:30:40 aoostar kernel: drm_suballoc_helper snd_pcm aesni_intel snd_pci_acp5x drm_display_helper crypto_simd snd_timer snd_rn_pci_acp3x cec cryptd snd_acp_config snd rc_core cm32181 snd_soc_acpi wmi_>
oct 29 17:30:40 aoostar kernel: CPU: 15 UID: 0 PID: 1525 Comm: kvm Tainted: P O 6.14.11-4-pve #1
oct 29 17:30:40 aoostar kernel: Tainted: [P]=PROPRIETARY_MODULE, [O]=OOT_MODULE
oct 29 17:30:40 aoostar kernel: Hardware name: TianBei WTR PRO/WTR PRO, BIOS 0.33 02/06/2025
oct 29 17:30:40 aoostar kernel: RIP: 0010:pcie_set_target_speed+0x1ad/0x280
oct 29 17:30:40 aoostar kernel: Code: 0f b6 75 d4 4c 89 e7 e8 b1 d8 fe ff 85 c0 78 74 49 8b 7c 24 18 48 85 ff 74 05 e8 ce 08 fe ff 41 bc 01 00 00 00 e9 75 ff ff ff <0f> 0b bb ea ff ff ff eb b0 89 f0 b9 3f 00 >
oct 29 17:30:40 aoostar kernel: RSP: 0018:ffffd40001023880 EFLAGS: 00010202
oct 29 17:30:40 aoostar kernel: RAX: 00000000000000eb RBX: ffff8e98425bd000 RCX: 0000000000000000
oct 29 17:30:40 aoostar kernel: RDX: 0000000000000001 RSI: 00000000000000ff RDI: ffff8e98425bd000
oct 29 17:30:40 aoostar kernel: RBP: ffffd400010238b0 R08: 0000000000000000 R09: 0000000000000000
oct 29 17:30:40 aoostar kernel: R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000043
oct 29 17:30:40 aoostar kernel: R13: ffffd400010238c4 R14: ffff8e9841f8c800 R15: ffff8e98425bd0c8
oct 29 17:30:40 aoostar kernel: FS: 0000724042d29840(0000) GS:ffff8ea70e980000(0000) knlGS:0000000000000000
oct 29 17:30:40 aoostar kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
oct 29 17:30:40 aoostar kernel: CR2: 000064895edca248 CR3: 0000000108fef000 CR4: 0000000000f50ef0
oct 29 17:30:40 aoostar kernel: PKRU: 55555554
oct 29 17:30:40 aoostar kernel: Call Trace:
oct 29 17:30:40 aoostar kernel: <TASK>
oct 29 17:30:40 aoostar kernel: pcie_failed_link_retrain.cold+0x65/0xf5
oct 29 17:30:40 aoostar kernel: pci_bridge_wait_for_secondary_bus+0x18d/0x2d0
oct 29 17:30:40 aoostar kernel: ? srso_alias_return_thunk+0x5/0xfbef5
oct 29 17:30:40 aoostar kernel: ? pci_reset_secondary_bus+0x72/0xa0
oct 29 17:30:40 aoostar kernel: pci_bridge_secondary_bus_reset+0x47/0x80
oct 29 17:30:40 aoostar kernel: __pci_reset_bus+0x77/0xe0
oct 29 17:30:40 aoostar kernel: pci_reset_bus+0x60/0x2e0
oct 29 17:30:40 aoostar kernel: vfio_pci_dev_set_hot_reset+0x1ce/0x200 [vfio_pci_core]
oct 29 17:30:40 aoostar kernel: vfio_pci_core_ioctl+0xc56/0xfa0 [vfio_pci_core]
oct 29 17:30:40 aoostar kernel: ? srso_alias_return_thunk+0x5/0xfbef5
oct 29 17:30:40 aoostar kernel: ? __mmu_notifier_invalidate_range_end+0x84/0xb0
oct 29 17:30:40 aoostar kernel: ? __pfx_vfio_pci_count_devs+0x10/0x10 [vfio_pci_core]
oct 29 17:30:40 aoostar kernel: vfio_device_fops_unl_ioctl+0xa6/0x800 [vfio]
oct 29 17:30:40 aoostar kernel: ? srso_alias_return_thunk+0x5/0xfbef5
oct 29 17:30:40 aoostar kernel: ? pci_conf1_write+0xad/0xf0
oct 29 17:30:40 aoostar kernel: ? srso_alias_return_thunk+0x5/0xfbef5
oct 29 17:30:40 aoostar kernel: ? pci_write+0x4a/0x90
oct 29 17:30:40 aoostar kernel: ? srso_alias_return_thunk+0x5/0xfbef5
oct 29 17:30:40 aoostar kernel: ? pci_user_write_config_word+0x67/0xb0
oct 29 17:30:40 aoostar kernel: __x64_sys_ioctl+0xa7/0xe0
oct 29 17:30:40 aoostar kernel: x64_sys_call+0x1053/0x2310
oct 29 17:30:40 aoostar kernel: do_syscall_64+0x7e/0x170
oct 29 17:30:40 aoostar kernel: ? srso_alias_return_thunk+0x5/0xfbef5
oct 29 17:30:40 aoostar kernel: ? pci_read+0x49/0x90
oct 29 17:30:40 aoostar kernel: ? srso_alias_return_thunk+0x5/0xfbef5
oct 29 17:30:40 aoostar kernel: ? pci_user_read_config_word+0x85/0x100
oct 29 17:30:40 aoostar kernel: ? srso_alias_return_thunk+0x5/0xfbef5
oct 29 17:30:40 aoostar kernel: ? vfio_user_config_read+0x85/0xa0 [vfio_pci_core]
oct 29 17:30:40 aoostar kernel: ? srso_alias_return_thunk+0x5/0xfbef5
oct 29 17:30:40 aoostar kernel: ? vfio_default_config_read+0xe6/0x110 [vfio_pci_core]
oct 29 17:30:40 aoostar kernel: ? srso_alias_return_thunk+0x5/0xfbef5
oct 29 17:30:40 aoostar kernel: ? __check_object_size+0x7f/0x260
oct 29 17:30:40 aoostar kernel: ? srso_alias_return_thunk+0x5/0xfbef5
oct 29 17:30:40 aoostar kernel: ? vfio_pci_config_rw+0x2c1/0x49b [vfio_pci_core]
oct 29 17:30:40 aoostar kernel: ? srso_alias_return_thunk+0x5/0xfbef5
oct 29 17:30:40 aoostar kernel: ? __pm_runtime_idle+0x75/0xd0
oct 29 17:30:40 aoostar kernel: ? srso_alias_return_thunk+0x5/0xfbef5
oct 29 17:30:40 aoostar kernel: ? vfio_pci_rw+0xaa/0x1b0 [vfio_pci_core]
oct 29 17:30:40 aoostar kernel: ? srso_alias_return_thunk+0x5/0xfbef5
oct 29 17:30:40 aoostar kernel: ? vfio_pci_core_read+0x28/0x40 [vfio_pci_core]
oct 29 17:30:40 aoostar kernel: ? srso_alias_return_thunk+0x5/0xfbef5
oct 29 17:30:40 aoostar kernel: ? vfio_device_fops_read+0x2a/0x50 [vfio]
oct 29 17:30:40 aoostar kernel: ? srso_alias_return_thunk+0x5/0xfbef5
oct 29 17:30:40 aoostar kernel: ? vfs_read+0x100/0x390
oct 29 17:30:40 aoostar kernel: ? srso_alias_return_thunk+0x5/0xfbef5
oct 29 17:30:40 aoostar kernel: ? __x64_sys_pwrite64+0x6a/0xd0
oct 29 17:30:40 aoostar kernel: ? srso_alias_return_thunk+0x5/0xfbef5
oct 29 17:30:40 aoostar kernel: ? arch_exit_to_user_mode_prepare.isra.0+0x22/0x120
oct 29 17:30:40 aoostar kernel: ? srso_alias_return_thunk+0x5/0xfbef5
oct 29 17:30:40 aoostar kernel: ? syscall_exit_to_user_mode+0x38/0x1d0
oct 29 17:30:40 aoostar kernel: ? srso_alias_return_thunk+0x5/0xfbef5
oct 29 17:30:40 aoostar kernel: ? __x64_sys_pread64+0x6a/0xd0
oct 29 17:30:40 aoostar kernel: ? srso_alias_return_thunk+0x5/0xfbef5
oct 29 17:30:40 aoostar kernel: ? arch_exit_to_user_mode_prepare.isra.0+0x22/0x120
oct 29 17:30:40 aoostar kernel: ? srso_alias_return_thunk+0x5/0xfbef5
oct 29 17:30:40 aoostar kernel: ? syscall_exit_to_user_mode+0x38/0x1d0
oct 29 17:30:40 aoostar kernel: ? srso_alias_return_thunk+0x5/0xfbef5
oct 29 17:30:40 aoostar kernel: ? do_syscall_64+0x8a/0x170
oct 29 17:30:40 aoostar kernel: ? srso_alias_return_thunk+0x5/0xfbef5
oct 29 17:30:40 aoostar kernel: ? do_syscall_64+0x8a/0x170
oct 29 17:30:40 aoostar kernel: ? srso_alias_return_thunk+0x5/0xfbef5
oct 29 17:30:40 aoostar kernel: ? do_syscall_64+0x8a/0x170
oct 29 17:30:40 aoostar kernel: ? srso_alias_return_thunk+0x5/0xfbef5
oct 29 17:30:40 aoostar kernel: ? irqentry_exit+0x43/0x50
oct 29 17:30:40 aoostar kernel: ? srso_alias_return_thunk+0x5/0xfbef5
oct 29 17:30:40 aoostar kernel: ? exc_page_fault+0x96/0x1e0
oct 29 17:30:40 aoostar kernel: entry_SYSCALL_64_after_hwframe+0x76/0x7e
oct 29 17:30:40 aoostar kernel: RIP: 0033:0x724045f1e8db
oct 29 17:30:40 aoostar kernel: Code: 00 48 89 44 24 18 31 c0 48 8d 44 24 60 c7 04 24 10 00 00 00 48 89 44 24 08 48 8d 44 24 20 48 89 44 24 10 b8 10 00 00 00 0f 05 <89> c2 3d 00 f0 ff ff 77 1c 48 8b 44 24 18 >
oct 29 17:30:40 aoostar kernel: RSP: 002b:00007ffcd7ecf1a0 EFLAGS: 00000246 ORIG_RAX: 0000000000000010
oct 29 17:30:40 aoostar kernel: RAX: ffffffffffffffda RBX: 000055e94eadabac RCX: 0000724045f1e8db
oct 29 17:30:40 aoostar kernel: RDX: 000055e94d2a85b0 RSI: 0000000000003b71 RDI: 000000000000004e
oct 29 17:30:40 aoostar kernel: RBP: 000000000000002c R08: 000055e94eadaba0 R09: 000055e94d2a85bc
oct 29 17:30:40 aoostar kernel: R10: 0000000000000000 R11: 0000000000000246 R12: 000055e94f385900
oct 29 17:30:40 aoostar kernel: R13: 000055e94d2a85b0 R14: 000055e927bdab8f R15: 0000000000000008
oct 29 17:30:40 aoostar kernel: </TASK>
oct 29 17:30:40 aoostar kernel: ---[ end trace 0000000000000000 ]---

Regards,