Sata PCI passthrough issues ....

B

Bobbbb

Well-Known Member
Jul 13, 2018
78
1
48
29
Hi,

I am trying to passthrough this device:
07:00.0 SATA controller: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] (rev ff)

I tried all the options though the GUI,
when switching the VM on i don't see the hard drive anymore but the hard drive doesn't appear on the VM.

Please assist.
 
If I remember correctly from previous threads about passthrough of that (or very similar) SATA controller: it does not reset properly using flr. I probably also supports reset via the bus, and that's the work-around when using kernel 5.15 or higher.
What is the output of cat '/sys/bus/pci/devices/0000:07:00.0/reset_method'?
Try doing echo bus >'/sys/bus/pci/devices/0000:07:00.0/reset_method' before starting the VM. If that works, look into a hookscript or other ways to do that every reboot of the Proxmox host.
 
looks like bus is there:
cat '/sys/bus/pci/devices/0000:07:00.0/reset_method'
pm bus
 
ummm I came across this in the vm log file:
AMD-Vi: AMD IOMMUv2 functionality not available on this system - This is not a bug.

I am trying to pass the SATA controller to an truenas guest, but i am passing 3 lan interfaces.
 
Bobbbb said:
looks like bus is there:
cat '/sys/bus/pci/devices/0000:07:00.0/reset_method'
pm bus
Click to expand...
Did doing echo bus >'/sys/bus/pci/devices/0000:07:00.0/reset_method' before starting the VM (for the first time after a reboot of the Proxmox host) work?
Bobbbb said:
ummm I came across this in the vm log file:
AMD-Vi: AMD IOMMUv2 functionality not available on this system - This is not a bug.

I am trying to pass the SATA controller to an truenas guest, but i am passing 3 lan interfaces.
Click to expand...
I don't know what you mean by this or what you are asking. If you want me to debug your VM configuration, please attach the configuration file (from /etc/pve/qemu-server/) and show the IOMMU groups with this command: for d in /sys/kernel/iommu_groups/*/devices/*; do n=${d#*/iommu_groups/*}; n=${n%%/*}; printf 'IOMMU group %s ' "$n"; lspci -nns "${d##*/}"; done
 
Can i maybe change the subject a bit?
sorry.
I managed to successfully passthrough the drive...are there any benefits in passing through the whole controller?
I am not going to use any RAID, only replicate important data between 2 drives.
I assume passing the controller will preform replications a little better?
 
Bobbbb said:
Can i maybe change the subject a bit?
Click to expand...
Only you can do that (and mark it solved if you feel like it).
Bobbbb said:
I managed to successfully passthrough the drive...are there any benefits in passing through the whole controller?
Click to expand...
There are mostly drawbacks to PCIe passthrough: it sometimes breaks on updates, you cannot use ballooning (because all memory must be pinned into actual RAM), the VM cannot be migrated, starts slower and the host cannot use the other ports of the SATA controller. The only benefit is SMART inside the VM, but you don't really need that and you can do that on the host.
Bobbbb said:
I am not going to use any RAID, only replicate important data between 2 drives.
I assume passing the controller will preform replications a little better?
Click to expand...
If you're doing disk passthrough, I don't expect much overhead.
 
Last edited:
Thanks....just to be clear...the negatives that you mentioned are for SATA controller passthrough?
(not disk passthrough)
 
Hi all....bumping this.
my truenas pool broke so trying to be a good boy and do a proper passthrough as was recommended by the truenas guys....

If I do:
echo bus >'/sys/bus/pci/devices/0000:07:00.0/reset_method'
the VM doesn't start and the hard drive dtop showing under disks.
 
Code: 
root@pve02:~# for d in /sys/kernel/iommu_groups/*/devices/*; do n=${d#*/iommu_groups/*}; n=${n%%/*}; printf 'IOMMU group %s ' "$n"; lspci -nns "${d##*/}"; done
IOMMU group 0 00:01.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-1fh) PCIe Dummy Host Bridge [1022:1452]
IOMMU group 10 00:18.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Raven/Raven2 Device 24: Function 0 [1022:15e8]
IOMMU group 10 00:18.1 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Raven/Raven2 Device 24: Function 1 [1022:15e9]
IOMMU group 10 00:18.2 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Raven/Raven2 Device 24: Function 2 [1022:15ea]
IOMMU group 10 00:18.3 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Raven/Raven2 Device 24: Function 3 [1022:15eb]
IOMMU group 10 00:18.4 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Raven/Raven2 Device 24: Function 4 [1022:15ec]
IOMMU group 10 00:18.5 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Raven/Raven2 Device 24: Function 5 [1022:15ed]
IOMMU group 10 00:18.6 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Raven/Raven2 Device 24: Function 6 [1022:15ee]
IOMMU group 10 00:18.7 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Raven/Raven2 Device 24: Function 7 [1022:15ef]
IOMMU group 11 01:00.0 Non-Volatile memory controller [0108]: Kingston Technology Company, Inc. OM3PDP3 NVMe SSD [2646:500d] (rev 01)
IOMMU group 12 02:00.0 Ethernet controller [0200]: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller [10ec:8168] (rev 0c)
IOMMU group 13 03:00.0 Ethernet controller [0200]: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller [10ec:8168] (rev 0c)
IOMMU group 14 04:00.0 Ethernet controller [0200]: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller [10ec:8168] (rev 0c)
IOMMU group 15 05:00.0 Ethernet controller [0200]: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller [10ec:8168] (rev 0c)
IOMMU group 16 06:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Picasso [1002:15d8] (rev de)
IOMMU group 17 06:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Raven/Raven2/Fenghuang HDMI/DP Audio Controller [1002:15de]
IOMMU group 17 06:00.2 Encryption controller [1080]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 10h-1fh) Platform Security Processor [1022:15df]
IOMMU group 17 06:00.3 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] Raven2 USB 3.1 [1022:15e5]
IOMMU group 17 06:00.5 Multimedia controller [0480]: Advanced Micro Devices, Inc. [AMD] Raven/Raven2/FireFlight/Renoir Audio Processor [1022:15e2]
IOMMU group 17 06:00.6 Audio device [0403]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 10h-1fh) HD Audio Controller [1022:15e3]
IOMMU group 17 06:00.7 Non-VGA unclassified device [0000]: Advanced Micro Devices, Inc. [AMD] Raven/Raven2/Renoir Non-Sensor Fusion Hub KMDF driver [1022:15e6]
IOMMU group 18 07:00.0 SATA controller [0106]: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] [1022:7901] (rev ff)
IOMMU group 1 00:01.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Raven/Raven2 PCIe GPP Bridge [6:0] [1022:15d3]
IOMMU group 2 00:01.2 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Raven/Raven2 PCIe GPP Bridge [6:0] [1022:15d3]
IOMMU group 3 00:01.3 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Raven/Raven2 PCIe GPP Bridge [6:0] [1022:15d3]
IOMMU group 4 00:01.4 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Raven/Raven2 PCIe GPP Bridge [6:0] [1022:15d3]
IOMMU group 5 00:01.5 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Raven/Raven2 PCIe GPP Bridge [6:0] [1022:15d3]
IOMMU group 6 00:08.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-1fh) PCIe Dummy Host Bridge [1022:1452]
IOMMU group 7 00:08.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Raven/Raven2 Internal PCIe GPP Bridge 0 to Bus A [1022:15db]
IOMMU group 8 00:08.2 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Raven/Raven2 Internal PCIe GPP Bridge 0 to Bus B [1022:15dc]
IOMMU group 9 00:14.0 SMBus [0c05]: Advanced Micro Devices, Inc. [AMD] FCH SMBus Controller [1022:790b] (rev 61)
IOMMU group 9 00:14.3 ISA bridge [0601]: Advanced Micro Devices, Inc. [AMD] FCH LPC Bridge [1022:790e] (rev 51
 
Bobbbb said:
IOMMU group 18 07:00.0 SATA controller [0106]: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] [1022:7901] (rev ff)
Click to expand...
Bobbbb said:
Any idea if I can pass my controller with this set up?
Click to expand...
It appears to be in a IOMMU group of its own, which is good and PCIe passthrough could work. It then depends on whether the device reset properly and will work (more than once) inside the VM.
 
Hi, I been messing around with the reset property.
and now, not only i don't see the hard drives being passed through, i dont see the hard drives at all.
(all vm's are off).
what is the default reset method here?
I need to at least see the drives :-(

By the way,
drives are being seen by the bios.
 
Bobbbb said:
Hi, I been messing around with the reset property.
and now, not only i don't see the hard drives being passed through, i dont see the hard drives at all.
(all vm's are off).
Click to expand...
It probably does not reset properly and is therefore not usable inside the VM. It happens a lot with on-board SATA controllers.
Bobbbb said:
what is the default reset method here?
Click to expand...
What is the output of cat /sys/bus/pci/devices/0000:07:00.0/reset_method after a fresh. clean reboot? The first one is the default.
Bobbbb said:
I need to at least see the drives :-(

By the way,
drives are being seen by the bios.
Click to expand...
The physical motherboard BIOS or the VM "virtual" BIOS?
 
Motherboard bios.
ok i created a hookup script and now its showing
cat '/sys/bus/pci/devices/0000:07:00.0/reset_method'
bus

Still not working.

I am working on the server remotely, let me try do a proper shutdown and see if i can get it switched on
 
Bobbbb said:
ok i created a hookup script and now its showing
cat '/sys/bus/pci/devices/0000:07:00.0/reset_method'
bus
Click to expand...
And you are sure nothing did a write to that? Then the SATA controller does not advertise flr (Function Level Reset) and the bus (PCI(e) disconnect, reconnect) is the only option.
Bobbbb said:
Still not working.
Click to expand...
Of course, because we did not change anything.
Bobbbb said:
I am working on the server remotely, let me try do a proper shutdown and see if i can get it switched on
Click to expand...
That's what you need to see the available reset methods. A fresh reboot and no writes to reset_method before reading it.

EDIT: It's unlikely you'll get this SATA controller to work with passthrough. Maybe get a different one that is known to work with passthrough?
 
Last edited:
okay, I am officially giving up on controller passthrough.
I am actually getting anew NAS soon also.

But i still can't see my drives listed under fdisk -l/or under "disks" in the GUI.

what is the "default" reset method/how do i fix this.....
 
"And you are sure nothing did a write to that? Then the SATA controller does not advertise flr (Function Level Reset) and the bus (PCI(e) disconnect, reconnect) is the only option."

not sure i understand this comment, please explain.

Also maybe we didn't change anything, but I
added a hookup script for echo bus for that interface plus did a complete shutdown and restart of the server :-)
 
Bobbbb said:
what is the "default" reset method/how do i fix this.....
Click to expand...
It depends on the device. You need to reboot, not touch reset_method and read it.
Bobbbb said:
"And you are sure nothing did a write to that? Then the SATA controller does not advertise flr (Function Level Reset) and the bus (PCI(e) disconnect, reconnect) is the only option."

not sure i understand this comment, please explain.

Also maybe we didn't change anything, but I
added a hookup script for echo bus for that interface plus did a complete shutdown and restart of the server :)
Click to expand...
I meant to convey that the reset_method will tell you what your options are. But when you write to it, you can't see the defaults anymore.
 
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom