Xeon D-1528 on-board SATA passthrough failure

Anthony VH

New Member
May 4, 2019
3
0
1
36
I'm trying to pass through the on-board SATA controller on a Supermicro X10SDV-6C-TLN4F, which has a Xeon D-1528.

Passing through the SATA controller to a VM is possible, the VM sees the attached drives, but actually mounting/formatting/repartitioning/... them results in boatloads of errors. It seems those kinds of errors are due to "incorrect" passthrough. As far as I can see, this is due to the SATA controller being in the same IOMMU group as 3 other devices which I'm not passing through (ISA bridge, SMBus controller, and some thermal management controller).

Google searches seem to indicate this passthrough is be possible, at least on ESXi, so I can't imagine it's not on Proxmox. I would link to some forum topics and blog posts, but the forum won't let me as a new user. Searching for "esxi x10sdv passthrough sata" gave me 3 matches as the first results.

Proxmox boots from a NVMe SSD in the M.2 slot, so no issues there. (The onboard SATA definitely can't be passed through if the M.2 slot contains a SATA SSD.)

Here's the settings I'm using:
  • BIOS: version 2.0a (newest one available on Supermicro's website)
    AFAIK all virtualization settings are enabled (i.e. VT-d, ACS, interrupt remapping). In fact, they were by default.
  • Proxmox configuration:
    • /etc/default/grub:
      GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on"
      Adding "pci_acs_override=downstream" didn't improve matters.
    • /etc/modules:
      vfio
      vfio_iommu_type1
      vfio_pci
      vfio_virqfd
    • /etc/modprobe.d/vfio.conf:
      options vfio-pci ids=8086:8c02
    • Problematic IOMMU group:
      Code:
      IOMMU Group 22 00:1f.0 ISA bridge [0601]: Intel Corporation C224 Series Chipset Family Server Standard SKU LPC Controller [8086:8c54] (rev 05)
      IOMMU Group 22 00:1f.2 SATA controller [0106]: Intel Corporation 8 Series/C220 Series Chipset Family 6-port SATA Controller 1 [AHCI mode] [8086:8c02] (rev 05)
      IOMMU Group 22 00:1f.3 SMBus [0c05]: Intel Corporation 8 Series/C220 Series Chipset Family SMBus Controller [8086:8c22] (rev 05)
      IOMMU Group 22 00:1f.6 Signal processing controller [1180]: Intel Corporation 8 Series Chipset Family Thermal Management Controller [8086:8c24] (rev 05)
Does anyone have a clue on how to get this to work?
 

wolfgang

Proxmox Retired Staff
Retired Staff
Oct 1, 2014
6,496
503
103

Anthony VH

New Member
May 4, 2019
3
0
1
36
Hm, that's a shame.

I'm a little confused by the various messages I find online about people pulling this exact thing off with ESXi. The fact that it works there seems to indicate to me that it's due to some software issue that this can't (currently) be done with Proxmox.
 

wolfgang

Proxmox Retired Staff
Retired Staff
Oct 1, 2014
6,496
503
103
You can't compare VMware with Proxmox VE.
 

Anthony VH

New Member
May 4, 2019
3
0
1
36
I guess this is kind of derailing the thread, but... Why can't I compare? From a user point of view both are supposed to act as a level 1 hypervisor. Both have access to exactly the same hardware. I'm assuming both use VT-x/VT-d/ACS/... to provide pass-through of physical hardware. So the fact that this SATA controller can be passed through in ESXi, but can't in Proxmox/KVM, seems to hint at a software issue in KVM to me. Clearly the underlying hardware is not at fault here, since ESXi can make it work...
 

wolfgang

Proxmox Retired Staff
Retired Staff
Oct 1, 2014
6,496
503
103
To keep it generall.
Some Hardware Vendors do not tell the opensource community everything.
Proprietary Software Vendors have contracts to get this information,
but they have to keep this information secret what is no problem because the sourcecode is not open.
 
  • Like
Reactions: James Crook

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get your own in 60 seconds.

Buy now!