HPE ML350 Gen10 - integrated SAS passthrue issue

[nektieri]

First of all hello to everyone!

I have a problem that I'm stuck on. I will be grateful for any suggestions.
I'm trying too passthru integrates sata (AHCI) and integrated sas controller to VM (ultimately for the needs of truenas).
For testing purpose I passthru to VM with windows 11

AHCI passthrue works fine.
SAS passthrue reboots whole server when i turn on VM.

What else can I test next ?

Problematic server:
HPE ML350 Gen10 - dual intel silver 4108
Proxmox is installed on two nvme disc in mirror configuration ZFS
Test VM uses q35 chipset.

Attaching few info that may be helpful:
pveversion pve-manager/8.0.3/bbf3993334bfa916 (running kernel: 6.2.16-3-pve)
verbose - file attached

/etc/modules
vfio vfio_iommu_type1 vfio_pci vfio_virqfd

/etc/kernel/cmdline
root=ZFS=rpool/ROOT/pve-1 boot=zfs quiet intel_iommu=on iommu=pt


/etc/modprobe.d/vfio.conf
options vfio-pci ids=9005:028f,8086:a182

/etc/modprobe.d/pve-blacklist.conf
blacklist nvidiafb blacklist smartpqi blacklist ahci

updated:
update-initramfs -u -k all

AHCI
for d in /sys/kernel/iommu_groups/*/devices/*; do n=${d#*/iommu_groups/*}; n=${n%%/*}; printf 'IOMMU group %s ' "$n"; lspci -nns "${d##*/}"; done | grep 'group 49' IOMMU group 49 00:17.0 SATA controller [0106]: Intel Corporation C620 Series Chipset Family SATA Controller [AHCI mode] [8086:a182] (rev 09) lspci -s 00:17.0 -v 00:17.0 SATA controller: Intel Corporation C620 Series Chipset Family SATA Controller [AHCI mode] (rev 09) (prog-if 01 [AHCI 1.0]) DeviceName: Embedded SATA Controller #1 Subsystem: Hewlett Packard Enterprise C620 Series Chipset Family SATA Controller [AHCI mode] Flags: bus master, 66MHz, medium devsel, latency 0, IRQ 255, NUMA node 0, IOMMU group 49 Memory at d9d94000 (32-bit, non-prefetchable) [size=8K] Memory at d9d96000 (32-bit, non-prefetchable) [size=256] I/O ports at 2040 [size=8] I/O ports at 2048 [size=4] I/O ports at 2020 [size=32] Memory at d9d00000 (32-bit, non-prefetchable) [size=512K] Capabilities: [80] MSI: Enable- Count=1/1 Maskable- 64bit- Capabilities: [70] Power Management version 3 Capabilities: [a8] SATA HBA v1.0 Kernel driver in use: vfio-pci Kernel modules: ahci

SAS
for d in /sys/kernel/iommu_groups/*/devices/*; do n=${d#*/iommu_groups/*}; n=${n%%/*}; printf 'IOMMU group %s ' "$n"; lspci -nns "${d##*/}"; done | grep 'group 16 ' IOMMU group 16 3b:00.0 Serial Attached SCSI controller [0107]: Adaptec Smart Storage PQI SAS [9005:028f] (rev 01) lspci -s 3b:00.0 -v 3b:00.0 Serial Attached SCSI controller: Adaptec Smart Storage PQI SAS (rev 01) DeviceName: Embedded RAID 1 Subsystem: Hewlett-Packard Company Smart Array P408i-a SR Gen10 Flags: bus master, fast devsel, latency 0, IRQ 255, NUMA node 0, IOMMU group 16 Memory at e4400000 (64-bit, non-prefetchable) [size=32K] I/O ports at 6000 [size=256] Capabilities: [80] Power Management version 3 Capabilities: [b0] MSI-X: Enable- Count=64 Masked- Capabilities: [c0] Express Endpoint, MSI 00 Capabilities: [100] Advanced Error Reporting Capabilities: [300] Secondary PCI Express Kernel driver in use: vfio-pci Kernel modules: smartpqi

Rest attached as a files - (post symbol limit)

 

[nektieri]

I gave up and added aditional HBA controller