I am following Proxmox documentation
https://pve.proxmox.com/wiki/PCI_Passthrough
https://pve.proxmox.com/pve-docs/pve-admin-guide.html#qm_pci_passthrough
and trying to passthrough onboard SATA controller to an OMV VM.
IOMMU is enabled in the BIOS/UEFI, SATA controller is the only device in IOMMU group #14.
If I remove PCI passthrough from VM, everything works like a charm, with SATA controller passthrough, host becomes unresponsive and hard reset is required.
Please help me understand what changes needs to be implemented for this PCI passthrough to work.
Host info:
SATA controller info:
Host config:
checking if PCIe passthrough is possible
VM config:
kernel log after starting the VM:
https://pve.proxmox.com/wiki/PCI_Passthrough
https://pve.proxmox.com/pve-docs/pve-admin-guide.html#qm_pci_passthrough
and trying to passthrough onboard SATA controller to an OMV VM.
IOMMU is enabled in the BIOS/UEFI, SATA controller is the only device in IOMMU group #14.
If I remove PCI passthrough from VM, everything works like a charm, with SATA controller passthrough, host becomes unresponsive and hard reset is required.
Please help me understand what changes needs to be implemented for this PCI passthrough to work.
Host info:
Code:
OS: Proxmox VE 8.2.2 x86_64
Host: 11A4000HGE ThinkCentre M75q-1
Kernel: 6.8.4-2-pve
CPU: AMD Ryzen 5 PRO 3400GE w/ Radeon Vega Graphics (8) @ 3.300GHz
GPU: AMD ATI Radeon Vega Series / Radeon Vega Mobile SeriesSATA controller info:
Code:
~# lspci -v
05:00.0 SATA controller: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] (rev 61) (prog-if 01 [AHCI 1.0])
Subsystem: Lenovo FCH SATA Controller [AHCI mode]
Flags: bus master, fast devsel, latency 0, IRQ 255, IOMMU group 14
Memory at fcc00000 (32-bit, non-prefetchable) [size=2K]
Capabilities: [48] Vendor Specific Information: Len=08 <?>
Capabilities: [50] Power Management version 3
Capabilities: [64] Express Endpoint, MSI 00
Capabilities: [a0] MSI: Enable- Count=1/2 Maskable- 64bit+
Capabilities: [d0] SATA HBA v1.0
Capabilities: [100] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
Capabilities: [150] Advanced Error Reporting
Capabilities: [270] Secondary PCI Express
Capabilities: [2a0] Access Control Services
Kernel driver in use: vfio-pci
Kernel modules: ahciHost config:
Code:
~# cat /etc/default/grub
GRUB_DEFAULT=0
GRUB_TIMEOUT=5
GRUB_DISTRIBUTOR=`lsb_release -i -s 2> /dev/null || echo Debian`
GRUB_CMDLINE_LINUX_DEFAULT="quiet amd_iommu=on iommu=pt vfio-pci.ids=1022:7901"
GRUB_CMDLINE_LINUX=""
~# cat /etc/modules
vfio
vfio_iommu_type1
vfio_pci
vfio_virqfd
~# cat /etc/modprobe.d/pve-blacklist.conf
blacklist nvidiafb
blacklist ahci
~# cat /etc/modprobe.d/vfio_iommu.conf
options vfio_iommu_type1 allow_unsafe_interrupts=1
~# cat /etc/modprobe.d/vfio-pci.conf
options vfio-pci ids=1022:7901
Code:
~# update-grub && update-initramfs -u -k all && reboot now
Generating grub configuration file ...
Found linux image: /boot/vmlinuz-6.8.4-2-pve
Found initrd image: /boot/initrd.img-6.8.4-2-pve
Found memtest86+ 64bit EFI image: /boot/memtest86+x64.efi
Adding boot menu entry for UEFI Firmware Settings ...
done
update-initramfs: Generating /boot/initrd.img-6.8.4-2-pve
Running hook script 'zz-proxmox-boot'..
Re-executing '/etc/kernel/postinst.d/zz-proxmox-boot' in new private mount namespace..
No /etc/kernel/proxmox-boot-uuids found, skipping ESP sync.checking if PCIe passthrough is possible
Code:
~# dmesg | grep -e DMAR -e IOMMU -e AMD-Vi -e remapping
[ 0.066674] AMD-Vi: Unknown option - 'on'
[ 0.159073] AMD-Vi: Using global IVHD EFR:0x4f77ef22294ada, EFR2:0x0
[ 0.439331] pci 0000:00:00.2: AMD-Vi: IOMMU performance counters supported
[ 0.440703] AMD-Vi: Extended features (0x4f77ef22294ada, 0x0): PPR NX GT IA GA PC GA_vAPIC
[ 0.440716] AMD-Vi: Interrupt remapping enabled
[ 0.440872] AMD-Vi: Virtual APIC enabled
[ 0.441003] perf/amd_iommu: Detected AMD IOMMU #0 (2 banks, 4 counters/bank).VM config:
Code:
~# cat /etc/pve/qemu-server/100.conf
balloon: 0
bios: ovmf
boot: order=scsi0;net0
cores: 6
cpu: x86-64-v2-AES
efidisk0: local-lvm:vm-100-disk-0,efitype=4m,pre-enrolled-keys=1,size=4M
hostpci0: 0000:05:00.0
machine: q35,viommu=intel
memory: 24576
meta: creation-qemu=8.1.5,ctime=1745780999
name: OMVvm
net0: virtio=BC:24:11:74:A4:F5,bridge=vmbr0,firewall=1
numa: 0
ostype: l26
scsi0: local-lvm:vm-100-disk-1,iothread=1,size=64G,ssd=1
scsihw: virtio-scsi-single
smbios1: uuid=c8365e26-85ba-431b-9a16-2f398e5f1ea4
sockets: 1
tags: openmediavault
tpmstate0: local-lvm:vm-100-disk-2,size=4M,version=v2.0
vmgenid: e8aeb4ed-def7-41e9-8ec9-b965f8647795kernel log after starting the VM:
Code:
~# echo "starting VM with PCI PT" > /dev/kmsg && dmesg -wH
[May 2 19:06] starting VM with PCI PT
[May 2 19:07] tap100i0: entered promiscuous mode
[ +0.083950] vmbr0: port 2(fwpr100p0) entered blocking state
[ +0.000008] vmbr0: port 2(fwpr100p0) entered disabled state
[ +0.000028] fwpr100p0: entered allmulticast mode
[ +0.000064] fwpr100p0: entered promiscuous mode
[ +0.000054] vmbr0: port 2(fwpr100p0) entered blocking state
[ +0.000003] vmbr0: port 2(fwpr100p0) entered forwarding state
[ +0.015343] fwbr100i0: port 1(fwln100i0) entered blocking state
[ +0.000006] fwbr100i0: port 1(fwln100i0) entered disabled state
[ +0.000019] fwln100i0: entered allmulticast mode
[ +0.000061] fwln100i0: entered promiscuous mode
[ +0.000053] fwbr100i0: port 1(fwln100i0) entered blocking state
[ +0.000002] fwbr100i0: port 1(fwln100i0) entered forwarding state
[ +0.014036] fwbr100i0: port 2(tap100i0) entered blocking state
[ +0.000006] fwbr100i0: port 2(tap100i0) entered disabled state
[ +0.000019] tap100i0: entered allmulticast mode
[ +0.000116] fwbr100i0: port 2(tap100i0) entered blocking state
[ +0.000003] fwbr100i0: port 2(tap100i0) entered forwarding state
