[SOLVED] Pass-through failure with ZFS RAID1 and other complexities.

[Nirgal]

Hi,
Sorry for my poor English. I installed PVE8.0 on a ZFS RAID1 with 2 NVME SSDs. My motherboard is X11SSM-F, which does not support booting from NVME SSD, so I added a SATA SSD and cloned the BIOS boot partition and EFI partition into it. Now the system can start up correctly, but when I went to configure pass-through, something went wrong.

I edited /etc/default/grub, chenged GRUB_CMDLINE_LINUX_DEFAULT="quiet" to GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on", then I executed update-grub. It warned me:
W: This system is booted via proxmox-boot-tool: W: Executing 'update-grub' directly does not update the correct configs! W: Running: 'proxmox-boot-tool refresh'
So I executed proxmox-boot-tool refresh as well.
Then I reboot the system and try to start a VM which has a passed-through SATA controller, but it failed with
TASK ERROR: cannot prepare PCI pass-through, IOMMU not present. Then I re-cloned the BIOS boot partition and EFI partition into that SATA SSD (using dd) and reboot, but it came with the same error as before when I try to start the VM.

Now I have no idea what to do.

 

[leesteken]

What is the output of cat /proc/cmdline? Maybe your change is not active or VT-d is not fully enabled in the motherboard BIOS.

 

[Nirgal]

What is the output of cat /proc/cmdline? Maybe your change is not active or VT-d is not fully enabled in the motherboard BIOS.

Here's the output:
initrd=\EFI\proxmox\6.2.16-10-pve\initrd.img-6.2.16-10-pve root=ZFS=rpool/ROOT/pve-1 boot=zfs
I'm sure VT-d is enabled. How do I know if my change is active?

 

[leesteken]

Here's the output:
initrd=\EFI\proxmox\6.2.16-10-pve\initrd.img-6.2.16-10-pve root=ZFS=rpool/ROOT/pve-1 boot=zfs
I'm sure VT-d is enabled. How do I know if my change is active?

intel_iommu=on is missing and therefore IOMMU is not activated in Linux/Proxmox. My guess is that your Proxmox uses systemd-boot instead of GRUB. Please see the manual on how to check. So very many threads on this forum about this... Make sure to edit the right configuration file in the right way. Please see the manual on editting the kernel command line. There is also a whole section on PCI(e) passthrough (but ignore the mistake about kernels later than 5.14, you still need intel_iommu=on). Or check out one of the many threads about enabling IOMMU and systemd-boot versus GRUB.

 

[Nirgal]

intel_iommu=on is missing and therefore IOMMU is not activated in Linux/Proxmox. My guess is that your Proxmox uses systemd-boot instead of GRUB. Please see the manual on how to check. So very many threads on this forum about this... Make sure to edit the right configuration file in the right way. Please see the manual on editting the kernel command line. There is also a whole section on PCI(e) passthrough (but ignore the mistake about kernels later than 5.14, you still need intel_iommu=on). Or check out one of the many threads about enabling IOMMU and systemd-boot versus GRUB.

Thank you. I will try to solve it.