1. Added intel_iommu=on to /etc/default/grub
2. update-grub
3. Added vfio vfio_iommu_type1 vfio_pci vfio_virqfd to /etc/modules
4. update-initramfs -u -k all
When I restarted Proxmox there was an error after the ram disk message:
Mar 20 22:40:19 xlogic kernel: DMAR: DRHD: handling fault status reg 2
Mar 20 22:40:19 xlogic kernel: DMAR: [DMA Read] Request device [0a:00.0] PASID ffffffff fault addr 9d000 [fault reason 06] PTE Read access is not set
.....
Mar 20 22:40:26 xlogic kernel: ata7: link is slow to respond, please be patient (ready=0)
Mar 20 22:40:26 xlogic kernel: ata9.00: qc timeout (cmd 0xec)
Mar 20 22:40:26 xlogic kernel: ata9.00: failed to IDENTIFY (I/O error, err_mask=0x4)
Mar 20 22:40:26 xlogic kernel: ata9: limiting SATA link speed to 3.0 Gbps
Mar 20 22:40:26 xlogic kernel: ata14.00: qc timeout (cmd 0xa1)
Mar 20 22:40:26 xlogic kernel: ata14.00: failed to IDENTIFY (I/O error, err_mask=0x4)
Mar 20 22:40:26 xlogic kernel: ata8.00: qc timeout (cmd 0xec)
Mar 20 22:40:26 xlogic kernel: ata8.00: failed to IDENTIFY (I/O error, err_mask=0x4)
Mar 20 22:40:26 xlogic kernel: ata8: limiting SATA link speed to 3.0 Gbps
Mar 20 22:40:26 xlogic kernel: ata10.00: qc timeout (cmd 0xec)
Mar 20 22:40:26 xlogic kernel: ata10.00: failed to IDENTIFY (I/O error, err_mask=0x4)
Mar 20 22:40:26 xlogic kernel: ata10: limiting SATA link speed to 3.0 Gbps
Mar 20 22:40:26 xlogic kernel: ata9: SATA link up 6.0 Gbps (SStatus 133 SControl 320)
Mar 20 22:40:26 xlogic kernel: ata14: SATA link up 1.5 Gbps (SStatus 113 SControl 300)
Mar 20 22:40:26 xlogic kernel: ata8: SATA link up 6.0 Gbps (SStatus 133 SControl 320)
Mar 20 22:40:26 xlogic kernel: ata10: SATA link up 6.0 Gbps (SStatus 133 SControl 320)
Mar 20 22:40:30 xlogic kernel: ata7: COMRESET failed (errno=-16)
Mar 20 22:40:36 xlogic kernel: ata7: link is slow to respond, please be patient (ready=0)
Mar 20 22:40:57 xlogic kernel: ata9.00: qc timeout (cmd 0xec)
Mar 20 22:40:57 xlogic kernel: ata9.00: failed to IDENTIFY (I/O error, err_mask=0x4)
Mar 20 22:40:57 xlogic kernel: ata14.00: qc timeout (cmd 0xa1)
Mar 20 22:40:57 xlogic kernel: ata14.00: failed to IDENTIFY (I/O error, err_mask=0x4)
Mar 20 22:40:57 xlogic kernel: ata10.00: qc timeout (cmd 0xec)
Mar 20 22:40:57 xlogic kernel: ata10.00: failed to IDENTIFY (I/O error, err_mask=0x4)
Mar 20 22:40:57 xlogic kernel: ata8.00: qc timeout (cmd 0xec)
Mar 20 22:40:57 xlogic kernel: ata8.00: failed to IDENTIFY (I/O error, err_mask=0x4)
Mar 20 22:40:57 xlogic kernel: ata9: SATA link up 6.0 Gbps (SStatus 133 SControl 320)
Mar 20 22:40:57 xlogic kernel: ata14: SATA link up 1.5 Gbps (SStatus 113 SControl 300)
Mar 20 22:40:57 xlogic kernel: ata10: SATA link up 6.0 Gbps (SStatus 133 SControl 320)
Mar 20 22:40:57 xlogic kernel: ata8: SATA link up 6.0 Gbps (SStatus 133 SControl 320)
Mar 20 22:41:05 xlogic kernel: ata7: COMRESET failed (errno=-16)
Mar 20 22:41:05 xlogic kernel: ata7: limiting SATA link speed to 3.0 Gbps
Mar 20 22:41:10 xlogic kernel: ata7: COMRESET failed (errno=-16)
Mar 20 22:41:10 xlogic kernel: ata7: reset failed, giving up
Mar 20 22:41:10 xlogic systemd[1]: Started Helper to synchronize boot up for ifupdown.
Mar 20 22:41:11 xlogic systemd[1]: Started udev Wait for Complete Device Initialization.
Mar 20 22:41:11 xlogic systemd[1]: Condition check resulted in Import ZFS pools by cache file being skipped.
Mar 20 22:41:11 xlogic systemd[1]: Reached target ZFS pool import target.
Mar 20 22:41:11 xlogic systemd[1]: Starting Mount ZFS filesystems...
Mar 20 22:41:11 xlogic systemd[1]: Starting Wait for ZFS Volume (zvol) links in /dev...
Mar 20 22:41:11 xlogic systemd[1]: Started Mount ZFS filesystems.
Mar 20 22:41:11 xlogic zvol_wait[1096]: No zvols found, nothing to do.
The results after this was that none of my hard disks (all CMR drives) were loaded and TrueNAS could not start due to the missing disks.
After removing the intel_iommu entry in the grub file and updating. A restart of the system was fully successful.
2. update-grub
3. Added vfio vfio_iommu_type1 vfio_pci vfio_virqfd to /etc/modules
4. update-initramfs -u -k all
When I restarted Proxmox there was an error after the ram disk message:
Mar 20 22:40:19 xlogic kernel: DMAR: DRHD: handling fault status reg 2
Mar 20 22:40:19 xlogic kernel: DMAR: [DMA Read] Request device [0a:00.0] PASID ffffffff fault addr 9d000 [fault reason 06] PTE Read access is not set
.....
Mar 20 22:40:26 xlogic kernel: ata7: link is slow to respond, please be patient (ready=0)
Mar 20 22:40:26 xlogic kernel: ata9.00: qc timeout (cmd 0xec)
Mar 20 22:40:26 xlogic kernel: ata9.00: failed to IDENTIFY (I/O error, err_mask=0x4)
Mar 20 22:40:26 xlogic kernel: ata9: limiting SATA link speed to 3.0 Gbps
Mar 20 22:40:26 xlogic kernel: ata14.00: qc timeout (cmd 0xa1)
Mar 20 22:40:26 xlogic kernel: ata14.00: failed to IDENTIFY (I/O error, err_mask=0x4)
Mar 20 22:40:26 xlogic kernel: ata8.00: qc timeout (cmd 0xec)
Mar 20 22:40:26 xlogic kernel: ata8.00: failed to IDENTIFY (I/O error, err_mask=0x4)
Mar 20 22:40:26 xlogic kernel: ata8: limiting SATA link speed to 3.0 Gbps
Mar 20 22:40:26 xlogic kernel: ata10.00: qc timeout (cmd 0xec)
Mar 20 22:40:26 xlogic kernel: ata10.00: failed to IDENTIFY (I/O error, err_mask=0x4)
Mar 20 22:40:26 xlogic kernel: ata10: limiting SATA link speed to 3.0 Gbps
Mar 20 22:40:26 xlogic kernel: ata9: SATA link up 6.0 Gbps (SStatus 133 SControl 320)
Mar 20 22:40:26 xlogic kernel: ata14: SATA link up 1.5 Gbps (SStatus 113 SControl 300)
Mar 20 22:40:26 xlogic kernel: ata8: SATA link up 6.0 Gbps (SStatus 133 SControl 320)
Mar 20 22:40:26 xlogic kernel: ata10: SATA link up 6.0 Gbps (SStatus 133 SControl 320)
Mar 20 22:40:30 xlogic kernel: ata7: COMRESET failed (errno=-16)
Mar 20 22:40:36 xlogic kernel: ata7: link is slow to respond, please be patient (ready=0)
Mar 20 22:40:57 xlogic kernel: ata9.00: qc timeout (cmd 0xec)
Mar 20 22:40:57 xlogic kernel: ata9.00: failed to IDENTIFY (I/O error, err_mask=0x4)
Mar 20 22:40:57 xlogic kernel: ata14.00: qc timeout (cmd 0xa1)
Mar 20 22:40:57 xlogic kernel: ata14.00: failed to IDENTIFY (I/O error, err_mask=0x4)
Mar 20 22:40:57 xlogic kernel: ata10.00: qc timeout (cmd 0xec)
Mar 20 22:40:57 xlogic kernel: ata10.00: failed to IDENTIFY (I/O error, err_mask=0x4)
Mar 20 22:40:57 xlogic kernel: ata8.00: qc timeout (cmd 0xec)
Mar 20 22:40:57 xlogic kernel: ata8.00: failed to IDENTIFY (I/O error, err_mask=0x4)
Mar 20 22:40:57 xlogic kernel: ata9: SATA link up 6.0 Gbps (SStatus 133 SControl 320)
Mar 20 22:40:57 xlogic kernel: ata14: SATA link up 1.5 Gbps (SStatus 113 SControl 300)
Mar 20 22:40:57 xlogic kernel: ata10: SATA link up 6.0 Gbps (SStatus 133 SControl 320)
Mar 20 22:40:57 xlogic kernel: ata8: SATA link up 6.0 Gbps (SStatus 133 SControl 320)
Mar 20 22:41:05 xlogic kernel: ata7: COMRESET failed (errno=-16)
Mar 20 22:41:05 xlogic kernel: ata7: limiting SATA link speed to 3.0 Gbps
Mar 20 22:41:10 xlogic kernel: ata7: COMRESET failed (errno=-16)
Mar 20 22:41:10 xlogic kernel: ata7: reset failed, giving up
Mar 20 22:41:10 xlogic systemd[1]: Started Helper to synchronize boot up for ifupdown.
Mar 20 22:41:11 xlogic systemd[1]: Started udev Wait for Complete Device Initialization.
Mar 20 22:41:11 xlogic systemd[1]: Condition check resulted in Import ZFS pools by cache file being skipped.
Mar 20 22:41:11 xlogic systemd[1]: Reached target ZFS pool import target.
Mar 20 22:41:11 xlogic systemd[1]: Starting Mount ZFS filesystems...
Mar 20 22:41:11 xlogic systemd[1]: Starting Wait for ZFS Volume (zvol) links in /dev...
Mar 20 22:41:11 xlogic systemd[1]: Started Mount ZFS filesystems.
Mar 20 22:41:11 xlogic zvol_wait[1096]: No zvols found, nothing to do.
The results after this was that none of my hard disks (all CMR drives) were loaded and TrueNAS could not start due to the missing disks.
After removing the intel_iommu entry in the grub file and updating. A restart of the system was fully successful.