[SOLVED] TASK ERROR: IOMMU not present adding two port nic to VM

[digitalbots]

So i am trying to load pfSense on a VM and do a PCI passthrough for the nic.

I am on 7.x

i followed these steps to enable Dells version of IOMMU
https://www.dell.com/support/kbdoc/...g-sr-iov-technology-on-dell-poweredge-servers

then followed
https://www.youtube.com/watch?v=GoZaMgEgrHw&t=378s

these steps.

rebooted and now all I am getting is DMAR: [DMA read] errors on the screen even though i put in quiet in the grub file?

Also i have 2 raid controllers and now it only sees one of the 2 even though the bios shows both of them up and running.

what did i do wrong? did i miss a setting?

(attached is the output of dmesg | grep -e DMAR -e IOMMU)

 

[Echoesofdreams]

I am having the same issue. I read among the lines of my Dell 3880 Desktop that trusted execution needs to be activated to activate VT-d

 

[digitalbots]

I am having the same issue. I read among the lines of my Dell 3880 Desktop that trusted execution needs to be activated to activate VT-d

Im not out of the woods yet BUT i fixed most of my issues
(https://pve.proxmox.com/wiki/Pci_passthrough#Enable_the_IOMMU)

as per that doc i needed to add iommu=pt to the grub entry and did this "echo "options vfio_iommu_type1 allow_unsafe_interrupts=1" > /etc/modprobe.d/iommu_unsafe_interrupts.conf"

rebooted the errors are GONE! BUTTTT when i do a PCI passthrough my whole box locks up. still working on it

 

[Echoesofdreams]

Im not out of the woods yet BUT i fixed most of my issues
(https://pve.proxmox.com/wiki/Pci_passthrough#Enable_the_IOMMU)

as per that doc i needed to add iommu=pt to the grub entry and did this "echo "options vfio_iommu_type1 allow_unsafe_interrupts=1" > /etc/modprobe.d/iommu_unsafe_interrupts.conf"

rebooted the errors are GONE! BUTTTT when i do a PCI passthrough my whole box locks up. still working on it

Thank you digitalbots

 

[digitalbots]

im about at my wits end here guys. Here are some outputs.


root@pve7:~# dmesg | grep -i vfio
[ 10.533481] VFIO - User Level meta-driver version: 0.3

root@pve7:~# find /sys/kernel/iommu_groups/ -type l | grep 01:00
/sys/kernel/iommu_groups/16/devices/0000:01:00.0
/sys/kernel/iommu_groups/16/devices/0000:01:00.1

So it shows the card. I add it to my VM via the pci passthrough turn the VM on and BOOM i lose connect via ssh and https to proxmox and I have to hardwire to it to reboot because all VMS go offline when turn on the VM with the pci passthrough.

also these are the settings for my VM

/etc/pve/qemu-server/105.conf
boot: order=scsi0
cores: 2
hostpci0: 0000:01:00,rombar=1
memory: 5048
name: pfsense
numa: 0
ostype: l26
scsi0: local-lvm:vm-105-disk-0,size=32G
scsihw: virtio-scsi-pci
smbios1: uuid=57eccb20-0773-4443-b0fa-e6f216565a54
sockets: 2
vmgenid: f6849b76-f0e6-4039-978e-35ef837abe5a

 

[bobmc]

Why do you feel you need to do pci passthrough? would it not be simpler to use the vm bridge method?

 

[Echoesofdreams]

Why do you feel you need to do pci passthrough? would it not be simpler to use the vm bridge method?

The PCI express holds the two Ethernet ports for LAN and WAN probably via a NIC

 

[digitalbots]

The PCI express holds the two Ethernet ports for LAN and WAN probably via a NIC

that is correct! i need the two ports for WAN/Lan for the pfsense

 

[digitalbots]

OK so i took the card out and found that it has the SAME NAME AS MY ONBOARD LAN. Because I have a poweredge I have 2 onboard Lans already. So it came down to the fact I picked the wrong one.

I would recommend anyone that tries to do this write down your PCI device ID's for your current NIC's so that you dont do what i did!

 

[bobmc]

Yes, you need two NICs (WAN/LAN) and the WAN port should be dedicated to the pfSense VM. But the LAN port can be shared with the rest of the system and this allows all your VM's and containers to communicate directly with the pfSense system.

Official how-to from Netgate
https://docs.netgate.com/pfsense/en/latest/recipes/virtualize-proxmox-ve.html

Just remember to disable 'Hardware Checksums'

 

[Echoesofdreams]

OK so i took the card out and found that it has the SAME NAME AS MY ONBOARD LAN. Because I have a poweredge I have 2 onboard Lans already. So it came down to the fact I picked the wrong one.

I would recommend anyone that tries to do this write down your PCI device ID's for your current NIC's so that you dont do what i did!

How do you find your PCI device IDs for the current NIC

 

[Echoesofdreams]

Yes, you need two NICs (WAN/LAN) and the WAN port should be dedicated to the pfSense VM. But the LAN port can be shared with the rest of the system and this allows all your VM's and containers to communicate directly with the pfSense system.

Official how-to from Netgate
https://docs.netgate.com/pfsense/en/latest/recipes/virtualize-proxmox-ve.html

Just remember to disable 'Hardware Checksums'

This was the best way to do it. This fixed my issue.