Help with pci passthrough for nics

juju01

Member
May 16, 2020
87
3
13
I have been trying to setup 2 nics on my freshly installed pfsense vm with pci passthrough. I have gone through all the guides and set things up as directed. Nothing seems to work. As soon as I try to start the pfsense vm for the first time, the whole host goes down - I have to delete the vm and reboot to get back into the proxmox gui. Could do with some help to troubleshoot this.

here is my dmesg | grep -e DMAR -e IOMMU

Code:
~# dmesg | grep -e DMAR -e IOMMU
[    0.013174] ACPI: DMAR 0x000000006C217FC0 000168 (v01 SUPERM SMCI--MB 00000001 INTL 20091013)
[    0.225576] DMAR: IOMMU enabled
[    0.414136] DMAR: Host address width 46
[    0.414138] DMAR: DRHD base: 0x000000c5ffc000 flags: 0x0
[    0.414143] DMAR: dmar0: reg_base_addr c5ffc000 ver 1:0 cap 8d2078c106f0466 ecap f020df
[    0.414144] DMAR: DRHD base: 0x000000e0ffc000 flags: 0x0
[    0.414148] DMAR: dmar1: reg_base_addr e0ffc000 ver 1:0 cap 8d2078c106f0466 ecap f020df
[    0.414149] DMAR: DRHD base: 0x000000fbffc000 flags: 0x0
[    0.414152] DMAR: dmar2: reg_base_addr fbffc000 ver 1:0 cap 8d2078c106f0466 ecap f020df
[    0.414153] DMAR: DRHD base: 0x000000aaffc000 flags: 0x1
[    0.414156] DMAR: dmar3: reg_base_addr aaffc000 ver 1:0 cap 8d2078c106f0466 ecap f020df
[    0.414157] DMAR: RMRR base: 0x0000006e813000 end: 0x0000006e823fff
[    0.414158] DMAR: ATSR flags: 0x0
[    0.414159] DMAR: RHSA base: 0x000000aaffc000 proximity domain: 0x0
[    0.414160] DMAR: RHSA base: 0x000000c5ffc000 proximity domain: 0x0
[    0.414161] DMAR: RHSA base: 0x000000e0ffc000 proximity domain: 0x0
[    0.414161] DMAR: RHSA base: 0x000000fbffc000 proximity domain: 0x0
[    0.414163] DMAR-IR: IOAPIC id 12 under DRHD base  0xfbffc000 IOMMU 2
[    0.414164] DMAR-IR: IOAPIC id 11 under DRHD base  0xe0ffc000 IOMMU 1
[    0.414164] DMAR-IR: IOAPIC id 10 under DRHD base  0xc5ffc000 IOMMU 0
[    0.414165] DMAR-IR: IOAPIC id 8 under DRHD base  0xaaffc000 IOMMU 3
[    0.414166] DMAR-IR: IOAPIC id 9 under DRHD base  0xaaffc000 IOMMU 3
[    0.414167] DMAR-IR: HPET id 0 under DRHD base 0xaaffc000
[    0.414168] DMAR-IR: x2apic is disabled because BIOS sets x2apic opt out bit.
[    0.414168] DMAR-IR: Use 'intremap=no_x2apic_optout' to override the BIOS setting.
[    0.414993] DMAR-IR: Enabled IRQ remapping in xapic mode
[    1.421598] DMAR: dmar2: Using Queued invalidation
[    1.421601] DMAR: dmar1: Using Queued invalidation
[    1.421603] DMAR: dmar3: Using Queued invalidation
[    1.435914] DMAR: Intel(R) Virtualization Technology for Directed I/O

pci devices:

Code:
~# lspci | egrep -i --color "network|ethernet"
68:00.0 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01)
68:00.1 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01)
68:00.2 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01)
68:00.3 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01)
b7:00.0 Ethernet controller: Intel Corporation Ethernet Connection X722 for 10GBASE-T (rev 04)
b7:00.1 Ethernet controller: Intel Corporation Ethernet Connection X722 for 10GBASE-T (rev 04)
b7:00.2 Ethernet controller: Intel Corporation Ethernet Connection X722 for 10GbE SFP+ (rev 04)
b7:00.3 Ethernet controller: Intel Corporation Ethernet Connection X722 for 10GbE SFP+ (rev 04)


setup
Code:
balloon: 2048
bootdisk: virtio0
cores: 2
hostpci0: 68:00
hostpci1: b7:00
ide2: synology-nfs:iso/pfSense-CE-2.4.5-RELEASE-p1-amd64.iso,media=cdrom
machine: q35
memory: 4096
name: pfsense
numa: 0
ostype: other
scsihw: virtio-scsi-pci
smbios1: uuid=f04e6475-9f63-4e87-80e3-70f1d64967a2
sockets: 1
virtio0: local-zfs-vmstore:vm-100-disk-0,size=40G
vmgenid: 498009d3-1f38-46da-b347-2e77d91d0d11
 
Update:

I posted this too soon. Whilst posting this, I noticed the following two lines:
Code:
hostpci0: 68:00
hostpci1: b7:00

I had been trying to specifically select 68.00.1 and b7:00.3. But somehow, even though I selected those specific nics when I added the pci hadware, they show up as seen above. So i edited the conf file and changed the above to the correct values - and pfsense came right up. Thanks

new values:
Code:
hostpci0: 68:00.1
hostpci1: b7:00.3
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!