NIC Passthrough failling

A

arrcy

Member
Mar 21, 2020
17
0
21
24
hi proxmox community


im trying to get a 4port NIC passthrough but seem te be failing
every time i try to start the vm the hypervisor crashes and i cant seem to find any logs

i want t setup pfsense inside a virtual box


some information

lspci -nnk
gives

Code: 
00:00.0 Host bridge [0600]: Intel Corporation 4th Gen Core Processor DRAM Controller [8086:0c00] (rev 06)
        Subsystem: Dell 4th Gen Core Processor DRAM Controller [1028:05a5]
        Kernel driver in use: hsw_uncore
00:01.0 PCI bridge [0604]: Intel Corporation Xeon E3-1200 v3/4th Gen Core Processor PCI Express x16 Controller [8086:0c01] (rev 06)
        Kernel driver in use: pcieport
00:02.0 VGA compatible controller [0300]: Intel Corporation Xeon E3-1200 v3/4th Gen Core Processor Integrated Graphics Controller [8086:0412] (rev 06)
        Subsystem: Dell Xeon E3-1200 v3/4th Gen Core Processor Integrated Graphics Controller [1028:05a5]
        Kernel driver in use: i915
        Kernel modules: i915
00:03.0 Audio device [0403]: Intel Corporation Xeon E3-1200 v3/4th Gen Core Processor HD Audio Controller [8086:0c0c] (rev 06)
        Subsystem: Dell Xeon E3-1200 v3/4th Gen Core Processor HD Audio Controller [1028:05a5]
        Kernel driver in use: snd_hda_intel
        Kernel modules: snd_hda_intel
00:14.0 USB controller [0c03]: Intel Corporation 8 Series/C220 Series Chipset Family USB xHCI [8086:8c31] (rev 04)
        Subsystem: Dell 8 Series/C220 Series Chipset Family USB xHCI [1028:05a5]
        Kernel driver in use: vfio-pci
00:16.0 Communication controller [0780]: Intel Corporation 8 Series/C220 Series Chipset Family MEI Controller #1 [8086:8c3a] (rev 04)
        Subsystem: Dell 8 Series/C220 Series Chipset Family MEI Controller [1028:05a5]
        Kernel driver in use: mei_me
        Kernel modules: mei_me
00:16.3 Serial controller [0700]: Intel Corporation 8 Series/C220 Series Chipset Family KT Controller [8086:8c3d] (rev 04)
        Subsystem: Dell 8 Series/C220 Series Chipset Family KT Controller [1028:05a5]
        Kernel driver in use: serial
00:19.0 Ethernet controller [0200]: Intel Corporation Ethernet Connection I217-LM [8086:153a] (rev 04)
        Subsystem: Dell Ethernet Connection I217-LM [1028:05a4]
        Kernel driver in use: e1000e
        Kernel modules: e1000e
00:1a.0 USB controller [0c03]: Intel Corporation 8 Series/C220 Series Chipset Family USB EHCI #2 [8086:8c2d] (rev 04)
        Subsystem: Dell 8 Series/C220 Series Chipset Family USB EHCI [1028:05a5]
        Kernel driver in use: vfio-pci
00:1b.0 Audio device [0403]: Intel Corporation 8 Series/C220 Series Chipset High Definition Audio Controller [8086:8c20] (rev 04)
        Subsystem: Dell 8 Series/C220 Series Chipset High Definition Audio Controller [1028:05a5]
        Kernel driver in use: snd_hda_intel
        Kernel modules: snd_hda_intel
00:1c.0 PCI bridge [0604]: Intel Corporation 8 Series/C220 Series Chipset Family PCI Express Root Port #1 [8086:8c10] (rev d4)
        Kernel driver in use: pcieport
00:1c.4 PCI bridge [0604]: Intel Corporation 8 Series/C220 Series Chipset Family PCI Express Root Port #5 [8086:8c18] (rev d4)
        Kernel driver in use: pcieport
00:1d.0 USB controller [0c03]: Intel Corporation 8 Series/C220 Series Chipset Family USB EHCI #1 [8086:8c26] (rev 04)
        Subsystem: Dell 8 Series/C220 Series Chipset Family USB EHCI [1028:05a5]
        Kernel driver in use: vfio-pci
00:1f.0 ISA bridge [0601]: Intel Corporation Q87 Express LPC Controller [8086:8c4e] (rev 04)
        Subsystem: Dell Q87 Express LPC Controller [1028:05a5]
        Kernel driver in use: lpc_ich
        Kernel modules: lpc_ich
00:1f.2 SATA controller [0106]: Intel Corporation 8 Series/C220 Series Chipset Family 6-port SATA Controller 1 [AHCI mode] [8086:8c02] (rev 04)
        Subsystem: Dell 8 Series/C220 Series Chipset Family 6-port SATA Controller 1 [AHCI mode] [1028:05a5]
        Kernel driver in use: ahci
        Kernel modules: ahci
00:1f.3 SMBus [0c05]: Intel Corporation 8 Series/C220 Series Chipset Family SMBus Controller [8086:8c22] (rev 04)
        Subsystem: Dell 8 Series/C220 Series Chipset Family SMBus Controller [1028:05a5]
        Kernel driver in use: i801_smbus
        Kernel modules: i2c_i801
01:00.0 VGA compatible controller [0300]: NVIDIA Corporation GP108 [10de:1d01] (rev a1)
        Subsystem: Gigabyte Technology Co., Ltd GP108 [GeForce GT 1030] [1458:375c]
        Kernel driver in use: vfio-pci
        Kernel modules: nvidiafb, nouveau
01:00.1 Audio device [0403]: NVIDIA Corporation GP108 High Definition Audio Controller [10de:0fb8] (rev a1)
        Subsystem: Gigabyte Technology Co., Ltd GP108 High Definition Audio Controller [1458:375c]
        Kernel driver in use: vfio-pci
        Kernel modules: snd_hda_intel
03:00.0 PCI bridge [0604]: Integrated Device Technology, Inc. [IDT] PES12N3A PCI Express Switch [111d:8018] (rev 0e)
04:02.0 PCI bridge [0604]: Integrated Device Technology, Inc. [IDT] PES12N3A PCI Express Switch [111d:8018] (rev 0e)
        Kernel driver in use: pcieport
04:04.0 PCI bridge [0604]: Integrated Device Technology, Inc. [IDT] PES12N3A PCI Express Switch [111d:8018] (rev 0e)
        Kernel driver in use: pcieport
05:00.0 Ethernet controller [0200]: Intel Corporation 82571EB Gigabit Ethernet Controller (Copper) [8086:10bc] (rev 06)
        Subsystem: Intel Corporation PRO/1000 PT Quad Port LP Server Adapter [8086:10bc]
        Kernel driver in use: e1000e
        Kernel modules: vfio-pci
05:00.1 Ethernet controller [0200]: Intel Corporation 82571EB Gigabit Ethernet Controller (Copper) [8086:10bc] (rev 06)
        Subsystem: Intel Corporation PRO/1000 PT Quad Port LP Server Adapter [8086:10bc]
        Kernel driver in use: vfio-pci
        Kernel modules: e1000e
06:00.0 Ethernet controller [0200]: Intel Corporation 82571EB Gigabit Ethernet Controller (Copper) [8086:10bc] (rev 06)
        Subsystem: Intel Corporation PRO/1000 PT Quad Port LP Server Adapter [8086:10bc]
        Kernel driver in use: vfio-pci
        Kernel modules: e1000e
06:00.1 Ethernet controller [0200]: Intel Corporation 82571EB Gigabit Ethernet Controller (Copper) [8086:10bc] (rev 06)
        Subsystem: Intel Corporation PRO/1000 PT Quad Port LP Server Adapter [8086:10bc]
        Kernel driver in use: vfio-pci
        Kernel modules: e1000e

Code: 
cat /etc/modprobe.d/vfio.conf
gives
options vfio-pci ids=10de:1d01,10de:0fb8 disable_vga=1
options vfio-pci ids=8086:10bc



gpu passthrough is working

but when starting vm proxmox reboots without logs

Code: 
cat /etc/pve/qemu-server/<myid>.conf
gives

args: -machine type=q35,kernel_irqchip=on -cpu 'host,+kvm_pv_unhalt,+kvm_pv_eoi,hv_vendor_id=NV43FIX,kvm=off'
bios: ovmf
bootdisk: scsi0
cores: 3
efidisk0: local-lvm:vm-101-disk-0,size=128K
hostpci0: 05:00,pcie=1
hostpci1: 06:00,pcie=1
ide2: local:iso/pfSense-CE-2.4.4-RELEASE-p3-amd64.iso,media=cdrom
machine: q35
memory: 8096
name: pfsense
numa: 0
ostype: l26
scsi0: hdd1:101/vm-101-disk-0.qcow2,size=32G
scsihw: virtio-scsi-pci
smbios1: uuid=4054d50c-07fc-41b0-93bd-92a368c8f465
sockets: 1
vga: std,memory=256
vmgenid: 3d1aba0a-1815-41bc-977d-d20212ecd221


thx in advance if you need aditional information pls ask


greetings, arrcy

i found 2 reports on the console

1 and 0 pages still available in the bound and unbound gpu page list
pve login: (2151.737540) softdog: initiating system reboot


update:

seemed to be that a pce passthough was only working at 1 vm at a time
why idk
figured it out with this error 1 and 0 pages still available in the bound and unbound gpu page list

but anyone idears on how i can passthorugh 2 cards to 2 different vms
 
Last edited:
And how do you suggest i do this
I already followed the entire wiki
With no succes

Ty in advance
 
arrcy said:
And how do you suggest i do this
I already followed the entire wiki
Click to expand...

Really? It's written there:

Code: 
find /sys/kernel/iommu_groups/ -type l

You need to check the correspondance with your pci ids.

Depending on the card and the iommu groups, you cannot passthrough a single port, but that depends heavily on the iommu groups (maybe change the PCIe socket to get another group).
 
LnxBil said:
Really? It's written there:

Code: 
find /sys/kernel/iommu_groups/ -type l

You need to check the correspondance with your pci ids.

Depending on the card and the iommu groups, you cannot passthrough a single port, but that depends heavily on the iommu groups (maybe change the PCIe socket to get another group).
Click to expand...
oh thats what you mean
anyway this is the output
is there a way you can manualy adjust IOMMU groups

/sys/kernel/iommu_groups/7/devices/0000:00:1a.0
/sys/kernel/iommu_groups/15/devices/0000:06:00.0
/sys/kernel/iommu_groups/15/devices/0000:04:04.0
/sys/kernel/iommu_groups/15/devices/0000:06:00.1
/sys/kernel/iommu_groups/5/devices/0000:00:16.0
/sys/kernel/iommu_groups/5/devices/0000:00:16.3
/sys/kernel/iommu_groups/13/devices/0000:03:00.0
/sys/kernel/iommu_groups/3/devices/0000:00:03.0
/sys/kernel/iommu_groups/11/devices/0000:00:1d.0
/sys/kernel/iommu_groups/1/devices/0000:00:01.0
/sys/kernel/iommu_groups/1/devices/0000:01:00.0
/sys/kernel/iommu_groups/1/devices/0000:01:00.1
/sys/kernel/iommu_groups/8/devices/0000:00:1b.0
/sys/kernel/iommu_groups/6/devices/0000:00:19.0
/sys/kernel/iommu_groups/14/devices/0000:05:00.1
/sys/kernel/iommu_groups/14/devices/0000:04:02.0
/sys/kernel/iommu_groups/14/devices/0000:05:00.0
/sys/kernel/iommu_groups/4/devices/0000:00:14.0
/sys/kernel/iommu_groups/12/devices/0000:00:1f.2
/sys/kernel/iommu_groups/12/devices/0000:00:1f.0
/sys/kernel/iommu_groups/12/devices/0000:00:1f.3
/sys/kernel/iommu_groups/2/devices/0000:00:02.0
/sys/kernel/iommu_groups/10/devices/0000:00:1c.4
/sys/kernel/iommu_groups/0/devices/0000:00:00.0
/sys/kernel/iommu_groups/9/devices/0000:00:1c.0

so i should do this right
nano /etc/default/grub
change GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on"
to GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on pcie_acs_override=downstream"
 
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom