PCIe passthrough of Chelsio T520-BT crashes under Proxmox 8, worked in 7

ohmantics

New Member
Jan 23, 2023
4
0
1
I have a Q35 guest passing through a Chelsio T520-BT ethernet card. It locks up with a black screen before anything is output to the console, text or graphics.

Code:
bios: ovmf
boot:
cores: 12
cpu: host
efidisk0: local-zfs:vm-100-disk-1,efitype=4m,size=1M
hostpci1: 0000:3b:00,pcie=1,rombar=0
hostpci2: 0000:5e:00,pcie=1,rombar=0
hostpci3: 0000:5f:00,pcie=1,rombar=0
machine: q35
memory: 196608
meta: creation-qemu=7.1.0,ctime=1674451925
name: freenas
net0: virtio=CE:89:5E:8D:F3:12,bridge=vmbr1
numa: 0
ostype: other
sata0: /dev/disk/by-id/ata-32GB_SATA_Flash_Drive_E0318293800000001245,size=31266648K
sata1: /dev/disk/by-id/ata-32GB_SATA_Flash_Drive_E0318293800000001249,size=31266648K
scsihw: virtio-scsi-single
smbios1: uuid=f8796147-7449-44ec-9fd7-cee5b0f8c66a
sockets: 2
unused0: local-zfs:vm-100-disk-0
vga: std
vmgenid: a8a24189-4891-49c5-9d83-95279b100a0d

Grabbed this by adding a serial0 with "qm terminal 100":
Code:
!!!! X64 Exception Type - 0D(#GP - General Protection)  CPU Apic ID - 00000000 !!!!
ExceptionData - 0000000000000000
RIP  - 000000006E71AFA4, CS  - 0000000000000038, RFLAGS - 0000000000010206
RAX  - 7FFFFFFFFFFFFF43, RCX - 800000000000001A, RDX - 000000006E7267D0
RBX  - 000000006E7267D0, RSP - 000000007EDAF780, RBP - FFFFFFFF918D982F
RSI  - 000000000000002D, RDI - 800000000000001A
R8   - 000000000000002E, R9  - 000000006D880598, R10 - 0000000000000080
R11  - 000000006D880698, R12 - 000000006DC12298, R13 - 800000000000001A
R14  - 0000000000000000, R15 - 000000000000002E
DS   - 0000000000000030, ES  - 0000000000000030, FS  - 0000000000000030
GS   - 0000000000000030, SS  - 0000000000000030
CR0  - 0000000080010033, CR2 - 0000000000000000, CR3 - 000000006EA01000
CR4  - 0000000000000668, CR8 - 0000000000000000
DR0  - 0000000000000000, DR1 - 0000000000000000, DR2 - 0000000000000000
DR3  - 0000000000000000, DR6 - 00000000FFFF0FF0, DR7 - 0000000000000400
GDTR - 000000006E653000 0000000000000047, LDTR - 0000000000000000
IDTR - 000000006E14C018 0000000000000FFF,   TR - 0000000000000000
FXSAVE_STATE - 000000007EDAF3E0
!!!! Find image based on IP(0x6E71AFA4) d:\edk4\edk2.4\myworkspace\Build\MdeModule\DEBUG_VS2008x86\X64\MdeModulePkg\Bus\Pci\ChelsioUEFI\ChelsioUD\DEBUG\ChelsioUD.pdb (ImageBase=000000006E6F7000, EntryPoint=000000006E6F72C0) !!!!

I tried swapping /usr/lib/firmware/cxgbe4/t5fw.bin back to the version that shipped with Proxmox 7 and there was no difference. The guest starts normally if hostpci3 (the Chelsio T520-BT) is removed from the config. Reverted to Proxmox 7 using ZFS boot environments and everything works.
 
Updated the firmware on the card from 1.27.0.0 to 1.27.3.0 and the problem seems to have resolved itself.

Seems like upgrades to TianoCore in Proxmox are a risk... perhaps there should be a feature to select which version of UEFI firmware is used for each VM (if there isn't already).
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!