[SOLVED] Chelsio T4 NIC Has Old PBA Offset Problem

[1ButteryAdmin]

Hello,

I'm trying to passthrough a Chelsio T420-CR to a pfSense VM running on Proxmox version 6.2, but it's preventing the VM from booting. It gives this error:

kvm: -device vfio-pci,host=0000:83:00.7,id=hostpci1.7,bus=pci.0,addr=0x11.7: vfio 0000:83:00.7: hardware reports invalid configuration, MSIX PBA outside of specified BAR
TASK ERROR: start failed: QEMU exited with code 1

Doing some research, apparently this is an old problem with Chelsio devices that was patched out for T5 series, and fixed in later series, according to this conversation:
https://patchwork.ozlabs.org/projec...5-32152-1-git-send-email-glaupre@chelsio.com/

On that page is a patch with a workaround for it, but from what i can gather it seems to reference files that no longer exist. I am very new to Proxmox though, and could easily be wrong. Is there any way that I can use that patch, or does another patch exist to fix this problem? Or is this possibly just a problem with my configuration?

 

[1ButteryAdmin]

Update: I have realized that the patch that I linked indicates a bug with qemu and not Proxmox. I found the file specified in the patch on the qemu GitHub page, but it seems like they implemented another fix for it that's supposed to work as a catchall for products that encounter this error. I'm not sure if his counts as a Proxmox bug or not, since it's a problem with the baked-in version of qemu, but I will submit a bug report to Proxmox's bugzilla. As for my machine though, is it a better idea to wait for a response on the bugzilla page, attempt to use the current version of qemu from their GitHub, or grab a copy of the version Proxmox uses and apply the patch I linked to? Any help on this would be greatly appreciated.

 

[1ButteryAdmin]

Update 2: I am trying to install the latest version of qemu on my server, but I am running into the issue of not having libproxmox-backup-qemu0-dev installed no matter what I try. I've tried downloading pve-qemu.git from the repository, but it does not contain the file, and running mk-build-deps returns:
pve-qemu-kvm-build-deps : Depends: libproxmox-backup-qemu0-dev (>= 0.6.0-1) but it is not installable

I've seen several other threads with similar issues, but either they get it working without mentioning how, or the fixes do not work for me.

In https://forum.proxmox.com/threads/how-to-update-qemu-version.68161/ they have the same problem of not being able to find the file, but end up finding it on the repo without saying where.

In https://forum.proxmox.com/threads/help-for-patching-pve-qemu.70936/#post-318279 I have run all the same steps recommended, but mk-build-deps --install debian/control returns/:
Broken pve-qemu-kvm-build-deps:amd64 Depends on libproxmox-backup-qemu0-dev:amd64

I've also tried using backup version with rust but got a rust environment error on attempting to compile. This error is discussed here: https://forum.proxmox.com/threads/h...st-cargo-environment-to-build-packages.72258/ but is not marked solved, and neither of the versions of pve-qemu listed in that thread can be compiled because of a missing libproxmox-backup-qemu0-dev.

I have tried searching through the main repository for the file, but it does not exist. The only place where I have seen anything close to it is in proxmox-backup-qemu.git which does contain two .install files in its debian folder, but this is the version that cannot be compiled due to rust environment problems, which, as mentioned before, do not seem to have any resolution.

Is there anyone that has a solution for this? It seems that all I need is a way to get libproxmox-backup-qemu0-dev installed, but I do not see a way how.

 

[1ButteryAdmin]

Update 3:
The issue I was having with downloading libproxmox-backup-qemu0-dev was caused by my sources.list file having a line that pointed to files for debian jessie instead of stretch.

The issue with Chelsio T4 devices is caused by a device, in my case 81:00.7 being exposed to proxmox despite it not providing any functions at all and its PBA was invalid. The problem is on Chelsio's part for this one, but it is possible to use PCI-e passthrough on these cards by adding only the functions that have ethernet controller as their descriptions in proxmox. for me those were 81:00.0-81:00.4. Passing through the other functions of the cards should be possible by only passing those functions through, but I didn't try it. I don't think it's possible to pass through both ports as ethernet devices and as other functions like FCoE HBAs, since adding each ethernet controller function separately requires using 5 passthrough slots, and that appears to be the hard limit on number of cards that can be passed through.

The upstream QEMU engineer I talked to said that it is possible that enabling SR-IOV on these cards might fix the issue, but as far as I can tell Chelsio removed the ability of the configuration software/option ROM of their cards to set SR-IOV on T4 devices. The product page for them still says that they are capable of using SR-IOV, but even the older version of Unified Wire that I downloaded, which did come with drivers for T4 devices, did not list any T4 cards as having the option to enable SR-IOV. Perhaps an even older version would be able to.

 

[ccp]

Did you ever manage to solve this? I am running in the same issue

 

[Max Kulik]

This is an old thread but I wanted to share my results since I was having the same problem.
I have a Chelsio T420-BT that I added to my server and wanted to pass to pfsence. It turns out that adding the pci devices 0-3 on the Proxmox web interface AND having "All Functions" UNCHECKED worked for me. I don't understand why that last function is exposed to Proxmox but that is an easy workaround.