For the benefit of others, I have managed an almost successful installation of latest Chelsio Unified Wire drivers on Proxmox 7.
My config is :
-HW : Chelsio T440-CR
-Proxmox 7.2-4
-Kernel : 5.15.35-2
-Chelsio Unified Driver Version (INSTALLED FROM SOURCE) : 3.16.0.1
Installation script result :
***********************
* Summary *
***********************
CONFIG = UNIFIED_WIRE
Protocol Modules\Libraries\Tools Action Status
------------------------------------------------------------------------------------------
Bonding-Offload bonding Build Successful
Chelsio-utils(tools) cxgbtool/cop Build Successful
FCoE(full-offload-initiator) csiostor Build Successful
IPv6-Offload t4_tom Build Successful
iSCSI(pdu-offload-target) chiscsi_t4 Build Not-supported
iWARP-lib libcxgb4 Build Successful
iWARP-lib libcxgb4_sock Build Failed
iWARP-lib libcxgb4_sock_dbg Build Failed
iWARP-lib libcxgb4_udp Build Failed
iWARP-lib libcxgb4_udp_dbg Build Failed
LIO-Target cxgbit Build Successful
Network(NIC) cxgb4 Build Successful
Network-Offload(TOE) t4_tom Build Successful
NVMe-Utils nvme Build Successful
ptp-utils ptp Build Successful
RDMA(iWARP) iw_cxgb4 Build Failed
Sniffer wd_tcpdump Build Successful
SR-IOV_networking(vNIC) cxgb4vf Build Successful
UDP_offload t4_tom Build Successful
Steps to reproduce :
1/ Download the latest tar.gz from Chelsio in a known temporary folder.
2/ Untar with : tar -zxvf ChelsioUwire-3.16.0.1.tar.gz (version will be different).
3/ Make sure you install the rpm package and associated dependencies : sudo apt install rpm
4/ Install the linux headers (kernel source) for your active kernel with command : sudo apt install pve-headers-`uname -r`
5/ Take note of the kernel sources installation path. In my case : usr/src/linux-headers-5.15.35-2-pve/
6/ Go to your untar folder. In my case /root/ChelsioUwire-3.16.0.1
7/ Edit the Makefile you find in this directory and modify the following capitalized variables at the top of the file :
DEBIAN := 1
PDEB := 1
DISTRO := Debian
This should trick the installer into believing it is building against a Debian System. Setting DEBIAN := 1 seems to be insufficient according to the tests the installer is running afterwards (and to my failed attempts), resulting in a wrong directory structures for the built packages.
8/ Now the last piece of the puzzle is to specify the kernel source (KSRC) / object (KOBJ) path both under the same variable KDIR. I have just tried to solve the recurring error message when the installer could not determine the Kernel source location.
I have done it by starting the build with the following command, at the same location than the modified Makefile.
make KDIR=/usr/src/linux-headers-5.15.35-2-pve/
It should be possible to specify this KDIR variable in a Makefile somewhere thus allowing the graphical installation.
The first time I attempted an installation I was asked KSRC and KOBJ paths, but none of the successive attempts allowed me to specify these path again, despite rolling back my root fs snapshot (I am using ZFS). My command launches the CLI installation which is more or less documented by Chelsio. Thus you could fine tune the components to be installed,
Important note :
Keep in mind that installing these drivers on top of Proxmox is far from ideal. First you'll have to reinstall them on every kernel update (which happen quite often). Second you cannot safely do updtate-initramfs anymore as it will break your boot image. The idea of this driver package is to deploy against a fix kernel version, such as in an Enterprise environment, which is usually not the case with Proxmox implementations. Also the driver installation recompiles the kernel in order to include the Chelsio driver. The standard cxgb4 driver provided by Proxmox is a module thus allowing dynamic loading on top of any kernel. It is thus much preferable to use the Proxmox standard driver in most cases even if it is an older version. From my tests, the stock driver allows most usual functionalities such as SR-IOV support.
