I also observed that a node with a vm was running longer. I decided to downgrade as fast as possible as you might just try your luck.
I downgraded/pined the kernel to 7.0.14-19 and everything now remains rock solid.
I must admit that iommu=pt did run longer (about 32 minutes, but crashed also in the end)
Unfortunately i made an error configuring the kernel-parameters, i am rechecking.
College told me to try to add
iommu=pt
in
/etc/kernel/cmdline
A similar bug seems to arise in current debian kernels (6.12.111) with crashes after arising after 5-30 minutes. I am checking.
I can basically confirm your findings, we have the following trouble:
7.0.14-20
Network connectivity on an empty host fails after a short time with
================
NETDEV WATCHDOG: CPU x; transmit queue 1 timed out 5700 ms
bnxt_en: 0000:01:00.1 NETDEV WATCHDOG CPU y: transmit queue 2 timed...
You could try jumbo-frames. Are you using a firewall (like pf) on FreeBSD ? Is the result CPU bound (are you using 100% cpu ?)?
Have you tried to set network-card-Features ? As far as i remember this could be due to LRO, TSO etc. not set on Freebsd.
Have you tried to set
Hardware Checksum Offloading
[X] Disable hardware checksum offload
I have the same problems with pfsense throughput without disabling the hardware checksum. Throughput is < 100 Kbit otherwise.