Very poor networking via thunderbolt

[rossd]

I have two nucs connected via thunderbolt (10.0.0.0/24). They are also connected via ethernet to a switch (192.168.86.0/24)

If I use iperf to test the speed from nuc a, to nuc b using the ethernet addresses I get speeds of about 935Mbits/sec. Using the thunderbolt addresses I get speeds of 7.38 Gbits/sec. Still slower than I would expect, but I'm ok with that.

I have several hosts running on the hosts, and all have two nics added with ips from both networks.

If I iperf from a host on nuc a, to nuc b using the thunderbolt addresses I get speeds of about 10Gbits/sec.

If I iperf from a host on nuc a, to a host on nuc b using the thunderbolt addresses I get speeds of 3.56 Mbits/sec!?!. Using the ethernet addresses I get speeds of 933 Mbits/sec.

and finally from a host on nuc b, to nuc using thunderbolts I get speeds of 10Gbits/sec.

I'm really confused why the speed is so terrible between hosts running on different nucs, but using the thunderbolt network. I must have something obvious wrong here!

 

[Docop2]

thunderbolt support is very <finiky > over linux and while it's fun to test out, all do simply get a proper network card. sonnet box are good for this.

 

[Allister]

Try rolling back to kernel 6.5.13-5-pve as it will improve Thunderbolt throughput quite a bit. The current 6.8 line of kernels causes all sorts of Thunderbolt issues. You have to also set the affinity of the CPUs for Thunderbolt as well, search these forums for thunderbolt affinity to find some info.