Can't get packet forwarding working from LAN > thunderbolt mesh network.

scyto

Well-Known Member
Aug 8, 2023
572
135
53
I have 3 nodes they have:
  • public interfaces facing the LAN
  • thunderbolt-net interfaces connected in a mesh
  • ceph configured to use the mesh as its public and private networks
  • for purposes of this scenario testing all firewalls on VMs
  • Node1's private addresses are:
    • 10.0.0.81/32
    • fc00::81/128
  • Node1s public addresses are:
    • 192.168.1.81/24
    • xxxx:xxxx:830::81/64
I used FRR to get each node routing to each other, even if one cable fails the mesh is resillant. see https://gist.github.com/scyto/4c664734535da122f4ab2951b22b2085

I would now like to get clients on my LAN to access ceph. however i cant get forwarding from the LAN > Mesh.
The actual client i am interested in doing this with is a VM on pve1. It has an address of 192.168.1.100 and xxxx:xxxx:830::<see the picture below>

From the VM i can ping both 10.0.0.81 and fc00::81 (same in the opposite direction).
From the VM if i try and ssh to fc00::81 it hangs and eventuall says connection reset by peer - i did a wirehshark and saw a single packet come back from fc00::81 - but no other packerts.

This tells me something really odd is going on, and i don't know what.

Yes kernel forwarding is turned on for both IPv4.
Yes SSHD is listening on :: and 0.0.0.0

Here is a wireshark from the client VM when i try to ssh into fc00::81.... (IPv4 has same symptoms)

1745019189474.png

Any one any ideas?
 
Last edited: