Proxmox Cluster over 2 Sites using VXLAN over Wireguard

[em3034]

Hi All,

I have 2 home locations: site1 and site2:
- Site1 has a pfSense router with a Public IP and a 2 subnets: 192.168.1.0/24 and 192.168.10.0/24
- Site2 has a unifi gateway max with also a public IP and 2 subnets: 192.168.2.0/24 and 192.168.20.0/24.

I managed to create a wireguard Site to Site VPN between the 2 routers, so that from each subnet I can access other subnets.
Later on I created a Proxmox Cluster with Node1 192.168.1.2 and Node2 192.168.2.2 and also a few VMs, and from each VM I'm able to ping the other VMs. So far so good!

Now the next step in my testing is to be able to create a subnet that is available in both sites (a kind of streched subnet) so that I can migrate a VM from one site to the other one without service interruption.
For this I want to use VXLAN (later on EVPN) and put VMs in a VXLAN subnet, so that the VM can run on any site.

I tried to create a VXLAN using SDN, this was straighward.
I created 2 VMs in Site1 and 2 VMs in Site2 having their NIC in the VXLAN subnet. VMs from the same site can ping each other but I'm not able to ping VMs located in different sites.

My SDN Config is as follow (on node1, same on node 2 apart from the remoteip:

auto vxlan_vxnet1
iface vxlan_vxnet1
    vxlan-id 11111
    vxlan_remoteip 192.168.2.2
    mtu 1350

auto vxnet1
iface vxnet1
    bridge_ports vxlan_vxnet1
    bridge_stp off
    bridge_fd 0
    mtu 1350
    alias Vxlan Test

Also when I run tcpdump port 4789 on the proxmox hosts I can see VXLAN traffic, but no response from the remote site.
This does mean that the VXLAN traffic can cross the WireGuard tunnel, but not coming back.

My network skills stop here and I'm wondering what do I miss to make this work.

Thanks a lot!