Problem setting up VXLAN

[rabinauget]

Hello,

I'm new and I don't have much experience with Proxmox.
I'd like to ask you to help me set up a VXLAN.

Here goes:
-I've got a cluster with three nodes, each with a different Public IP address because they're with different hosts,
-I was able to integrate them into one cluster without any problem,
-Now I want to create a VxLAN so that the VMs on the different nodes can communicate;
-I've already created a zone, a vnet, a subnet and the vxnet is already visible on the three nodes, they are already available on SDN;
-I then created a PfSense VM and a Windows VM on each node;
-But the problem is that the Windows VM and the PfSense VM on the same node are able to communicate, but they can't communicate with the VMs on the other nodes; it's the same on all nodes;
-And I've set a DHCP IP allocation but when my VMs can't get an IP via DHCP, I think I've configured the SDN correctly.
-I've already authorized port 4789/UDP on the nodes.

I've set the MTU in the VXLAN area to 1450 and for the VM vxnet map to 1400 but it still doesn't work.

Could you please help me?

My goal is to set up a cluster, then a network on the cluster, then Ceph for shared storage, then HA, then backup and restore.

Don't hesitate to let me know if I need to correct anything in my publication.

SDN state capture:


Zone Capture:


Vnet capture:


Subnet capture:



Capture networks on nodes


IP Manager:


Regards,

 

[derstandard]

In the moment, the DHCP-Feature works only for simple-zones, DHCP support for other kind of SDN is planned for 8.2 ...
So use your PFsense (with an interface IN the vxlan) to be DHCP-Server

 

[rabinauget]

Thank you for your feedback.
In fact, my problem is that even though my virtual machines are statically configured, they can't join despite the fact that they're on the SDN network I've created. Can you please help me?

The DHCP I created was a test to confirm whether the machines manage to have a dynamic address or not.

 

[derstandard]

I had a similar case ( inter-node connection got brocken when added a bgppeer)... I removed all sdn settings, applied them, deleted /etc/frr/frr.conf at all nodes, systemctl restart frr and then build the sdn complete from scratch... Maybe this could help you here too?

Are there any Firewalls active in Front of the servers or enabled in proxmox itself?

 

[rabinauget]

I corrected the problem by changing the MTU of all the cards that use sdn to 1450. I then used pfsense to do the DHCP and it works correctly. Thanks for your feedback.

Now my concern is to get the three nodes to use the SDN VXLAN for the CEPH storage I'm going to create. When I try to configure CEPH, it doesn't show me the VXLAN network but just the WAN and LAN. Can you help me please? Or should I create another extension for this problem?

The root of the problem is that my nodes each have a different public ip and when I try to create CEPH storage, the other two nodes don't see the IP of the one on which I activated CEPH first. It looks like they're scanning the range of the first node's network even though they have a different network's public IP.