VxLAN and 1500 MTU

M

mkapalla

New Member
Feb 16, 2025
2
0
1
Hi everybody,

i have a question regarding the MTU whilst using VxLAN.

My setup is as follows:
I have my servers connected to a mikrotik swtich via 25 GBit link.
The servers are situated on a VLAN using a OVS Bridge and an IntPort. (cant be changed due to other constraints)

Now i have the need for certain VMs to be on the same layer 2 network without assigning VLANs to them directly (at least virtually, some VMs need the ability to do VLAN, etc.) - thats why i enabled VxLAN and added the VMs to the VxLAN. Connectivity so far is fine, only issue is that my maximum achievable MTU is 1426. The issue is that the VMs need a MTU of 1500 for all services to work right.

My thought now is to increase the MTU in my mikrotik switch to something higher (maybe even just to 9000) and then set the MTU of the VxLAN to 1500.

Would that lead to my desired outcome, or are there better ways to achieve this? In my mind this might work just fine.
 
Im curious, I'm doing exaaactly the same and stumbling upon some issues but if I achieve something I will let you know
 
I've increased the mtu, but I'm still having issues to achieve connection between nodes. I've set up my fwl to be as lax as possible; even logging firewall rules can see the packets passing thru but I'm still unable to communicate two separated vms.
 
freakingObelix said:
I've increased the mtu, but I'm still having issues to achieve connection between nodes. I've set up my fwl to be as lax as possible; even logging firewall rules can see the packets passing thru but I'm still unable to communicate two separated vms.
Click to expand...
proxmox firewall ? or a physical firewall/router somewhere on your network ? (in this case, the mtu of the interfaces of the firewall need to be increased too)
 
I actually achieved something. Here is what I did:

I set the MTU on my switches L2 interfaces to 9284, on L3 to 9200.

On the proxmox hosts I set the interfaces to 9200 too. Then I created a second OVS IntPort that dedicated to VXLAN.
There I set the MTU to 9200 too.

In my VXLAN config I set the MTU to 9000 and between the VMs I can now use 1500 MTU safely and also have capacity for jumbo frames.
 
@spirit my phisical fabrics all have 9200 by default (tested that manually just to be safe).
But some devices can't tolerate more than 1500, and even some don't have the option to set the mtu (like environment sensors, controllers, and UPS). My doubt begin there: apart from ICMP doing a role, what happens when you set a different MTU per port on a l3 switch or a router? Does the router or switch play a role or packets get forwarded as is? (Mikrotik, and Linux Based OSes)
 
You must log in or register to reply here.
Top Bottom