On my system, I have a 3-node HCI PVE cluster with Ceph storage and a SDN VLAN zone with about 21 VNETs copied from the previous system's VMware vDS. So far, all of this has worked out for me for almost a year. Currently, my version is on 9.0.10, but I can see that 9.0.11 is out so I'll upgrade soon (but I'm not too far behind).
Networking wise, the hosts communicate to each other and a NAS using MTU 9000 jumbo frames on a dedicated backend network. All other networks utilize more "standard" 1500 MTU (the inter-VLAN router uses MTU 1500).
Today, I ran into a slight snag with that SDN config: I'm adding a set of multi-homed containers (LXC) that have IPs both on the frontend 1500 MTU network and the hosts' backend 9000 MTU network. (if they work, they're going to serve as file gateways connected directly to Ceph at full 10G speed) When I set up the second NIC in the container, I got an error that 9000 was an invalid MTU. Researching that, I found this thread and its associated patch: https://forum.proxmox.com/threads/s...-enabled-for-inter-node-communication.143678/
Based on the thread and patch, I located the MTU config on the SDN zone, which was utilizing an MTU of 1500 by default and that was the source of my error message.
However, that patch (to my untrained eye) appears to only allow setting the MTU at the zone level, not at the VNET level. In my case, I have need of the latter - one or two backend VLANs that utilize a different MTU than the rest of the zone.
I know one way to fix this would be to change my whole SDN zone to 9000 MTU - but because I have a number of containers deployed now that I didn't have when I migrated that are all set to "use bridge default MTU", I'd rather not have to reconfigure them each by hand and set a potential configuration landmine for myself if I ever forget to change that from default on a container deploy.
The way I ended up working around this problem is to create a second SDN zone with a different MTU and re-assign the VNET that required jumbo frames to this zone.
So far, it's seemingly functional with the container able to run
I've got a workaround so I'm not "dead" in the water on this (so to speak) - but is this the "right"/expected way that this type of situation should be handled? Having to create two zones targeting the same OpenVSwitch bridge seems like a hack / band-aid solution to such a problem.
Networking wise, the hosts communicate to each other and a NAS using MTU 9000 jumbo frames on a dedicated backend network. All other networks utilize more "standard" 1500 MTU (the inter-VLAN router uses MTU 1500).
Today, I ran into a slight snag with that SDN config: I'm adding a set of multi-homed containers (LXC) that have IPs both on the frontend 1500 MTU network and the hosts' backend 9000 MTU network. (if they work, they're going to serve as file gateways connected directly to Ceph at full 10G speed) When I set up the second NIC in the container, I got an error that 9000 was an invalid MTU. Researching that, I found this thread and its associated patch: https://forum.proxmox.com/threads/s...-enabled-for-inter-node-communication.143678/
Based on the thread and patch, I located the MTU config on the SDN zone, which was utilizing an MTU of 1500 by default and that was the source of my error message.
However, that patch (to my untrained eye) appears to only allow setting the MTU at the zone level, not at the VNET level. In my case, I have need of the latter - one or two backend VLANs that utilize a different MTU than the rest of the zone.
I know one way to fix this would be to change my whole SDN zone to 9000 MTU - but because I have a number of containers deployed now that I didn't have when I migrated that are all set to "use bridge default MTU", I'd rather not have to reconfigure them each by hand and set a potential configuration landmine for myself if I ever forget to change that from default on a container deploy.
The way I ended up working around this problem is to create a second SDN zone with a different MTU and re-assign the VNET that required jumbo frames to this zone.
So far, it's seemingly functional with the container able to run
ping -s 8972 -c 2 -M do 10.77.1.129 to a jumbo frame target outside of my trio of hosts (the NAS)I've got a workaround so I'm not "dead" in the water on this (so to speak) - but is this the "right"/expected way that this type of situation should be handled? Having to create two zones targeting the same OpenVSwitch bridge seems like a hack / band-aid solution to such a problem.
