Connectivity issues with multiple VLANs

matius

New Member
Nov 29, 2024
6
1
3
Hello,

I'm running a Ubuntu Virtual Machine on a Proxmox node and have been having experiences some strange behaviours when using multiple NICs with different VLANS.
When loading a website it will take 5-11s to load even though the server is locally hosted running nginx. It appears the time taken is related to TCP? I'm not sure what this means.
Another symptom is when connecting to the SSH it will also take a similar time 5-11s to establish a connection.
Strangely this behaviour only occurs when client is on VLAN 14 connecting to server's IP on VLAN 10. These issues are non-existent when communicating within the same VLAN client to server. I have scoured the entire internet and tried everything from ensuring only a default route is set to capturing traffic via tcpdump and trying to troubleshoot Asymmetric routing but I have failed to spot anything that might be causing this, let alone resolve it.
For some perspective the Virtual Machines is attached to vmbr10 with bridge port VLAN 10, vmbr14 with bridge port VLAN 13, vmbr13 with bridge port VLAN 13. I have also tried using a single vmbr with Vlan Aware enabled and configuring VLAN Tag when adding the interface to the VM.

The reason server's IP on VLAN 10 is being accessed by all other VLANs is due to it being used for DNS records. The only purpose of having other VLANs attached as seperate interfaces are for mDNS. I understand that this may be broader issue potentially more Linux / Networking rather than Proxmox but if you're a smart person please help me out with this one!
The client and server are directly connected to a Mikrotik RB5009 which has all VLANs in a bridge allowing for inter-VLAN routing, I doubt that this issue is related to the router but I thought it was worth mentioning anyway.
Screenshot 2025-04-19 at 16.18.35.pngScreenshot 2025-04-19 at 16.30.32.pngScreenshot 2025-04-19 at 16.30.42.png
 
After many many hours of research I managed to resolve the issue. Not at all Proxmox related but maybe this will help someone in the future.
Due to having the server available from both VLAN 14 and VLAN 10 network interfaces traffic to came in through VLAN 10 when accessed by its IP and exited through VLAN 14. This caused TCP retransmissions resulting in the 5-11s delay as ACK packets were exchanged until they somehow exited the correct network interface.
I created a Custom Route Table to direct traffic from VLAN 10 to exit through the correct interface and gateway.
 
  • Like
Reactions: SteveITS
Your set up seems very convoluted/complex to me. Why are you creating these VLANs and Bridges on Proxmox if they are already configured on your router? I have 6 VLANs in use here and I only use one bridge to access them all (vmbr0). vmbr0.100 is the web interface for Proxmox. My connection to the switch is a pair of NICs in a link aggregation bond. But leaving that aside, I did the same thing when I only had a single NIC, just minus the bond. I configure the VLAN in the network hardware section of the VM. Do you have VMs that need to reside in more than one VLAN?

1745070091426.png

1745070246713.png
 
Hi louie
It's funny you should say that it is convoluted, I thought so as well. It was purely for debugging purposes, I am currently running a single Vlan aware bridge (the same setup as you).
Does anyone know what the Vlan aware tick box actually does? It seems that Vlan tagging works regardless whether it is enabled or disabled.
 
VLAN aware means it passes tagged traffic (i.e., packets with a VLAN tag. It also means you need to connect that device to a trunk port on your switch and not an access port in order for more than one VLAN's traffic to traverse it.


Here's what my /etc/network/interfaces file looks like

root@pve-01:~# cat /etc/network/interfaces
auto lo
iface lo inet loopback

iface enp1s0f0 inet manual

iface enp1s0f1 inet manual

auto bond0
iface bond0 inet static
bond-slaves enp1s0f0 enp1s0f1
bond-mode 802.3ad
bond-miimon 100
bond-lacp-rate fast
bond-xmit-hash-policy layer3+4

auto vmbr0
iface vmbr0 inet static
bridge-ports bond0
bridge-stp off
bridge-fd 0
bridge-vlan-aware yes
bridge-vids 2-4092

auto vmbr0.100
iface vmbr0.100 inet static
address 10.10.10.2/24
gateway 10.10.10.1

iface wlp5s0 inet manual