Bridge network setup -> very low MTU from VMs.. why?

N

nanankcornering

Member
Feb 23, 2022
8
1
8
28
Hi,

Running a proxmox setup where all VMs are bridged to a single physical interface in host, all with default MTU of 1500.

We noticed that websocket connections are closing abruptly, and running the following bash script:

Bash: 
size=1272
while ping -s $size -c1 -M do google.com >&/dev/null; do
  ((size+=4))
done
echo "Max MTU size: $((size-4+28))"

Showed an MTU of 1296 in VMs, while running in host shows 1500.

Any ideas why? Have limited understanding in MTU/MSS, but 1296 is quite low..

ifconfig on host shows everything on 1500:
Code: 
enp152s0f0np0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        ether 3c:ec:ef:b7:fa:8c  txqueuelen 40000  (Ethernet)
        RX packets 13086020619  bytes 10072373360184 (9.1 TiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 16793858705  bytes 21713322348584 (19.7 TiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

fwbr100i0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        ether 52:fa:5f:9f:6f:e1  txqueuelen 1000  (Ethernet)
        RX packets 99617  bytes 7196620 (6.8 MiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

fwln100i0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        ether 52:fa:5f:9f:6f:e1  txqueuelen 1000  (Ethernet)
        RX packets 6574036135  bytes 4458682824606 (4.0 TiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 4943211571  bytes 17199642687289 (15.6 TiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

fwpr100p0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        ether a2:29:04:b5:16:07  txqueuelen 1000  (Ethernet)
        RX packets 4943211570  bytes 17199642687223 (15.6 TiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 6574036133  bytes 4458682820130 (4.0 TiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 1000  (Local Loopback)
        RX packets 572662  bytes 70686791 (67.4 MiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 572662  bytes 70686791 (67.4 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

tailscale0: flags=4305<UP,POINTOPOINT,RUNNING,NOARP,MULTICAST>  mtu 1280
        inet 100.93.48.90  netmask 255.255.255.255  destination 100.93.48.90
        inet6 fd7a:115c:a1e0::501:305a  prefixlen 128  scopeid 0x0<global>
        inet6 fe80::d98d:6ec5:2612:593b  prefixlen 64  scopeid 0x20<link>
        unspec 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00  txqueuelen 500  (UNSPEC)
        RX packets 656845  bytes 110539025 (105.4 MiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 543198  bytes 426247653 (406.5 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

tap100i0: flags=4419<UP,BROADCAST,RUNNING,PROMISC,MULTICAST>  mtu 1500
        ether 46:45:f4:6f:6f:df  txqueuelen 1000  (Ethernet)
        RX packets 4943216384  bytes 17199643015080 (15.6 TiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 6574078516  bytes 4458684523783 (4.0 TiB)
        TX errors 0  dropped 890 overruns 0  carrier 0  collisions 0

tap101i0: flags=4419<UP,BROADCAST,RUNNING,PROMISC,MULTICAST>  mtu 1500
        ether ce:38:e0:2c:7f:53  txqueuelen 1000  (Ethernet)
        RX packets 603091249  bytes 454414676344 (423.2 GiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 1003225136  bytes 1944134992270 (1.7 TiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

tap102i0: flags=4419<UP,BROADCAST,RUNNING,PROMISC,MULTICAST>  mtu 1500
        ether 6e:f9:c3:7a:7e:21  txqueuelen 1000  (Ethernet)
        RX packets 72500042  bytes 122667409552 (114.2 GiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 108024949  bytes 124454557216 (115.9 GiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

tap103i0: flags=4419<UP,BROADCAST,RUNNING,PROMISC,MULTICAST>  mtu 1500
        ether 4e:c8:6e:2c:5d:17  txqueuelen 1000  (Ethernet)
        RX packets 58501  bytes 4549026 (4.3 MiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 80886  bytes 364295030 (347.4 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

vmbr0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 1.1.1.1  netmask 255.255.255.248  broadcast 0.0.0.0
        inet6 fe80::3eec:efff:feb7:fa8c  prefixlen 64  scopeid 0x20<link>
        ether 3c:ec:ef:b7:fa:8c  txqueuelen 1000  (Ethernet)
        RX packets 6126418  bytes 9702077777 (9.0 GiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 4866497  bytes 9877701568 (9.1 GiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

same thing with VM:
Code: 
enp6s18: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 103.107.198.45  netmask 255.255.255.248  broadcast 103.107.198.47
        inet6 fe80::be24:11ff:fe38:5f5c  prefixlen 64  scopeid 0x20<link>
        ether bc:24:11:38:5f:5c  txqueuelen 1000  (Ethernet)
        RX packets 110679489  bytes 127374723957 (127.3 GB)
        RX errors 0  dropped 1155  overruns 0  frame 0
        TX packets 74351454  bytes 125550170072 (125.5 GB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 1000  (Local Loopback)
        RX packets 432  bytes 36844 (36.8 KB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 432  bytes 36844 (36.8 KB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

tailscale0: flags=4305<UP,POINTOPOINT,RUNNING,NOARP,MULTICAST>  mtu 1280
        inet 100.96.250.21  netmask 255.255.255.255  destination 100.96.250.21
        inet6 fd7a:115c:a1e0::7101:fa15  prefixlen 128  scopeid 0x0<global>
        inet6 fe80::b0f7:5995:a771:74c1  prefixlen 64  scopeid 0x20<link>
        unspec 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00  txqueuelen 500  (UNSPEC)
        RX packets 2809  bytes 244135 (244.1 KB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 2095  bytes 3034235 (3.0 MB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
 
ok so apparently the ping process is not processing incoming ICMP replies.. that's why mtu of 1296 is returned from the script.

x > 8.8.8.8: ICMP echo request, id 9, seq 1, length 1480
8.8.8.8 > x: ICMP echo reply, id 9, seq 1, length 76
x > 8.8.8.8: ICMP echo request, id 9, seq 2, length 1480
8.8.8.8 > x: ICMP echo reply, id 9, seq 2, length 76
x > 8.8.8.8: ICMP echo request, id 9, seq 3, length 1480
8.8.8.8 > x: ICMP echo reply, id 9, seq 3, length 76

Code: 
root@test:~# ping -A -c 3 -M do -s 1472 8.8.8.8
PING 8.8.8.8 (8.8.8.8) 1472(1500) bytes of data.

--- 8.8.8.8 ping statistics ---
3 packets transmitted, 0 received, 100% packet loss, time 2003ms

anyone knows why?
 
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom
Back