Setting MTU to 9000 blocks access to node, but node can ping

[UntouchedWagons]

I've got a test machine with a 10 gig NIC and two 1 gig NICs. The 10 gig NIC is connected to a 10 gig interface on my switch and one of the gigabit NICs is also connected to the same switch. As part of my testing I have set up these two NICs in an active-backup bond like so:

    auto lo

    iface lo inet loopback

    auto enp1s0
    iface enp1s0 inet manual

    auto eno1
    iface eno1 inet manual

    iface eno2 inet manual

    auto bond0
    iface bond0 inet manual
            bond-slaves eno1 enp1s0
            bond-miimon 100
            bond-mode active-backup
            bond-primary enp1s0

    auto vmbr0
    iface vmbr0 inet static
            address 192.168.0.37/24
            gateway 192.168.0.1
            bridge-ports bond0
            bridge-stp off
            bridge-fd 0
            bridge-vlan-aware yes
            bridge-vids 2-4094

This works perfectly fine, I can SSH in, access the web UI, the whole nine yards. But if I go into the network tab and set the MTU for all the interfaces (physical and virtual) to 9000 and click Apply Configuration I can no longer SSH in or access the web UI. If I remote in using IPMI the node can ping google just fine. If I remove the MTU directives and reboot everything is fine again.

Anyone have any idea why this is happening?

 

[gurubert]

When you want to increase the MTU all stations in that Ethernet segment and all switches have to be configured and support the larger MTU.
You cannot just increase the MTU on one host.

 

[sg90]

Ping by default wont be using the MTU9000 hence why it still works.

Your need to enable MTU 9000/Jumbo Frames on the switch config also.

 

[UntouchedWagons]

My switch does support jumbo frames, I just can't find the setting for it in the Omada controller.

 

[vesalius]

Jumbo frames are typically more trouble than they are worth particularly at 10g or less. Typically only bother with them for storage only networks.

jumbo frames intermittently and seemingly randomly can screw up some normal background network traffic, from mDNS to ssh and others. Switches and network devices handle it differently.

 

[mr44er]

My switch does support jumbo frames, I just can't find the setting for it in the Omada controller.

As already said, every connected device in the chain has to have mtu9000 set. Bonds, bridges,ports,switches...everything.
Bad case: sometimes you can set mtu9000, but the port or chip doesn't support it. You'll get no error and no connection.