LACP Bond interface with ceph on VLAN interface

[billy12347]

Hi all, fairly new to Proxmox, looking for some info and couldn't find an existing thread related to this.

I've got 3 servers in a cluster with 2x 25G interfaces each, current setup is one port on each server is for management and guest traffic, and the other is a 25G backbone with jumbo frames enabled on a VLAN exclusively for Ceph traffic.

I was thinking of swapping this for a LACP bond, and setting the management and Ceph interfaces to a VLAN interface in the bond. My question is if I set the bond and Ceph interface to have an MTU of 9000, would the management interface also end up with an MTU of 9000, causing fragmentation issues, or can I manually set the VLAN interface in the bond to not be at the same MTU as the bond it's in?


Update for anyone who finds this and has the same question, I ended up just doing it and it seems to work fine. Haven't run a full range of tests yet, but it's running well so far.

 

[Alwin Antreich]

Just in case, please separate storage from workload traffic and in particular from corosync. During recovery Ceph may fully utilize the link which will have problematic effects if HA is enabled (node reset).

 

[billy12347]

Just in case, please separate storage from workload traffic and in particular from corosync. During recovery Ceph may fully utilize the link which will have problematic effects if HA is enabled (node reset).

This is good advice, my setup is just a test environment I'm using to get used to Proxmox, and currently I've only got 1 PCIe3 x4 drive in each box, so worst case I should have 10G left over on the bonded 50G link for other traffic if the drives are going full speed. If this were in a prod environment I would definitely have a 100G backbone.