Risks of changing CEPH MTU to 9200

[Evgeniy Zakharov]

Hello,

I want to change MTU to 9200 in Proxmox cluster with CEPH working on dedicated network interfaces with default MTU 1500.
Do I need to do this node by node? Will individual nodes be unavailable until all nodes are set to 9200?
Is there a high risk of a CEPH collapse?
Can a 'noout' flag help?

Perhaps someone already has such experience.

 

[alexskysilk]

I want to change MTU to 9200 in Proxmox cluster with CEPH working on dedicated network interfaces with default MTU 1500.

you cant have mixed mtu on an interface. If you mean you want to CHANGE the interface mtu, just do it. Changing from 1.5 to 9k will work, even if it may result in reduced performance until ALL nodes switch to 9k; the reverse is problematic.

 

[mr44er]

Is there a high risk of a CEPH collapse?

Jup.
Don't do this within a small cluster! Just don't! because:

Will individual nodes be unavailable until all nodes are set to 9200?

YES!
And if a single device in between cannot speak MTU9200 ceph is likely offline.


If you ever need replacement parts (Switches/NICs) and those cannot speak MTU9200 (even more exotic than MTU9000) this will be more problematic.