Proxmox HA with ceph slow migration after fail

[Cert Polska]

Hi

I'm testing HA using proxmox and ceph storage

This is my testing environment:

Latest Proxmox and Ceph

3 nodes each connected using 1 Gb/s network + 1 extra network for management

Each node has 3 2Tb hdd one for system and two as a ceph osd

On each node there are 5 VM's

If we shutdown one we need to wait 2,5 minutes until all migration finish, is it normal with this configuration?


Po production cluster we will have 10 Gb/s network and 10 ssd disks for each node.



Best regards,
Matthew

 

[Alwin]

3 nodes each connected using 1 Gb/s network + 1 extra network for management

I hope corosync is on a separate physical network interface.

If we shutdown one we need to wait 2,5 minutes until all migration finish, is it normal with this configuration?

It depends on size of your VMs/CTs and how much data ceph has to cope with. But you are saturating your 1GbE link for sure.

Po production cluster we will have 10 Gb/s network and 10 ssd disks for each node.

Well, see above, you will also be able to saturate the 10GbE with your SSDs. A separate network for corosync, ceph and client traffic is advised.

 

[Cert Polska]

Yes, corosync is on separated network interface

On production environment will be the same 10GbE on separated interface only for node replication


VMs for tests was 36 GB size