HA + performance for cluster and migration

T

Tipenso

Member
May 12, 2022
15
2
8
Good morning to all.

I have 4 network cards to dedicate to cluster and migration, VM network uses already 2 separate LACP ports.
Conceptually speaking, it would occur to me that a best practice to combine HA and performance, might be to use 2 ports in lacp for migration, and 2 separate networks for cluster + cluster backup, as latency is very important for the latter.
Am I in the right?

Thank you.
 
Lacp for redundancy, but not for Performance.
The only cool thing that you can do is disabling encryption for the Cluster/Migration Network. That makes migration at least twice as fast, but i would do it only if you make a separate closed vlan on the switches. (So that its still safe)

I have a Separate "Backup" Network/Vlan either between my pve servers and pbs.
But there is at the moment nothing you can do to speed things up, the Backup speed limit is at around 900mb-1GB/s due to something in the PBS Backup Pipeline. I think its the encryption that limits the speed between PVE/PBS.
Here is the Bug-Report: https://bugzilla.proxmox.com/show_bug.cgi?id=5481

But i think to be honest, that there are not really "best practice's", just your own preferences.
I have Clusters with dedicated vlans/network interfaces/back2back connections/nothing dedicated and encrypted/not encrypted etc...
Everything works perfectly fine without issues on all clusters, since forever.

I would see it from another point maybe:
- If you have the possibility, to make a dedicated network for migration and backups, do it.
- Migration-Speed can benefit on a dedicated network if you disable encryption.
- Backup-Speed cant be improved above 1GB/s, but below that limit its only a Disk Read on PVE and Disk Write/NVME on PBS limit.
- The only best practice is not to disable encryption in a network that is reachable by everything :-)

Cheers :-)
 
  • Like
Reactions: Johannes S
Ramalama said:
Lacp for redundancy, but not for Performance.
The only cool thing that you can do is disabling encryption for the Cluster/Migration Network. That makes migration at least twice as fast, but i would do it only if you make a separate closed vlan on the switches. (So that its still safe)

I have a Separate "Backup" Network/Vlan either between my pve servers and pbs.
But there is at the moment nothing you can do to speed things up, the Backup speed limit is at around 900mb-1GB/s due to something in the PBS Backup Pipeline. I think its the encryption that limits the speed between PVE/PBS.
Here is the Bug-Report: https://bugzilla.proxmox.com/show_bug.cgi?id=5481

But i think to be honest, that there are not really "best practice's", just your own preferences.
I have Clusters with dedicated vlans/network interfaces/back2back connections/nothing dedicated and encrypted/not encrypted etc...
Everything works perfectly fine without issues on all clusters, since forever.

I would see it from another point maybe:
- If you have the possibility, to make a dedicated network for migration and backups, do it.
- Migration-Speed can benefit on a dedicated network if you disable encryption.
- Backup-Speed cant be improved above 1GB/s, but below that limit its only a Disk Read on PVE and Disk Write/NVME on PBS limit.
- The only best practice is not to disable encryption in a network that is reachable by everything :-)

Cheers :-)
Click to expand...
Hi Ramalama, thanks for answering.

I knew about the encryption. However, it is my understanding that the migration of multiple VMs also gains in performance with LACP, since it can thus leverage multiple streams in parallel.
Is that not the case?
 
Tipenso said:
Hi Ramalama, thanks for answering.

I knew about the encryption. However, it is my understanding that the migration of multiple VMs also gains in performance with LACP, since it can thus leverage multiple streams in parallel.
Is that not the case?
Click to expand...

Yeah it does only in one case, if you migrate multiple VM's/Containers at the same time.
But in real life thats almost never the case (at least i never use that), i usually migrate only one vm.

If you Shutdown one Server from a Cluster that is in HA, it does migrate only one VM at once either, and never multiple at the same time...
 
Ramalama said:
Yeah it does only in one case, if you migrate multiple VM's/Containers at the same time.
But in real life thats almost never the case (at least i never use that), i usually migrate only one vm.

If you Shutdown one Server from a Cluster that is in HA, it does migrate only one VM at once either, and never multiple at the same time...
Click to expand...
Actually in my case, I migrate VMs mostly when I have to do maintenance on the hosts. So I find myself more often migrating all the VMs from one node to another.
Really if I shut down a node in HA, does Proxmox migrate only one VM at a time? I didn't know that, and it seems like a big limitation....
 
You must log in or register to reply here.
Top Bottom