Load balancing and future features

[spirit]

spirit are you developing a feature for integration into Proxmox itself?

yes

I have send prelimary patches on the pve-devel mailing

https://lists.proxmox.com/pipermail/pve-devel/2022-April/052780.html

It's almost working, I need to add more tests, do tests at scale,
and first I need to add new cpu,mem pressure metrics in pvestatd.

https://www.mail-archive.com/pve-devel@lists.proxmox.com/msg08781.html

 

[Stefan_Malte_Schumacher]

This is exactly what I am looking for. I think that balancing for RAM makes most sense. In an HA-Environment Storage should always be accessible or its simply not HA. It is very easy to oversubscribe CPU (most of my VMs are idling their time away, only a few large websites really create load and that is still low 10s as compared to the entire node) but you can't duplicate RAM. The mass migration is also a good idea. Ideally the tool would save the current state and restore it after the reboot or upgrade of the server. (This is what I am going to do after dinner. Migrate everything thats on the storage to other machines, reboot the node and move them back manually. Would be great to have a GUI for it)

 

[spirit]

This is exactly what I am looking for. I think that balancing for RAM makes most sense. In an HA-Environment Storage should always be accessible or its simply not HA. It is very easy to oversubscribe CPU (most of my VMs are idling their time away, only a few large websites really create load and that is still low 10s as compared to the entire node) but you can't duplicate RAM. The mass migration is also a good idea. Ideally the tool would save the current state and restore it after the reboot or upgrade of the server. (This is what I am going to do after dinner. Migrate everything thats on the storage to other machines, reboot the node and move them back manually. Would be great to have a GUI for it)

I'm planning to add options to choose if you want balance only ram, cpu, both, storage, .. Everybody have different workload with different need. (for example, personnaly, I can overscribe a little bit the cpu, but not too much, or I'll have performance impact on the vms).

I'll try to finish the code this summer.

 

[xiaolin0199]

I'm planning to add options to choose if you want balance only ram, cpu, both, storage, .. Everybody have different workload with different need. (for example, personnaly, I can overscribe a little bit the cpu, but not too much, or I'll have performance impact on the vms).

I'll try to finish the code this summer.

Great feature, is it finished now?

 

[spirit]

Great feature, is it finished now?

no sorry.
Proxmox team woud like to rework on my code, write some part of algo in rust,...
So,I think It'll take a little bit more time.

 

[cvk98]

What language did you write your code in?

 

[spirit]

What language did you write your code in?

perl, as it's need to be integrated in current proxmox code.

 

[Elfy]

Thanks for your hard work on this @spirit I'll definitely be keeping an eye on this.

 

[spirit]

Thanks for your hard work on this @spirit I'll definitely be keeping an eye on this.

I'll try to release some deb on my own repo, and try to maintain them until we have an official solution.

I'll send a post on the forum when I'll ready. (I still need some missing cpu/mem pressure stats from pvestatd, but I think Ill not wait to get them upstream, as I really need this feature for my customers)

But my longterm plain is to get it upstream, as I don't like to maintain my own packages

 

[gulas2vladimir]

I'll try to release some deb on my own repo, and try to maintain them until we have an official solution.

I'll send a post on the forum when I'll ready. (I still need some missing cpu/mem pressure stats from pvestatd, but I think Ill not wait to get them upstream, as I really need this feature for my customers)

But my longterm plain is to get it upstream, as I don't like to maintain my own packages

Is it finished now?

 

[CLE]

I'd hope so Although the last reply by @spirit is almost 15 months ago, it seems that this wasn't fully done / implemented yet?