Managing Multiple Clusters

H

hypsin

Member
Jan 13, 2017
7
2
23
39
I am currently running a single 6-node Proxmox cluster and looking to add 3 more nodes. There is really no need for me to be running all 9 nodes in a single cluster so I thinking about split it up into 3 clusters x 3 nodes each.

Couple questions if anyone has experience running multi-cluster setups:

1. Will I be able to administer all 3 clusters from a single node GUI?
2. Is there any limitation on how many clusters/nodes can reside on the same network? Or would it be best to allocate a separate VLAN to each cluster?
3. Does Proxmox allow offline/online VM migration between different clusters with identical node hardware/layout?
 
  • Like
Reactions: Jarvar
1. I don't believe that you can. Each cluster would have a separate GUI.
2. I am sure you could put them on the same network, but I think it would be better to separate them out into their own network or VLAN.
3. I don't believe you can migrate VMs or containers between clusters.

I looked as doing a 3 and 3 node split on my setup but in the end, decided to do a 6 node cluster so that I could use 1 GUI and also move VMs and containers between all the nodes on the cluster. I have 3 different computer types/hardware setups in my cluster and I can move between all of them no problem in online and offline mode provided the allocated CPUs and RAM do not exceed the limits of the node I am moving to.
 
So, let's say I want to scale to 24 nodes with a dedicated 24-port gigabit switch for corosync, would a single cluster be able to support it or would corosync still have a full meltdown?

It would be neat to be able to create "quorum groups", each with its own corosync domain, while still being part of the same cluster for ease of management. But not sure if that's possible with the way how pmxcfs works.
 
At proxmox system, the cluster is exactly the host itself, it is mean cluster managign by all host.. But you can use diffrent ethernet for guest migration I think this feature was come with 6.**, for cluster system you really have very big host group it is like professional DC services... I think 24 host management can not be easy with proxmox standart system; so what you can ?

1. You can use oVirt instead of Proxmox, also oVirt better than proxmox for professional services... For little cluster for SOHO or SMB or for learning LAB, for Home LAB best of best Proxmox but wihtout Centrall managemenet system for cluster group can not be good for professional services...

2. if you are programmer or you can pay for centrall management software. Proxmox also API solition, like this " https://pve.proxmox.com/pve-docs/api-viewer/ " or "https://pve.proxmox.com/wiki/Proxmox_VE_API "
 
Custom VM migration between clusters seems to be trivial and can probably be wrapped in a very simple shell script (SSH config/disks over).

But lack of centralized management for a large cluster or multiple small clusters is a bummer. If it bugs me too much I may indeed hook into API and write my own front end, but that's extra work I just don't have time for.

So, is the general consensus that Proxmox is for small business and hobby deployments and does not scale well for production use? Because you've could have fooled me - I've been running my 6 node cluster with ton of VMs (web hosting) for over a year now with literally 0 issues.
 
Hi,

If I remember I think that multi-data-center option is on the devs list. Many users had ask this option for many times.
As I see on others users post, some guys use several different clusters without problems. Yes live-migration between different cluster could not work. But with zfs storage, you can migrate(offline) any VM/CT with a small bash script.

Good luck!
 
  • Like
Reactions: Jarvar
hypsin said:
Custom VM migration between clusters seems to be trivial and can probably be wrapped in a very simple shell script (SSH config/disks over).

But lack of centralized management for a large cluster or multiple small clusters is a bummer. If it bugs me too much I may indeed hook into API and write my own front end, but that's extra work I just don't have time for.

So, is the general consensus that Proxmox is for small business and hobby deployments and does not scale well for production use? Because you've could have fooled me - I've been running my 6 node cluster with ton of VMs (web hosting) for over a year now with literally 0 issues.
Click to expand...


If you have more time for manage HOST and GUEST whay not ?

For your purpose, best is management sofware for HOST and cluster... Also Management software not means only manage HOST / CLUSTER / GUEST; management software also means DRS, sDRS or similar system for easly total system source use without planing... Also proxmox have many diffrent storage soliton but can not spport SAN Shared storage system direcly, so SAN means IO king but PROXMOX have hyperconverged with local disk ( ZFS,LVM ), glusterFS, CEPH or remote system with Ethernet Based ISCSI, Gluster, RDB ( Ceph ) or Soft Link ZFS,NFS so can not create big IO like SAN.. also you tell to me I was manage my web server, web server means low BW high IO like DB what you use on storage side how you can support that IO request without SAN ...?

Brother I was work on this market more than 20 years and please do not tell to me Proxmox ok for DC management. I was manage more than hundreds Virtualization HOST and thousands thousands diffrent type GUEST with another virtualization software... If proxmox want enter enterprise virtualization market must be need centrall management ( Free or licanced... ) and need automize source management system ( Free or Licanced ) also must be fully support SAN Storage system
 
  • Like
Reactions: Jarvar
ertanerbek said:
also you tell to me I was manage my web server, web server means low BW high IO like DB what you use on storage side how you can support that IO request without SAN ...?
Click to expand...

1. Reverse Proxy your webservers so you don't have to render or even hit storage with every request.
2. 4x SAS SSD in local RAID10 is more than enough to keep up with dozens of webserver VMs on same node.
3. DB - cache what you can, leverage columnstore compression and tune your indexes to keep DBMS IO down. If you need dedicated SAN to host webserver back-end then maybe overall architecture is the bottleneck, not storage.
 
I talking about 12~20 gigabit real time traffic... moret han 14K session on only one country based TV Page.. I was survive that structure with dual A10 LB ...
 
So, back to my question - can Proxmox 6.x easily handle 24 nodes on a single cluster without too much fiddling with corosync?
 
https://pve.proxmox.com/wiki/Cluster_Manager

You can grow up your cluster until 32 Node.. For management ? What you want from Proxmox ? What is your requiriment or expentations and what is you storage infrastructure, after all node menas CPU adn RAM and KVM easly transfer memory from network but disk not like that ...
 
Hi all!

After those years, do we have some improvements in that topic? Especially when Broadcom is highly overkilling the VMware prices for small business and removes small vendors from the market?

There will be no better time for taking the most of the virtualization world by the Proxmox. Really!

Tighten up your forces and accelerate your work hardly.
 
Last edited:
LnxBil said:
What small business does need multiple PVE clusters and a single management platform?
Click to expand...

Any which needs two types of environments: production and non-production.
And would like to manage it via one web UI or migrate some VMs from prod to non-prod and vice versa.
Any which has developer's team onsite and make software for own purposes or maintain any, ERP for instance.
 
patefoniq said:
Any which needs two types of environments: production and non-production.
Click to expand...
We have those and some more environments (restore test, pre-production, debugging)... all in one UI.

patefoniq said:
migrate some VMs from prod to non-prod and vice versa.
Click to expand...
Already possible.

patefoniq said:
Any which has developer's team onsite and make software for own purposes or maintain any, ERP for instance.
Click to expand...
We have that too ... You can have different pools with different permissions, own network segments and everything else that has been around for 40+ years ... I really don't get your point, PVE can do everything you need.
 
LnxBil said:
What small business does need multiple PVE clusters and a single management platform?
Click to expand...
It's not limited to SMB. This could benefit enterprises as well as they deploy more and more clusters potentially. The "versions" of clusters could have MANY different environments.
 
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom