Does Proxmox support this feature (-list)?

[virt-cluster]

Hello all,

I'm curently evaluating a new virtualization environment and since I generally like to stick to OSS of course Proxmox is on the list.

I'm especially interested in the following features (order is prioritized, top prio comes first). I've also added examples which are contained in brackets.

1. Cluster updates without downtimes of VMs (update a 3-node HA-Cluster without any downtime for VMs)
2. The possibility to have tiered storage configurable via the GUI of Proxmox (SSD,SAS and NLSAS)
3. Pin VMs to certain storage tiers ( pinning a database VM to SSD tier, a regular VM to SAS tier and an archive system to NLSAS tier)
4. Expand single storage tiers (database VM outgrows SSD storage, so add more SSDs to Ceph)
5. Maintenance mode of a node (evacuating all VMs from the node being maintenanced)
6. "Hot-Add" of CPUs and Memory (Adding CPUs and RAM to the VM while it's running)
7. Node balancing with live-migration (Avoid situations where VM count per node is imbalanced)
8. Storage Tiering with automatic balancing (High I/O "portions" being moved from SAS to SSD tier)
9. (optional) Support of SR-IOV/IOMMU (Passing multiple graphics cards to a single VM, passing a single graphics card to multiple VMs)

My plan on building the infrastructure lies a little bit ahead in the future, so it's also interesting to me if an unavailable feature is on some roadmap or will be added soon. (~6 months to 1 year).

Thanks in advance.

 

[Alwin]

Cluster updates without downtimes of VMs (update a 3-node HA-Cluster without any downtime for VMs)

yes.

• The possibility to have tiered storage configurable via the GUI of Proxmox (SSD,SAS and NLSAS)

• Pin VMs to certain storage tiers ( pinning a database VM to SSD tier, a regular VM to SAS tier and an archive system to NLSAS tier)

• Expand single storage tiers (database VM outgrows SSD storage, so add more SSDs to Ceph)

Yes with some configuration. Ceph's device classes & pools + Proxmox storage config.

Maintenance mode of a node (evacuating all VMs from the node being maintenanced)

Migrate all with a click.

"Hot-Add" of CPUs and Memory (Adding CPUs and RAM to the VM while it's running)

If the VM's OS can handle it.

Node balancing with live-migration (Avoid situations where VM count per node is imbalanced)

Manually

Storage Tiering with automatic balancing (High I/O "portions" being moved from SAS to SSD tier)

Depends on the storage, Ceph has support for it.

(optional) Support of SR-IOV/IOMMU (Passing multiple graphics cards to a single VM, passing a single graphics card to multiple VMs)

yes.

https://pve.proxmox.com/wiki/Roadmap
https://www.proxmox.com/en/news/press-releases

 

[virt-cluster]

Okay, thats really nice so far. But I have a further question regarding the cluster update. I've read in this forum (2011) that it's not possible to update a cluster without downtime for the VMs.

IMO the update process is as follows: (assuming a two Node cluster here)
1. Migrate all VMs from Node A to Node B (maintenence)
2. Upgrade Node A
3. Migrate all VMs from Node B to Node A
4. Upgrade Node B
5. Rebalance cluster

Step 3 implies backwards compatibility for live migration in KVM and LXC, because you need to migrate VMs and containers from an outdated Node B to an updated Node A. In other virtualization environments this has been taken care of, but I've read many times that for KVM this was not the case.

Can you please explain this process in Proxmox a little deeper?

 

[Alwin]

Okay, thats really nice so far. But I have a further question regarding the cluster update. I've read in this forum (2011) that it's not possible to update a cluster without downtime for the VMs.

Well, a post from 8yrs ago. Things evolved quite a bit since then.

IMO the update process is as follows: (assuming a two Node cluster here)
1. Migrate all VMs from Node A to Node B (maintenence)
2. Upgrade Node A
3. Migrate all VMs from Node B to Node A
4. Upgrade Node B
5. Rebalance cluster

Yup, this is how we do it.

Step 3 implies backwards compatibility for live migration in KVM and LXC, because you need to migrate VMs and containers from an outdated Node B to an updated Node A. In other virtualization environments this has been taken care of, but I've read many times that for KVM this was not the case.

Forward migration is always possible and gets good testing, otherwise the no-downtime upgrades per se wouldn't be possible.

EDIT:

Our docs are also online available.
https://pve.proxmox.com/pve-docs/

And as its open source, you can just create three VMs and a Proxmox VE cluster + Ceph for testing.

 

[virt-cluster]

I've posted my questions after reading the docs, but i think it was v5.X. It wasn't that clear so I've asked the questions

There were just two new ones which came to my mind:
1. Is it supported to create a 3-node setup with 2 nodes being actual hypervisors and the third one being just a lightweight witness host for quorum with corosync configured? (with supported I mean out of the box and not by adding 3rd party stuff to the OS which might break upgrading)

2. According to this article it's possible to use kronosnet's built in feature to create a fallback NIC.

pvecm create CLUSTERNAME --link0 10.10.10.1,priority=20 --link1 10.20.20.1,priority=15

Which is the recommended way to realize this fallback? Using bonds and passing just the IP or using the built-in feature? Or is there no difference between them at all.

 

[Alwin]

1. Is it supported to create a 3-node setup with 2 nodes being actual hypervisors and the third one being just a lightweight witness host for quorum with corosync configured? (with supported I mean out of the box and not by adding 3rd party stuff to the OS which might break upgrading)

For corosync maybe, as it also allows a qdisk for quorum. But not for Ceph, it needs three MONs for quorum and they are all active and queried by all participating Ceph services. A "lightweight" (slow) witness will only slow down performance (most likely significantly).

2. According to this article it's possible to use kronosnet's built in feature to create a fallback NIC.
Which is the recommended way to realize this fallback? Using bonds and passing just the IP or using the built-in feature? Or is there no difference between them at all.

The closer HA is to the application layer the better. Corosync would not have a fallback if it only has one (or all) link on a bond and that bond doesn't pass traffic through anymore. TL;TR use independent links.