So Far So Good

[Vedmia]

Having converted most of my VM’s from xcp-ng ( from citrix) to proxmox I will share my observations

proxmox boots faster
proxmox is debian based, the OS of choice
proxmox gui is more logical
proxmox is not as tied down, for example I have not lost any bits I have added between updates
hopefully this will continue

However ( (if I am wrong then please correct me)
proxmox migration without a cluster does not seem possible other than pve - pbs - pve
proxmox clusters are a bit of a challenge to the two hypervisor setup , development and production, with out having
a third pseudo device to keep the quorum

Conversion from xcp-ng was a challenge as there is no OVA facility, so it was VDK to cow2 then qm import.
(The multi disk 2T vm was a squeeze, but got there in the end)

Managed to get a VM pbs on TruNas

Thanks

Aimee

 

[Dunuin]

Even a 12$ Raspberry Pi Zero 2 W with only 1-2W power consumption could act as a third voter/qdevice. Not that I would recommand it because of missing reliability, but atleast it wouldn't be expensive.

 

[Vedmia]

Hi
If I had two production machines and one developmnt. then I would go that solution as a production cluster. I only have two because I asses software on the development then transfer it to the production for user assesment. My only issue is the migration is slightly easier with xcp-ng as there is no need for a cluster, it uses the concept of a pool, which is easy to disconnect from without any disruption to other members.
However its not a big issue, the pve pbs-pve works as most VM's are < 128GB (apart a couple of huge ones!!)

 

[bbgeek17]

@aimdev I think if you dive deeper into either xcp-ng or pve, you will realize they are not so different.

Quiet simply pool=cluster: https://xcp-ng.org/docs/ha.html

WARNING

Even if you can have HA with only 2 hosts, it's strongly recommended to do it with at least 3 hosts, for obvious split-brains issues you might encounter.

The second warning on that page is also very true...

The pool concept allows hosts to exchange their data and status:

if you lose a host, it will be detected by the pool master.
if you lose the master, another host will take over the master role automatically.

Thats literally what a cluster does...

---

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox

 

[Vedmia]

@aimdev I think if you dive deeper into either xcp-ng or pve, you will realize they are not so different.

Quiet simply pool=cluster: https://xcp-ng.org/docs/ha.html

WARNING

Even if you can have HA with only 2 hosts, it's strongly recommended to do it with at least 3 hosts, for obvious split-brains issues you might encounter.

The second warning on that page is also very true...

The pool concept allows hosts to exchange their data and status:

if you lose a host, it will be detected by the pool master.
if you lose the master, another host will take over the master role automatically.

Thats literally what a cluster does...

---

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox

Thanks for your reply

Please can you tell me how I can migrate a vm from one hypervisor to another without
requiring a cluster or pve - pbs - pve

A bit like linux mv

 

[bbgeek17]

You cant. What PVE does not provide is a sleek global cluster manager that sits above PVE clusters, like the Orchestrator sits above the pools.
There is a feature request for it with many people asking to add it. But its unlikely to be on a short list to be done.

my point is:

proxmox clusters are a bit of a challenge to the two hypervisor setup , development and production, with out having
a third pseudo device to keep the quorum

This ^ is absolutely identical between two platforms. Both can run in a pool/cluster with two nodes, both strongly recommend 3. This is the same for practically any software cluster.

My only issue is the migration is slightly easier with xcp-ng as there is no need for a cluster, it uses the concept of a pool

Here ^ you are comparing different technologies. PVE cluster is an XCP-ng pool. What XCP-ng has is an orchestrator VM that is a multi-pool manager.
PVE does not have it https://forum.proxmox.com/threads/managing-multiple-clusters.56416/

---

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox

 

[Vedmia]

You cant. What PVE does not provide is a sleek global cluster manager that sits above PVE clusters, like the Orchestrator sits above the pools.
There is a feature request for it with many people asking to add it. But its unlikely to be on a short list to be done.

my point is:

This ^ is absolutely identical between two platforms. Both can run in a pool/cluster with two nodes, both strongly recommend 3. This is the same for practically any software cluster.


Here ^ you are comparing different technologies. PVE cluster is an XCP-ng pool. What XCP-ng has is an orchestrator VM that is a multi-pool manager.
PVE does not have it https://forum.proxmox.com/threads/managing-multiple-clusters.56416/

---

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox

You cant. What PVE does not provide is a sleek global cluster manager that sits above PVE clusters, like the Orchestrator sits above the pools.
There is a feature request for it with many people asking to add it. But its unlikely to be on a short list to be done.


I rest my case.

Thanks

 

[bbgeek17]

However ( (if I am wrong then please correct me)

I misread the original post , it seemed like it was looking for a feedback on technological aspects.

 

[Vedmia]

No. I was stating my experience with proxmox, see the , and asking if there was a VM migration path that was easier than the pve-pbs-pve route.
It appears there is not an easy one click migration procedure, I don't see this, considering all the other positive aspects of proxmox I have identified, as a major issue, it would be desirable in my (and probably others) environments. In my environment cluster is an impediment, and I am not going to purchase a piece of hardware to overcome it. I have tried to compile the Qdevice using FreeBSD, where it could be added to the router, but not succeed yet. I have other tasks to carry out, external to proxmox so the Qdevice/FreeBSD solution will have to wait.

 

[Dunuin]

I also would like to see some way to migrate between nodes without doing a PVE -> PBS -> PVE. Creating a cluster and using offline/onlihe migration won't work here as I use ZFS native encryption and migrating VMs between ZFS pools on different nodes will rely on ZFS replication and this is not compatible with encrypted datasets/zvols. So even if I would create a cluster, my only option to move guests between clustered nodes would be the PVE -> PBS -> PVE route.