PVE VM replication standalone host to 3 node PVE cluster with CEPH shared storage

J

Jera92

Active Member
May 14, 2019
28
0
41
34
Dear members of the Proxmox forum

I'm researching the possibilities of replication of a VM on a standalone Proxmox Virtual Environment Host to a 3 node PVE cluster with CEPH.
For legal reasons, we have to run a VM on a separate physical server, but due to the current server hardware prices, we can't afford buying 3 servers right now, next to our existing 3 servers in the cluster.

My question is:
What 's the best option to sync/replicate a VM's storage and VM config file from the standalone PVE host to the PVE cluster?
The main goal is that the VM is running on the standalone PVE host, and only when the standalone host is failing, I manually start the VM on the PVE cluster.
So I have the time to investigate and fix the standalone host.
There is a database running on the VM that may not lose any written transaction onto it before the standalone PVE host is getting unavailable.

The PVE CEPH cluster is using a CEPH meshed network on 10 Gbps links, without using a SAN switch.

Options I'm thinking about:
  • Adding the CEPH cluster storage to the standalone node and place VM-disks on the 3 node PVE CEPH cluster (so storage of the VM is over the network)
    And then the other way around, replicate the VM disk of the CEPH cluster back to the standalone hosts, although it's not used on the VM, except when the standalone host can't reach the CEPH cluster.
  • Store the VM disks on the standalone host and run a constant replication to the PVE cluster with CEPH.
    • I'm not sure of Proxmox Backup Server is able to sync for example every minute or every 5 minutes (to reduce the amount of bandwidth used while syncing)
    • Or using Veeam Backup and Replication to replicate the VM every minute or every 5 minutes
I haven't made a decision yet about using ZFS or LVM on the standalone PVE host, might prefer ZFS.

I hope with all the knowledge on this forum, someone can inform me better and provide me the best approach on how to handle this correctly, stable and as fast as possible :)

Can you help me further investigate this?

Kind regards
 
I think I would start with suggesting the easiest answer first. Just add the dedicated host for this VM to the cluster, and use HA Affinity Rules to keep the rest of the VMs in the cluster off of this node, and keep this VM off the other nodes as long as this node is available. This will let you use the ceph storage easily, and easily move to one of the other hosts when needed. Seems like the easiest option, if it works for your situation.
 
Hello @tomservo and @Francois Baillargeon
Thanks for your reply!

Well, indeed adding the host to the CEPH cluster is indeed the easiest, but I'm afraid for legal reasons, they won't believe that the VM is actually running separately on a dedicated physical host.

Another consideration I have:
The 3 x PVE nodes in cluster with CEPH have 6 disks of 1.92TB SSD's,
The new host will have in total 6 x 480GB SSD disks, where my idea was to have 2 of the 6 disks in RAID 1 for the PVE OS, and thus leaving 4 directly attached with a HBA for ZFS or either CEPH.
I thought it's not required for CEPH to have on every server the same amount and size of disks, isn't it?
As far as I understand, CEPH distributes the storage of the VM's to each node their disks used as OSD's.
I have no requirement of having VM/LXC disks of the existing cluster being stored on the new host.
Instead I only want the 1 VM running on the new host being stored on the PVE cluster, so only when I must recover the VM on the cluster, I can do so.

I hope I made my thoughts clear in this post, I'm looking forward to your opinion or other suggestions on this case :)

Kind regards
 
Ceph is pretty lenient, it's always easier to have the same disks on all hosts, but really it can accomodate pretty heterogenous hardware.

I'm just thinking that having your storage on the other cluster might run afoul of their requirements, but as I am not a lawyer, I don't know much about those questions lol
 
You must log in or register to reply here.
Top Bottom
Back