Hi there
I'm sorry if this topic aready appears here, but I cam looking for possible solution with my trouble.
I followed the Proxmox wiki regarding CEPH rbd-mirror.[0]
It's a 3-node Proxmox VE and Ceph, created with virtual machine over virt-manager. It's just a lab.
So, let's named this 3-node as CLUSTER100.
The other side, the backup side, it's single node, but with an actived cluster and ceph installed. Let's named CLUSTER200.
All hosts are in the same network:
- Management access: vmbr0 -> 192.168.122.0/24
- Cluster: nic1 -> 172.16.100.0/24
- Migrate: nic2 -> 172.16.110.0/24
- CEPH-NET: nic3 -> 172.19.150.0/24
- CEPH-OSD: nic4 -> 172.19.250.0/24
Everything went fine.
I was enabled to create the snapshot task with a interval of 5 minutes between the hosts.
So, I simulated a failover, by shuting down the vm on the CLUSTER100, the "production" side.
I shut down the vm in CLUSTER100 and did:
On the other side, in the CLUSTER200, I did:
So far so good.
I was able to started up the vm normaly.
Did some changes and so on... So far so good.
In order to do the failback, I shut down the vm on CLUSTER200, the backup side.
And I did:
Now, here is the problem:
No matter if I do
on CLUSTER100, and then
on CLUSTER200 or CLUSTER100.
When I promote the vm disk on the CLUSTER100, I got a split-brain scenario.
Sometime I was able to "solve" the split-brain thing with resync in the CLUSTER200, but then the disk on that side disappears and after some minutes, the disk appears again, because the mirror is back to work.
But for my surprise, when I turned on the vm on CLUSTER100, every thing I did on the vm while it's was on the other side (CLUSTER200) has gone!
I can't live with that.
Perhaps, it's my fault.
Perhaps I am doing something very wrong.
I need help to better understand how to do the failback.
Any assistance with be welcome.
Thanks
[0] https://pve.proxmox.com/wiki/Ceph_RBD_Mirroring
I'm sorry if this topic aready appears here, but I cam looking for possible solution with my trouble.
I followed the Proxmox wiki regarding CEPH rbd-mirror.[0]
It's a 3-node Proxmox VE and Ceph, created with virtual machine over virt-manager. It's just a lab.
So, let's named this 3-node as CLUSTER100.
The other side, the backup side, it's single node, but with an actived cluster and ceph installed. Let's named CLUSTER200.
All hosts are in the same network:
- Management access: vmbr0 -> 192.168.122.0/24
- Cluster: nic1 -> 172.16.100.0/24
- Migrate: nic2 -> 172.16.110.0/24
- CEPH-NET: nic3 -> 172.19.150.0/24
- CEPH-OSD: nic4 -> 172.19.250.0/24
Everything went fine.
I was enabled to create the snapshot task with a interval of 5 minutes between the hosts.
So, I simulated a failover, by shuting down the vm on the CLUSTER100, the "production" side.
I shut down the vm in CLUSTER100 and did:
Code:
rbd mirror image demote VMS/vm-101-disk-0
Code:
rbd mirror image promote VMS/vm-101-disk-0I was able to started up the vm normaly.
Did some changes and so on... So far so good.
In order to do the failback, I shut down the vm on CLUSTER200, the backup side.
And I did:
Code:
rbd mirror image demote VMS/vm-101-disk-0Now, here is the problem:
No matter if I do
Code:
rbd mirror image promote VMS/vm-101-disk-0
Code:
rbd mirror image resync VMS/vm-101-disk-0on CLUSTER200 or CLUSTER100.
When I promote the vm disk on the CLUSTER100, I got a split-brain scenario.
Sometime I was able to "solve" the split-brain thing with resync in the CLUSTER200, but then the disk on that side disappears and after some minutes, the disk appears again, because the mirror is back to work.
But for my surprise, when I turned on the vm on CLUSTER100, every thing I did on the vm while it's was on the other side (CLUSTER200) has gone!
I can't live with that.
Perhaps, it's my fault.
Perhaps I am doing something very wrong.
I need help to better understand how to do the failback.
Any assistance with be welcome.
Thanks
[0] https://pve.proxmox.com/wiki/Ceph_RBD_Mirroring
Last edited:
