[SOLVED] rbd mirror - cannot setup peer on site-a

seanetjersey

New Member
May 30, 2024
3
1
3
Hello,

I would be very grateful for some help, I have been trying to setup an rbd mirror on a pool between two proxmox ceph clusters.
I have followed the instructions here : https://pve.proxmox.com/wiki/Ceph_RBD_Mirroring
And also here : https://satishdotpatel.github.io/ceph-rbd-mirroring/

Where I think I am :
- pools on site-a and site-b setup with the same name
- mirroring enabled on the pools
- client auth setup and keyring copied to site-b
- site-a conf file copied to site-b
- installed the rbd-mirror daemon in site-b, and configured the /etc/systemd/system/ceph-rbd-mirror@.service ExecStart line to use the client username and keyring from site-a.
- daemon starts without error (so far as I can tell)

Code:
root@pve11:/etc/systemd/system# systemctl status ceph-rbd-mirror@rbd-mirror.pve11.service
● ceph-rbd-mirror@rbd-mirror.pve11.service - Ceph rbd mirror daemon
     Loaded: loaded (/etc/systemd/system/ceph-rbd-mirror@.service; enabled; preset: enabled)
     Active: active (running) since Mon 2024-07-01 11:05:15 BST; 6min ago
   Main PID: 1000105 (rbd-mirror)
      Tasks: 16
     Memory: 13.8M
        CPU: 148ms
     CGroup: /system.slice/system-ceph\x2drbd\x2dmirror.slice/ceph-rbd-mirror@rbd-mirror.pve11.service
             └─1000105 /usr/bin/rbd-mirror -f --cluster site-a --id rbd-mirror-peer-a --keyring=/etc/pve/priv/site-a.client.rbd-mirror-peer-a.keyring --setuser root --setgroup root

Jul 01 11:05:15 pve11 systemd[1]: Started ceph-rbd-mirror@rbd-mirror.pve11.service - Ceph rbd mirror daemon.
root@pve11:/etc/systemd/system#

- from site-b, the peer is setup :

Code:
root@pve11:/var/log/ceph# rbd mirror pool info cephpool01
Mode: image
Site Name: site-b

Peer Sites:

UUID: b271253c-ec99-4a4b-907c-94da92ef2851
Name: site-a
Direction: rx-only
Client: client.rbd-mirror-peer
root@pve11:/var/log/ceph#

- but on site-a, no peer is setup :

Code:
root@pve01:/etc/ceph# rbd mirror pool info cephpool01
Mode: image
Site Name: site-a

Peer Sites: none
root@pve01:/etc/ceph#

Any help gratefully received - I am completely stuck.

Thank you in advance
 
Hi,
just to update, this is solved.
The destination cluster had CEPH errors which were unresolved.
(In my case the CEPH errors related to the fact it is a test/lab environment and with a single host, and CEPH wasnt able to honour the replication rule requiring data be spread across different hosts. I edited the crush map, and recreated the replication pool for good measure, restarted the services and then it worked.)
 
Hello @seanetjersey I'm following the guide:
https://pve.proxmox.com/wiki/Ceph_RBD_Mirroring
but I've a situationo similar to you about site-a does't show peer when i ask info.

root@node01:/var/log/ceph# rbd mirror pool info pool01
Mode: image
Site Name: 16c6e89a-34cc-4672-86a1-b265f0f897dd

Peer Sites: none

did you follow the same guide? I'm not really shure how site-a is able to know site-b
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!