Problem with RBD on Ceph IPv6 external cluster

blaz

New Member
Feb 14, 2019
3
0
1
24
Hi all,

So I have two external Ceph clusters - one is using IPv4 and the other is using IPv6. When I'm using IPv4 cluster, the dashborad shows status and content of RBD and I have no problem creating new images for VMs. But when I try to configure storage information about IPv6 cluster, I get Not Available tag on usage. For both clusters I make the same procedure for assigning keyrings and configuration settings for Proxmox.

Ceph clusters have installed Mimic version and the Proxmox cluster has version Luminous. min_compat_client is set to Luminous version and the Crush tunnables are set on optimal on both clusters.

Does anyone have any clue what could be wrong, or how to debug this issue?

Thanks.
 

Alwin

Proxmox Staff Member
Staff member
Aug 1, 2017
2,672
233
63
The Proxmox cluster can reach the Ceph cluster over IPv6?
 

blaz

New Member
Feb 14, 2019
3
0
1
24
The Proxmox cluster can reach the Ceph cluster over IPv6?
Yes. I can even manually list or create image on pool with command:
"rbd create --size 1024 libvirt-pool/temp_img -m [2001:1470:8000:403::xxx:xxx]:6789 ..."
 

Alwin

Proxmox Staff Member
Staff member
Aug 1, 2017
2,672
233
63
Yes. I can even manually list or create image on pool with command:
"rbd create --size 1024 libvirt-pool/temp_img -m [2001:1470:8000:403::xxx:xxx]:6789 ..."
You are creating the rbd images on Ceph from the proxmox cluster? If you didn't specify an alternative ceph.conf, then you need to add the IPs in '/etc/ceph/ceph.conf' too (may be a symlink to /etc/pve/ceph.conf).
 

blaz

New Member
Feb 14, 2019
3
0
1
24
You are creating the rbd images on Ceph from the proxmox cluster? If you didn't specify an alternative ceph.conf, then you need to add the IPs in '/etc/ceph/ceph.conf' too (may be a symlink to /etc/pve/ceph.conf).
Yes, from the Proxmox cluster, I just wanted to saw if Proxmox can access the pool. I tried to make ceph.conf file with public_network, ms_bind_ipv6, mon_initial_members and mon_host, but no luck. Am I doing something wrong? Should I specify IPs in different way?
 

Alwin

Proxmox Staff Member
Staff member
Aug 1, 2017
2,672
233
63
The pve-storage code uses the IPs set in the storage.cfg, but for the ceph tools you need to set the IPs in the ceph.conf. You can create one in /etc/pve/ceph.conf and symlink it on the hosts, so only one update is needed for the cluster.
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE and Proxmox Mail Gateway. We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get your own in 60 seconds.

Buy now!