Problem with RBD on Ceph IPv6 external cluster

Discussion in 'Proxmox VE: Installation and configuration' started by blaz, Feb 14, 2019.

  1. blaz

    blaz New Member

    Joined:
    Feb 14, 2019
    Messages:
    3
    Likes Received:
    0
    Hi all,

    So I have two external Ceph clusters - one is using IPv4 and the other is using IPv6. When I'm using IPv4 cluster, the dashborad shows status and content of RBD and I have no problem creating new images for VMs. But when I try to configure storage information about IPv6 cluster, I get Not Available tag on usage. For both clusters I make the same procedure for assigning keyrings and configuration settings for Proxmox.

    Ceph clusters have installed Mimic version and the Proxmox cluster has version Luminous. min_compat_client is set to Luminous version and the Crush tunnables are set on optimal on both clusters.

    Does anyone have any clue what could be wrong, or how to debug this issue?

    Thanks.
     
  2. Alwin

    Alwin Proxmox Staff Member
    Staff Member

    Joined:
    Aug 1, 2017
    Messages:
    2,027
    Likes Received:
    175
    The Proxmox cluster can reach the Ceph cluster over IPv6?
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  3. blaz

    blaz New Member

    Joined:
    Feb 14, 2019
    Messages:
    3
    Likes Received:
    0
    Yes. I can even manually list or create image on pool with command:
    "rbd create --size 1024 libvirt-pool/temp_img -m [2001:1470:8000:403::xxx:xxx]:6789 ..."
     
  4. Alwin

    Alwin Proxmox Staff Member
    Staff Member

    Joined:
    Aug 1, 2017
    Messages:
    2,027
    Likes Received:
    175
    You are creating the rbd images on Ceph from the proxmox cluster? If you didn't specify an alternative ceph.conf, then you need to add the IPs in '/etc/ceph/ceph.conf' too (may be a symlink to /etc/pve/ceph.conf).
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  5. blaz

    blaz New Member

    Joined:
    Feb 14, 2019
    Messages:
    3
    Likes Received:
    0
    Yes, from the Proxmox cluster, I just wanted to saw if Proxmox can access the pool. I tried to make ceph.conf file with public_network, ms_bind_ipv6, mon_initial_members and mon_host, but no luck. Am I doing something wrong? Should I specify IPs in different way?
     
  6. Alwin

    Alwin Proxmox Staff Member
    Staff Member

    Joined:
    Aug 1, 2017
    Messages:
    2,027
    Likes Received:
    175
    The pve-storage code uses the IPs set in the storage.cfg, but for the ceph tools you need to set the IPs in the ceph.conf. You can create one in /etc/pve/ceph.conf and symlink it on the hosts, so only one update is needed for the cluster.
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.
    Dismiss Notice