Is there any different of RBD(PVE) and RBD externel under proxmox?

Discussion in 'Proxmox VE: Installation and configuration' started by danielc, Jun 30, 2018.

  1. danielc

    danielc New Member

    Joined:
    Feb 28, 2018
    Messages:
    26
    Likes Received:
    1
    Hello,

    In my ceph cluster /etc/pve/storage.cfg, i found that:

    This works (show as:RBD externel)
    rbd: ceph-hdd
    content images,rootdir
    krbd 1
    monhost 10.10.10.1;10.10.10.2;10.10.10.3;10.10.10.4
    pool hdd
    username admin

    This also works (show as: RBD(PVE))
    rbd: backup
    content images
    krbd 0
    pool backup

    In the ceph tutorial of proxmox, it teachs us using ebd externel.
    Why not just use RBD(PVE)? is there any advantages or any problems afterwards???
    It is confusing me.
    Thanks!
     
  2. spirit

    spirit Well-Known Member

    Joined:
    Apr 2, 2010
    Messages:
    3,302
    Likes Received:
    131
    rbd(PVE), autoconfigure the rbd storage is installed on proxmox. (no need to specify ip, username,....)

    rbd(external), is for define an external cluster (with the monitor ips). But you can also use it with ceph installed on proxmox, no problem.
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
    danielc likes this.
  3. danielc

    danielc New Member

    Joined:
    Feb 28, 2018
    Messages:
    26
    Likes Received:
    1
    Hello spirit,

    Thank you very much for the explaination. So will stick with rbd pve then.
     
  4. RobFantini

    RobFantini Active Member
    Proxmox Subscriber

    Joined:
    May 24, 2012
    Messages:
    1,488
    Likes Received:
    21
    Hello
    I did find a difference between storage rbd pve and external. this may be expected or a bug.

    we have a pool created before the option of pve rbd type was created:
    Code:
    rbd: lxc-ceph
            content rootdir
            krbd 1
            monhost 10.11.12.3;10.11.12.8;10.11.12.10
            pool lxc-ceph
            username admin
    
    I tried creating a new rbd type pve . there is an issue with moving vm storage from the old to new:
    Code:
       TASK ERROR: error with cfs lock 'storage-lxc-ceph-new': rbd error: rbd: list: (95) Operation not supported
    
    that error went away when I deleted the rbd/ceph and used rbd/external.
     
  5. udo

    udo Well-Known Member
    Proxmox Subscriber

    Joined:
    Apr 22, 2009
    Messages:
    5,829
    Likes Received:
    158
    Hi Rob,
    but your storage aren't named storage-lxc-ceph-new - your example named lxc-ceph! So the error message don't fit to the confg entry - hard to find an error.

    Udo
     
  6. RobFantini

    RobFantini Active Member
    Proxmox Subscriber

    Joined:
    May 24, 2012
    Messages:
    1,488
    Likes Received:
    21
    I think I messed up the copy and pastes done. I had tried a few different times and may have mixed up the pad i put notes on...

    So I'll try recreating the issue and post better debugging data...
     
  7. RobFantini

    RobFantini Active Member
    Proxmox Subscriber

    Joined:
    May 24, 2012
    Messages:
    1,488
    Likes Received:
    21
    move disk from
    Code:
    rbd: lxc-ceph
            content rootdir
            krbd 1
            monhost 10.11.12.3;10.11.12.8;10.11.12.10
            pool lxc-ceph
            username admin
    
    to
    Code:
    rbd: lxc-test-pve
            content rootdir
            krbd 1
            pool lxc-test
    
    result
    Code:
    TASK ERROR: error with cfs lock 'storage-lxc-test-pve': rbd error: rbd: list: (95) Operation not supported
    
     
  8. RobFantini

    RobFantini Active Member
    Proxmox Subscriber

    Joined:
    May 24, 2012
    Messages:
    1,488
    Likes Received:
    21
    Also on pve screen, the 'lxc-test-pve' the storage icons at each host show a ? and when clicked display:
    Code:
    rbd error: rbd: list: (95) Operation not supported (500)
    
    So it looks like I did something wrong on the set up. I'll check on this tomorrow.
     
  9. RobFantini

    RobFantini Active Member
    Proxmox Subscriber

    Joined:
    May 24, 2012
    Messages:
    1,488
    Likes Received:
    21
    Hello Udo,

    I rechecked the set up and can not figure out why the 'not supported (500)' error occurs. the setup of pool and storage look ok. [ see post #7 above]

    /var/log/syslog shows this error :
    Code:
    pvestatd[5655]: rados_connect failed - Operation not supported
    
    any clues to fix this?
     
  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.
    Dismiss Notice