Is there any different of RBD(PVE) and RBD externel under proxmox?

danielc

New Member
Feb 28, 2018
29
2
3
36
Hello,

In my ceph cluster /etc/pve/storage.cfg, i found that:

This works (show as:RBD externel)
rbd: ceph-hdd
content images,rootdir
krbd 1
monhost 10.10.10.1;10.10.10.2;10.10.10.3;10.10.10.4
pool hdd
username admin

This also works (show as: RBD(PVE))
rbd: backup
content images
krbd 0
pool backup

In the ceph tutorial of proxmox, it teachs us using ebd externel.
Why not just use RBD(PVE)? is there any advantages or any problems afterwards???
It is confusing me.
Thanks!
 

spirit

Famous Member
Apr 2, 2010
3,565
164
83
www.odiso.com
rbd(PVE), autoconfigure the rbd storage is installed on proxmox. (no need to specify ip, username,....)

rbd(external), is for define an external cluster (with the monitor ips). But you can also use it with ceph installed on proxmox, no problem.
 
  • Like
Reactions: danielc

RobFantini

Renowned Member
May 24, 2012
1,664
35
68
Boston,Mass
Hello
I did find a difference between storage rbd pve and external. this may be expected or a bug.

we have a pool created before the option of pve rbd type was created:
Code:
rbd: lxc-ceph
        content rootdir
        krbd 1
        monhost 10.11.12.3;10.11.12.8;10.11.12.10
        pool lxc-ceph
        username admin
I tried creating a new rbd type pve . there is an issue with moving vm storage from the old to new:
Code:
   TASK ERROR: error with cfs lock 'storage-lxc-ceph-new': rbd error: rbd: list: (95) Operation not supported
that error went away when I deleted the rbd/ceph and used rbd/external.
 

udo

Famous Member
Apr 22, 2009
5,866
161
83
Ahrensburg; Germany
Hello
I did find a difference between storage rbd pve and external. this may be expected or a bug.

we have a pool created before the option of pve rbd type was created:
Code:
rbd: lxc-ceph
        content rootdir
        krbd 1
        monhost 10.11.12.3;10.11.12.8;10.11.12.10
        pool lxc-ceph
        username admin
I tried creating a new rbd type pve . there is an issue with moving vm storage from the old to new:
Code:
   TASK ERROR: error with cfs lock 'storage-lxc-ceph-new': rbd error: rbd: list: (95) Operation not supported
that error went away when I deleted the rbd/ceph and used rbd/external.
Hi Rob,
but your storage aren't named storage-lxc-ceph-new - your example named lxc-ceph! So the error message don't fit to the confg entry - hard to find an error.

Udo
 

RobFantini

Renowned Member
May 24, 2012
1,664
35
68
Boston,Mass
Hi Rob,
but your storage aren't named storage-lxc-ceph-new - your example named lxc-ceph! So the error message don't fit to the confg entry - hard to find an error.

Udo
I think I messed up the copy and pastes done. I had tried a few different times and may have mixed up the pad i put notes on...

So I'll try recreating the issue and post better debugging data...
 

RobFantini

Renowned Member
May 24, 2012
1,664
35
68
Boston,Mass
Hi Rob,
but your storage aren't named storage-lxc-ceph-new - your example named lxc-ceph! So the error message don't fit to the confg entry - hard to find an error.

Udo
move disk from
Code:
rbd: lxc-ceph
        content rootdir
        krbd 1
        monhost 10.11.12.3;10.11.12.8;10.11.12.10
        pool lxc-ceph
        username admin
to
Code:
rbd: lxc-test-pve
        content rootdir
        krbd 1
        pool lxc-test
result
Code:
TASK ERROR: error with cfs lock 'storage-lxc-test-pve': rbd error: rbd: list: (95) Operation not supported
 

RobFantini

Renowned Member
May 24, 2012
1,664
35
68
Boston,Mass
Also on pve screen, the 'lxc-test-pve' the storage icons at each host show a ? and when clicked display:
Code:
rbd error: rbd: list: (95) Operation not supported (500)
So it looks like I did something wrong on the set up. I'll check on this tomorrow.
 

RobFantini

Renowned Member
May 24, 2012
1,664
35
68
Boston,Mass
Hello Udo,

I rechecked the set up and can not figure out why the 'not supported (500)' error occurs. the setup of pool and storage look ok. [ see post #7 above]

/var/log/syslog shows this error :
Code:
pvestatd[5655]: rados_connect failed - Operation not supported
any clues to fix this?
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE and Proxmox Mail Gateway. We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get your own in 60 seconds.

Buy now!