Is there any different of RBD(PVE) and RBD externel under proxmox?

D

danielc

Member
Feb 28, 2018
35
2
13
41
Hello,

In my ceph cluster /etc/pve/storage.cfg, i found that:

This works (show as:RBD externel)
rbd: ceph-hdd
content images,rootdir
krbd 1
monhost 10.10.10.1;10.10.10.2;10.10.10.3;10.10.10.4
pool hdd
username admin

This also works (show as: RBD(PVE))
rbd: backup
content images
krbd 0
pool backup

In the ceph tutorial of proxmox, it teachs us using ebd externel.
Why not just use RBD(PVE)? is there any advantages or any problems afterwards???
It is confusing me.
Thanks!
 
rbd(PVE), autoconfigure the rbd storage is installed on proxmox. (no need to specify ip, username,....)

rbd(external), is for define an external cluster (with the monitor ips). But you can also use it with ceph installed on proxmox, no problem.
 
  • Like
Reactions: danielc
Hello
I did find a difference between storage rbd pve and external. this may be expected or a bug.

we have a pool created before the option of pve rbd type was created:
Code: 
rbd: lxc-ceph
        content rootdir
        krbd 1
        monhost 10.11.12.3;10.11.12.8;10.11.12.10
        pool lxc-ceph
        username admin

I tried creating a new rbd type pve . there is an issue with moving vm storage from the old to new:
Code: 
   TASK ERROR: error with cfs lock 'storage-lxc-ceph-new': rbd error: rbd: list: (95) Operation not supported

that error went away when I deleted the rbd/ceph and used rbd/external.
 
RobFantini said:
Hello
I did find a difference between storage rbd pve and external. this may be expected or a bug.

we have a pool created before the option of pve rbd type was created:
Code: 
rbd: lxc-ceph
        content rootdir
        krbd 1
        monhost 10.11.12.3;10.11.12.8;10.11.12.10
        pool lxc-ceph
        username admin

I tried creating a new rbd type pve . there is an issue with moving vm storage from the old to new:
Code: 
   TASK ERROR: error with cfs lock 'storage-lxc-ceph-new': rbd error: rbd: list: (95) Operation not supported

that error went away when I deleted the rbd/ceph and used rbd/external.
Click to expand...
Hi Rob,
but your storage aren't named storage-lxc-ceph-new - your example named lxc-ceph! So the error message don't fit to the confg entry - hard to find an error.

Udo
 
udo said:
Hi Rob,
but your storage aren't named storage-lxc-ceph-new - your example named lxc-ceph! So the error message don't fit to the confg entry - hard to find an error.

Udo
Click to expand...

I think I messed up the copy and pastes done. I had tried a few different times and may have mixed up the pad i put notes on...

So I'll try recreating the issue and post better debugging data...
 
udo said:
Hi Rob,
but your storage aren't named storage-lxc-ceph-new - your example named lxc-ceph! So the error message don't fit to the confg entry - hard to find an error.

Udo
Click to expand...

move disk from
Code: 
rbd: lxc-ceph
        content rootdir
        krbd 1
        monhost 10.11.12.3;10.11.12.8;10.11.12.10
        pool lxc-ceph
        username admin
to
Code: 
rbd: lxc-test-pve
        content rootdir
        krbd 1
        pool lxc-test

result
Code: 
TASK ERROR: error with cfs lock 'storage-lxc-test-pve': rbd error: rbd: list: (95) Operation not supported
 
Also on pve screen, the 'lxc-test-pve' the storage icons at each host show a ? and when clicked display:
Code: 
rbd error: rbd: list: (95) Operation not supported (500)

So it looks like I did something wrong on the set up. I'll check on this tomorrow.
 
Hello Udo,

I rechecked the set up and can not figure out why the 'not supported (500)' error occurs. the setup of pool and storage look ok. [ see post #7 above]

/var/log/syslog shows this error :
Code: 
pvestatd[5655]: rados_connect failed - Operation not supported

any clues to fix this?
 
You must log in or register to reply here.
Top Bottom