PVE 2.3 Ceph multiple disk images

[f.cuseo]

Hello.
I am testing the last (yesterday) pvetest software.

Using ceph storage backend, all works ok (not backup, of course); but if I add a second hard drive image on the ceph storage, i have an error (image vm-XXX-disk-1 already exists).

Regards, Fabrizio Cuseo

 

[spirit]

Hello.
I am testing the last (yesterday) pvetest software.

Using ceph storage backend, all works ok (not backup, of course); but if I add a second hard drive image on the ceph storage, i have an error (image vm-XXX-disk-1 already exists).

Regards, Fabrizio Cuseo

Hi, this is strange, I can't reproduce the bug here. I'm currently working on new rbd feature, I'll recheck the code this week.

(Note: the new live backup works also with rbd in pvetest

 

[spirit]

Oh, I forgot : You need a ceph cluster version 0.56 for last pvestest code.

 

[f.cuseo]

Oh, I forgot : You need a ceph cluster version 0.56 for last pvestest code.

Hello.
Thank's for your reply.

This is my configuration:
- 3 x ceph nodes (as kvm virtual machines, one per host, using all local space on separate disks); this is the version:
[root@ceph-1 ~]# rpm -q ceph
ceph-0.56.2-0.el6.x86_64

- 3 x proxmox hosts, with this version:

root@nodo01:~# pveversion -v
pve-manager: 2.3-7 (pve-manager/2.3/1fe64d18)
running kernel: 2.6.32-18-pve
proxmox-ve-2.6.32: 2.3-88
pve-kernel-2.6.32-16-pve: 2.6.32-82
pve-kernel-2.6.32-18-pve: 2.6.32-88
pve-kernel-2.6.32-17-pve: 2.6.32-83
lvm2: 2.02.95-1pve2
clvm: 2.02.95-1pve2
corosync-pve: 1.4.4-4
openais-pve: 1.1.4-2
libqb: 0.10.1-2
redhat-cluster-pve: 3.1.93-2
resource-agents-pve: 3.9.2-3
fence-agents-pve: 3.1.9-1
pve-cluster: 1.0-36
qemu-server: 2.3-8
pve-firmware: 1.0-21
libpve-common-perl: 1.0-44
libpve-access-control: 1.0-25
libpve-storage-perl: 2.3-2
vncterm: 1.0-3
vzctl: 4.0-1pve2
vzprocps: 2.0.11-2
vzquota: 3.1-1
pve-qemu-kvm: 1.3-18
ksm-control-daemon: 1.1-1


with this running kernel:

root@nodo01:~# uname -a
Linux nodo01 2.6.32-18-pve #1 SMP Mon Jan 21 12:09:05 CET 2013 x86_64 GNU/Linux

and this ceph version (installed from pvetest repositary):

root@nodo01:~# ceph -v
ceph version 0.56.2 (586538e22afba85c59beda49789ec42024e7a061)

Now, if I try to add a second disk i have this error:
rbd create vm-108-disk-1' error: 2013-02-13 14:41:08.591644 7fe436dfa760 -1 librbd: rbd image vm-108-disk-1 already exists (500)

Is correct that the image already exists... but i don't know why proxmox is trying to add a disk that already exists.

This is the config of this machine:

balloon: 1024
bootdisk: sata0
cores: 2
ide2: mfsISO:iso/CentOS-6.3-x86_64-bin-DVD1.iso,media=cdrom,size=4188854K
memory: 3072
name: ZimbraRestore
net0: e1000=92:B3:469:4D:9A,bridge=vmbr1,tag=101
ostype: l26
sata0: CephCluster:vm-108-disk-1,cache=unsafe,size=160G
sockets: 1


But, strange, if I try to add a second hard disk to another virtual machine, it works (you will find in the conf. file my added and deleted hard drives):

bootdisk: sata0
cores: 1
ide2: mfsISO:iso/CentOS-6.3-x86_64-bin-DVD1.iso,media=cdrom,size=4188854K
memory: 1024
name: LinuxCeph01
net0: e1000=EE:40:AE:BF:55:E6,bridge=vmbr0
onboot: 1
ostype: l26
sata0: CephCluster:vm-104-disk-1,size=80G
sockets: 1
unused0: CephCluster:vm-104-disk-2
unused1: CephCluster:vm-104-disk-3


PS: How can i try live backup ? I can't see anything different on the "backup" tab, and if I try I have this error:
INFO: starting new backup job: vzdump 104 --remove 0 --mode snapshot --compress lzo --storage mfscluster2 --node nodo03
INFO: Starting Backup of VM 104 (qemu)
INFO: status = running
ERROR: Backup of VM 104 failed - no such volume 'CephCluster:vm-104-disk-1'
INFO: Backup job finished with errors
TASK ERROR: job errors




Thank's again, Fabrizio

 

[f.cuseo]

I also noticed this differences with disk image names:

[root@ceph-1 ~]# rados --pool=rbd ls | grep vm-10
vm-104-disk-1.rbd
vm-102-disk-1.rbd
rbd_id.vm-104-disk-2
vm-102-disk-2.rbd
rbd_id.vm-106-disk-1
rbd_id.vm-104-disk-3
rbd_id.vm-108-disk-1
rbd_id.vm-107-disk-1
vm-105-disk-1.rbd

 

[spirit]

when you create a new disk, proxmox code do a "rbd ls -l" to list the images from the pool, then choose the next available disk number.
Do maybe something is wrong here. (you could post the result of "rbd ls -l" ) ?

For backup, I didn't have tested it myself, but you can ask to Dietmar on pve-user mailing (pve-user@pve.proxmox.com.)

 

[f.cuseo]

when you create a new disk, proxmox code do a "rbd ls -l" to list the images from the pool, then choose the next available disk number.
Do maybe something is wrong here. (you could post the result of "rbd ls -l" ) ?

For backup, I didn't have tested it myself, but you can ask to Dietmar on pve-user mailing (pve-user@pve.proxmox.com.)

[root@ceph-1 ~]# rbd ls -l
NAME SIZE PARENT FMT PROT LOCK
vm-102-disk-1 120G 1
vm-102-disk-2 1024G 1
vm-104-disk-1 81920M 1
vm-105-disk-1 61440M 1
vm-110-disk-1 1024G 1
vm-104-disk-2 32768M 2
vm-104-disk-3 36864M 2
vm-106-disk-1 32768M 2
vm-107-disk-1 65536M 2
vm-108-disk-1 160G 2

 

[f.cuseo]

when you create a new disk, proxmox code do a "rbd ls -l" to list the images from the pool, then choose the next available disk number.
Do maybe something is wrong here. (you could post the result of "rbd ls -l" ) ?

For backup, I didn't have tested it myself, but you can ask to Dietmar on pve-user mailing (pve-user@pve.proxmox.com.)

I think that the problem is with the rbd image format.

Works with the virtual machine with "format type 1" disk image (the default format); doesn't work with the virtual machine with "format type 2" disk image (that supports cloning).
I have created the first machine with the 2.2 PVE release; after I have upgraded to pvetest (2.3) i have created the other machine, so I think that with pve 2.3, all the images are created with type 2 and not type 1 format.
But now I have created a new VM, with a type 2 disk image, and i can add a second disk image:

vm-109-disk-1 32768M 2
vm-109-disk-2 32768M 2

I don't know why only for machine 108 I have the problem !

 

[spirit]

I think that the problem is with the rbd image format.

Works with the virtual machine with "format type 1" disk image (the default format); doesn't work with the virtual machine with "format type 2" disk image (that supports cloning).
I have created the first machine with the 2.2 PVE release; after I have upgraded to pvetest (2.3) i have created the other machine, so I think that with pve 2.3, all the images are created with type 2 and not type 1 format.
But now I have created a new VM, with a type 2 disk image, and i can add a second disk image:

vm-109-disk-1 32768M 2
vm-109-disk-2 32768M 2

I don't know why only for machine 108 I have the problem !

I don't think it's related to image format . (no reason).
But their was a bug to parse size unit (M|G|K), this is fixed in this commit
https://git.proxmox.com/?p=pve-storage.git;a=commit;h=249cb64756a5685799c588942d90950dd391cfd2
But I'm not sure is already in pvetest repo

 

[f.cuseo]

I don't think it's related to image format . (no reason).
But their was a bug to parse size unit (M|G|K), this is fixed in this commit
https://git.proxmox.com/?p=pve-storage.git;a=commit;h=249cb64756a5685799c588942d90950dd391cfd2
But I'm not sure is already in pvetest repo

I think you have found the problem

Thanks, Fabrizio