QEMU 1.4, Ceph RBD support (pvetest)

[Milkwyrm]

DOH!! yep that did it... thanks Tom.

 

[johjoh]

Do you suggest to adopt a new KVM standard VM creation based on QCOW2, KVM64, e1000?
What kind of disk do you suggest to use between SATA, VIRTIO, IDE and SCSI?
It this valid for Linux and Windows OS?

Thank you

 

[nz_monkey]

Will live storage migration be coming in the 2.3 release ?

 

[ebiss]

upload iso error:

starting file import from: /var/tmp/CGItemp33871
target node: proxmox20151
target file: /var/lib/vz/template/iso/ubuntu-12.04.2-server-amd64.iso
file size is: 687865856
command: /usr/bin/scp -o BatchMode=yes /var/tmp/CGItemp33871 172.18.xxx.xxx:/var/lib/vz/template/iso/ubuntu-12.04.2-server-amd64.iso
TASK ERROR: import failed: /var/tmp/CGItemp33871: No such file or directory

It's a two nodes cluster.

 

[spirit]

Will live storage migration be coming in the 2.3 release ?

Hi, This is currenly in progress (80%) but will not be ready for 2.3, sorry.

 

[Cyberco Ltd]

Hello,

Was there any specific reason for changing in the default storage disk format from "raw" to "qcow2"?

Thanks,

 

[tom]

yes, because of http://pve.proxmox.com/wiki/Live_Snapshots

and as we pre-allocate now, qcow2 performance is quite ok (compared to raw)

 

[badji]

hello,
I try to create a cluster ceph of 3 servers each with 3 discs, following the tutorial of Alexandre. everything works until the commande :

mkcephfs -a -c /etc/ceph/ceph.conf -k ceph.keyring

and there, I get this error message:

** ERROR: error creating empty object store in /var/lib/ceph/osd/ceph-0: (22) Invalid argument
failed: '/sbin/mkcephfs -d /tmp/mkcephfs.uEIoAFoyHv --init-daemon osd.0'

I see that error message is often returned in the net.

and when i use : $ceph -s
i have this message :
2013-02-27 16:23:14.369605 7f05ecbc6780 -1 WARNING: unknown auth protocol defined: ceph1
2013-02-27 16:23:14.369637 7f05ecbc6780 -1 monclient(hunting): ERROR: missing keyring, cannot use cephx for authentication
2013-02-27 16:23:14.369665 7f05ecbc6780 -1 ceph_tool_common_init failed.

Thank you for your help.

 

[tom]

if have issues and question regarding the ceph cluster setup, you should think of posting this in the ceph community. the experts are there.

 

[Cyberco Ltd]

yes, because of http://pve.proxmox.com/wiki/Live_Snapshots

and as we pre-allocate now, qcow2 performance is quite ok (compared to raw)

Thank you for the feedback Tom.
I'll start converting existing disks to qcow2 format.

 

[spirit]

hello,
I try to create a cluster ceph of 3 servers each with 3 discs, following the tutorial of Alexandre. everything works until the commande :

mkcephfs -a -c /etc/ceph/ceph.conf -k ceph.keyring

and there, I get this error message:

** ERROR: error creating empty object store in /var/lib/ceph/osd/ceph-0: (22) Invalid argument
failed: '/sbin/mkcephfs -d /tmp/mkcephfs.uEIoAFoyHv --init-daemon osd.0'

I see that error message is often returned in the net.

and when i use : $ceph -s
i have this message :
2013-02-27 16:23:14.369605 7f05ecbc6780 -1 WARNING: unknown auth protocol defined: ceph1
2013-02-27 16:23:14.369637 7f05ecbc6780 -1 monclient(hunting): ERROR: missing keyring, cannot use cephx for authentication
2013-02-27 16:23:14.369665 7f05ecbc6780 -1 ceph_tool_common_init failed.

Thank you for your help.

Seem that you don't have the keyring,

as workaround, you can try to disable authentification:

auth supported = none
auth cluster required = none
auth service required = none
auth client required = none


For cephx, verify that you have a /etc/ceph/ceph.keyring on each server

 

[dietmar]

I get some problem with offline KVM migration with last update
problem code here

This will be fixed in qemu-server 2.3-14

 

[nz_monkey]

Hi, This is currenly in progress (80%) but will not be ready for 2.3, sorry.

Thanks for all the hard work guys!

I cant wait for 2.3 to be released so I can start testing the features slated for the next release such as "Live Storage Migration"

We specifically need to be able to migrate from iSCSI/LVM to RBD and vice versa.

 

[spirit]

Thanks for all the hard work guys!

I cant wait for 2.3 to be released so I can start testing the features slated for the next release such as "Live Storage Migration"

We specifically need to be able to migrate from iSCSI/LVM to RBD and vice versa.

Yes, live storage migration works really fine, from any storage to any storage

I think the patchs will be pushed soon in pvetest after 2.3 release.

 

[mir]

Yes, live storage migration works really fine, from any storage to any storage

I think the patchs will be pushed soon in pvetest after 2.3 release.

Yes, they work very fine and I am already using it in production since a couple of weeks. Disclaimer: Above statement can be subject to subjectivity since I wrote the patch

 

[Milkwyrm]

This is very cool stuff. Now, who do we thank for pushing out spice patches *** ***

 

[Norman Uittenbogaart]

mapping the rbd device like

rbd map rbdname

format the rbd device and then mount it to a directory, which you can then add to proxmox and run your containers on.
Doing the same now in a test, only now I have mounted the rbd on a proxy and running a iSCSI export to export the device, then add it to proxmox and mount it, a bit of a poor solution but is working excellent in the test so far.
So adding the rbd block module would be great as it saves a lot of steps.

See http://ceph.com/docs/master/start/quick-rbd/ for the same solution.

Is this now added in the new kernel ?

 

[dietmar]

Is this now added in the new kernel ?

no, not added.

 

[Norman Uittenbogaart]

no, not added.

Can I request it as a loadable module ?

 

[dietmar]

Can I request it as a loadable module ?

Wer use RHEL based kernel (2.6.32), and AFAIK there are not stable rbd patches available for that kernel?