Proxmox+Ceph: Got timeout during start Vms

[mrbat]

Hi Guys,


I just created a Proxmox - Ceph cluster.
Everything looks good except two things.

When I create a VM on the Ceph storage it's really slow.
But what is a bigger problem I can't start the VM.
I got a timeout error when I try to start it.

TASK ERROR: start failed: command '/usr/bin/kvm -id 100 -chardev 'socket,id=qmp,path=/var/run/qemu-server/100.qmp,server,nowait' -mon 'chardev=qmp,mode=control' -vnc unix:/var/run/qemu-server/100.vnc,x509,password -pidfile /var/run/qemu-server/100.pid -daemonize -smbios 'type=1,uuid=6d86f4ac-6e2a-4270-908c-377f43f3aae3' -name test -smp 'sockets=1,cores=2' -nodefaults -boot 'menu=on' -vga cirrus -cpu qemu64 -k en-gb -m 10024 -device 'piix3-usb-uhci,id=uhci,bus=pci.0,addr=0x1.0x2' -device 'usb-tablet,id=tablet,bus=uhci.0,port=1' -device 'virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x3' -iscsi 'initiator-name=iqn.1993-08.org.debian:01:9d70341282d8' -drive 'file=/var/lib/vz/template/iso/ubuntu-14.04.1-server-amd64.iso,if=none,id=drive-ide2,media=cdrom,aio=native' -device 'ide-cd,bus=ide.1,unit=0,drive=drive-ide2,id=ide2,bootindex=200' -drive 'file=rbd:rbd/vm-100-disk-1:mon_host=10.180.40.121\:6789 10.180.40.127\:6789:id=admin:auth_supported=cephx:keyring=/etc/pve/priv/ceph/dqsdb.keyring,if=none,id=drive-virtio0,aio=native,cache=none' -device 'virtio-blk-pci,drive=drive-virtio0,id=virtio0,bus=pci.0,addr=0xa,bootindex=100' -netdev 'type=tap,id=net0,ifname=tap100i0,script=/var/lib/qemu-server/pve-bridge,downscript=/var/lib/qemu-server/pve-bridgedown,vhost=on' -device 'virtio-net-pci,mac=1A:AF:24:A3:E9:A6,netdev=net0,bus=pci.0,addr=0x12,id=net0,bootindex=300' -machine 'accel=tcg'' failed: got timeout


If I create a VM on local storage it's pretty fast and I can start it without any problem...

Somebody has any idea what could be the problem?
Thank you for your help.

 

[udo]

Hi,
how looks your ceph-cluster?

Can you post the output of following commands:

ceph health detail
ceph -s
ceph osd tree

Udo

 

[arif17]

Hi, I got the same problem

TASK ERROR: start failed: command '/usr/bin/kvm -id 120 -chardev 'socket,id=qmp,path=/var/run/qemu-server/120.qmp,server,nowait' -mon 'chardev=qmp,mode=control' -vnc unix:/var/run/qemu-server/120.vnc,x509,password -pidfile /var/run/qemu-server/120.pid -daemonize -smbios 'type=1,uuid=2cd7dd2d-25ab-46f1-8a8a-02d397ce24a7' -name ceph01 -smp '1,sockets=1,cores=1,maxcpus=1' -nodefaults -boot 'menu=on,strict=on,reboot-timeout=1000' -vga cirrus -cpu qemu64 -m 512 -k en-us -cpuunits 1000 -device 'piix3-usb-uhci,id=uhci,bus=pci.0,addr=0x1.0x2' -device 'usb-tablet,id=tablet,bus=uhci.0,port=1' -device 'virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x3' -iscsi 'initiator-name=iqn.1993-08.org.debian:01:f1312d5e6238' -drive 'file=/var/lib/vz/template/iso/ubuntu-14.04.1-server-amd64_2.iso,if=none,id=drive-ide2,media=cdrom,aio=native' -device 'ide-cd,bus=ide.1,unit=0,drive=drive-ide2,id=ide2,bootindex=200' -drive 'file=rbd:ceph-test/vm-120-disk-1:mon_host=10.6.252.111 10.6.252.112 10.6.252.113:id=admin:auth_supported=cephx:keyring=/etc/pve/priv/ceph/ceph-test.keyring,if=none,id=drive-ide0,aio=native,cache=none,detect-zeroes=on' -device 'ide-hd,bus=ide.0,unit=0,drive=drive-ide0,id=ide0,bootindex=100' -netdev 'type=tap,id=net0,ifname=tap120i0,script=/var/lib/qemu-server/pve-bridge,downscript=/var/lib/qemu-server/pve-bridgedown' -device 'e1000,mac=2A:88:44:8F:A9:55,netdev=net0,bus=pci.0,addr=0x12,id=net0,bootindex=300' -machine 'accel=tcg'' failed: got timeout

ceph health detail
HEALTH_OK

ceph -s
cluster d6607e1b-7ade-4662-b52f-9dcdad2872dd
health HEALTH_OK
monmap e3: 3 mons at {0=10.6.252.111:6789/0,1=10.6.252.112:6789/0,2=10.6.252.113:6789/0}, election epoch 12, quorum 0,1,2 0,1,2
osdmap e98: 9 osds: 9 up, 9 in
pgmap v1617: 364 pgs, 2 pools, 16 bytes data, 3 objects
326 MB used, 64086 MB / 64412 MB avail
364 active+clean

ceph osd tree
# id weight type name up/down reweight
-1 0.08995 root default
-2 0.02998 host proxmoxsrv01
0 0.009995 osd.0 up 1
1 0.009995 osd.1 up 1
2 0.009995 osd.2 up 1
-3 0.02998 host proxmoxsrv02
3 0.009995 osd.3 up 1
4 0.009995 osd.4 up 1
5 0.009995 osd.5 up 1
-4 0.02998 host proxmoxsrv03
6 0.009995 osd.6 up 1
7 0.009995 osd.7 up 1
8 0.009995 osd.8 up 1

any idea what could be the problem?

 

[nethfel]

Your ceph cluster looks really small - 64Gig across 9 osds? Are you using partitions on drives to be a single OSD or is this all virtualized (each OSD is a VHD) or what? Depending on how you've created your ceph cluster, you may have really gimped yourself in terms of speed. Plus with so few, even if it was all physical the speed would be limited without using SSD journals. If this whole thing is virtual and the OSDs are VHDs all stored on a single physical disk, your performance will be dreadful.