[SOLVED] Proxmox cannot open RBD image

pieterk

Renowned Member
Feb 7, 2014
17
1
68
Hi,

Got a weird problem. Proxmox when I try to start a VM says Operation not supported when trying to attach to the RBD image.
The ceph keys are working as I can list the images of the pool on the proxmox host via the command line and check their info.

I did recently add monitors and remove monitors (old ones)
Is there a cache of config or something else that could be causing a problem in Proxmox? (error below)

Code:
kvm: -drive file=rbd:ephemeral/vm-200-disk-1:mon_host=10.40.0.33,10.40.0.34,10.40.0.35:id=proxmox:auth_supported=cephx:keyring=/etc/pve/priv/ceph/ephemeral.keyring,if=none,id=drive-virtio0,aio=native,cache=none: error connecting

kvm: -drive file=rbd:ephemeral/vm-200-disk-1:mon_host=10.40.0.33,10.40.0.34,10.40.0.35:id=proxmox:auth_supported=cephx:keyring=/etc/pve/priv/ceph/ephemeral.keyring,if=none,id=drive-virtio0,aio=native,cache=none: could not open disk image rbd:ephemeral/vm-200-disk-1:mon_host=10.40.0.33: Could not open 'rbd:ephemeral/vm-200-disk-1:mon_host=10.40.0.33': Operation not supported

TASK ERROR: start failed: command '/usr/bin/kvm -id 200 -chardev 'socket,id=qmp,path=/var/run/qemu-server/200.qmp,server,nowait' -mon 'chardev=qmp,mode=control' -vnc unix:/var/run/qemu-server/200.vnc,x509,password -pidfile /var/run/qemu-server/200.pid -daemonize -name thing -smp 'sockets=1,cores=2' -nodefaults -boot 'menu=on' -vga std -no-hpet -cpu 'kvm64,hv_spinlocks=0xffff,hv_relaxed,+lahf_lm,+x2apic,+sep' -k en-gb -m 2048 -device 'piix3-usb-uhci,id=uhci,bus=pci.0,addr=0x1.0x2' -device 'usb-tablet,id=tablet,bus=uhci.0,port=1' -device 'virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x3' -drive 'if=none,id=drive-ide2,media=cdrom,aio=native' -device 'ide-cd,bus=ide.1,unit=0,drive=drive-ide2,id=ide2,bootindex=200' -drive 'file=rbd:ephemeral/vm-200-disk-1:mon_host=10.40.0.33,10.40.0.34,10.40.0.35:id=proxmox:auth_supported=cephx:keyring=/etc/pve/priv/ceph/ephemeral.keyring,if=none,id=drive-virtio0,aio=native,cache=none' -device 'virtio-blk-pci,drive=drive-virtio0,id=virtio0,bus=pci.0,addr=0xa,bootindex=100' -netdev 'type=tap,id=net0,ifname=tap200i0,script=/var/lib/qemu-server/pve-bridge' -device 'rtl8139,mac=B2:15:36:99:60:C4,netdev=net0,bus=pci.0,addr=0x12,id=net0,bootindex=300' -rtc 'driftfix=slew,base=localtime' -global 'kvm-pit.lost_tick_policy=discard'' failed: exit code 1

Really worried (and worrying) that changing monitors can take down proxmox like this.
 
Last edited:
Hi,

Got a weird problem. Proxmox when I try to start a VM says Operation not supported when trying to attach to the RBD image.
The ceph keys are working as I can list the images of the pool on the proxmox host via the command line and check their info.

I did recently add monitors and remove monitors (old ones)
Is there a cache of config or something else that could be causing a problem in Proxmox? (error below)

Code:
kvm: -drive file=rbd:ephemeral/vm-200-disk-1:mon_host=10.40.0.33,10.40.0.34,10.40.0.35:id=proxmox:auth_supported=cephx:keyring=/etc/pve/priv/ceph/ephemeral.keyring,if=none,id=drive-virtio0,aio=native,cache=none: error connecting

kvm: -drive file=rbd:ephemeral/vm-200-disk-1:mon_host=10.40.0.33,10.40.0.34,10.40.0.35:id=proxmox:auth_supported=cephx:keyring=/etc/pve/priv/ceph/ephemeral.keyring,if=none,id=drive-virtio0,aio=native,cache=none: could not open disk image rbd:ephemeral/vm-200-disk-1:mon_host=10.40.0.33: Could not open 'rbd:ephemeral/vm-200-disk-1:mon_host=10.40.0.33': Operation not supported

TASK ERROR: start failed: command '/usr/bin/kvm -id 200 -chardev 'socket,id=qmp,path=/var/run/qemu-server/200.qmp,server,nowait' -mon 'chardev=qmp,mode=control' -vnc unix:/var/run/qemu-server/200.vnc,x509,password -pidfile /var/run/qemu-server/200.pid -daemonize -name thing -smp 'sockets=1,cores=2' -nodefaults -boot 'menu=on' -vga std -no-hpet -cpu 'kvm64,hv_spinlocks=0xffff,hv_relaxed,+lahf_lm,+x2apic,+sep' -k en-gb -m 2048 -device 'piix3-usb-uhci,id=uhci,bus=pci.0,addr=0x1.0x2' -device 'usb-tablet,id=tablet,bus=uhci.0,port=1' -device 'virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x3' -drive 'if=none,id=drive-ide2,media=cdrom,aio=native' -device 'ide-cd,bus=ide.1,unit=0,drive=drive-ide2,id=ide2,bootindex=200' -drive 'file=rbd:ephemeral/vm-200-disk-1:mon_host=10.40.0.33,10.40.0.34,10.40.0.35:id=proxmox:auth_supported=cephx:keyring=/etc/pve/priv/ceph/ephemeral.keyring,if=none,id=drive-virtio0,aio=native,cache=none' -device 'virtio-blk-pci,drive=drive-virtio0,id=virtio0,bus=pci.0,addr=0xa,bootindex=100' -netdev 'type=tap,id=net0,ifname=tap200i0,script=/var/lib/qemu-server/pve-bridge' -device 'rtl8139,mac=B2:15:36:99:60:C4,netdev=net0,bus=pci.0,addr=0x12,id=net0,bootindex=300' -rtc 'driftfix=slew,base=localtime' -global 'kvm-pit.lost_tick_policy=discard'' failed: exit code 1

Really worried (and worrying) that changing monitors can take down proxmox like this.

Hi,
you got an output of following command?
Code:
ceph -m 10.40.0.33:6789 osd getmap -o osdmap
Udo
 
When I do this on the CEPH nodes it works fine

Code:
ceph -m 10.40.0.33:6789 osd getmap -o osdmap


got osdmap epoch 778


CEPH Server is not on Proxmox as it is a separate stand-alone CEPH cluster so the "ceph" command on proxmox just fails.

Code:
ceph -m 10.40.0.33:6789 osd getmap -o osdmap --keyring /etc/pve/priv/ceph/ephemeral.keyring --id proxmox

Traceback (most recent call last):
  File "/usr/bin/ceph", line 777, in <module>
    sys.exit(main())
  File "/usr/bin/ceph", line 563, in main
    conf_defaults=conf_defaults, conffile=conffile)
  File "/usr/lib/python2.7/dist-packages/rados.py", line 221, in __init__
    self.conf_read_file(conffile)
  File "/usr/lib/python2.7/dist-packages/rados.py", line 272, in conf_read_file
    raise make_ex(ret, "error calling conf_read_file")
rados.Error: error calling conf_read_file: errno EINVAL
 
Ok, this is a major fail on my part (some online tutorial also shows it wrong)

I have the mon IP addresses in the format of x.x.x.x,x.x.x.x,x.x.x.x but it should have been x.x.x.x;x.x.x.x;x.x.x.x <<-- yup them semicolons

All working now. Thanks for the responder (udo) though for at least trying to help. Much appreciated.
 
  • Like
Reactions: janos

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!