RBD: error connecting: Operation not supported

J

janos

Well-Known Member
Aug 24, 2017
174
16
58
Hungary
www.awh.hu
I updated one of my node to the latest version, but on this node, KVM guest with Ceph RBD (we have external ceph cluster) not starting.

Code: 
pveversion -v
proxmox-ve: 5.1-43 (running kernel: 4.13.16-2-pve)
pve-manager: 5.1-52 (running version: 5.1-52/ba597a64)
pve-kernel-4.13: 5.1-44
pve-kernel-4.15: 5.1-3
pve-kernel-4.15.15-1-pve: 4.15.15-6
pve-kernel-4.13.16-2-pve: 4.13.16-47
pve-kernel-4.13.16-1-pve: 4.13.16-46
pve-kernel-4.13.13-6-pve: 4.13.13-42
pve-kernel-4.13.13-5-pve: 4.13.13-38
pve-kernel-4.13.13-3-pve: 4.13.13-34
corosync: 2.4.2-pve5
criu: 2.11.1-1~bpo90
glusterfs-client: 3.8.8-1
ksm-control-daemon: not correctly installed
libjs-extjs: 6.0.1-2
libpve-access-control: 5.0-8
libpve-apiclient-perl: 2.0-4
libpve-common-perl: 5.0-30
libpve-guest-common-perl: 2.0-15
libpve-http-server-perl: 2.0-8
libpve-storage-perl: 5.0-19
libqb0: 1.0.1-1
lvm2: 2.02.168-pve6
lxc-pve: 3.0.0-2
lxcfs: 3.0.0-1
novnc-pve: 0.6-4
openvswitch-switch: 2.7.0-2
proxmox-widget-toolkit: 1.0-15
pve-cluster: 5.0-26
pve-container: 2.0-22
pve-docs: 5.1-17
pve-firewall: 3.0-8
pve-firmware: 2.0-4
pve-ha-manager: 2.0-5
pve-i18n: 1.0-4
pve-libspice-server1: 0.12.8-3
pve-qemu-kvm: 2.11.1-5
pve-xtermjs: 1.0-3
qemu-server: 5.0-25
smartmontools: 6.5+svn4324-1
spiceterm: 3.0-5
vncterm: 1.5-3

However, manually i can open the image, even i can attach it (using rbd-nbd):

Code: 
~ # rbd-nbd  -m 172.24.2.31 --user proxmox -k /etc/pve/priv/ceph/rbd.keyring map proxmox/vm-211-disk-1
/dev/nbd0

This is the command what Proxmox try to start:

Code: 
'file=rbd:proxmox/vm-189-disk-1:mon_host=172.24.2.31,172.24.2.32,172.24.2.33:auth_supported=cephx:id=proxmox:keyring=/etc/pve/priv/ceph/rbd.keyring,if=none,id=drive-scsi0,discard=on,format=raw,cache=none,aio=native,detect-zeroes=unmap'

And looks like, this is the good, what need to be, so ceph mon hosts need to be separated by ; instead of ,

Code: 
'file=rbd:proxmox/vm-189-disk-1:mon_host=172.24.2.31;172.24.2.32;172.24.2.33:auth_supported=cephx:id=proxmox:keyring=/etc/pve/priv/ceph/rbd.keyring,if=none,id=drive-scsi0,discard=on,format=raw,cache=none,aio=native,detect-zeroes=unmap'

Server, what have pve-qemu-kvm_2.9.1-9 working without any issue with column separated monitoring hosts.
 
/etc/pve/storage.cfg part:

Code: 
rbd: rbd
        content images
        krbd 0
        monhost 172.24.2.31;172.24.2.32;172.24.2.33
        pool proxmox
        username proxmox
 
So, i found the issue.

In /usr/share/perl5/PVE/Storage/RBDPlugin.pm file, older version have the correct semicolon list for mon hosts, but the new version have simply .

Around line 63:

Code: 
$cmd_option->{mon_host} = $hostlist->($scfg->{monhost}, ',') if (defined($scfg->{monhost}));

I modified this line, and everything working without issue.
 
However, i guess there is some other place where this wrong, because i can start VM with qm commadn, but i can not from the web interface, its existing with the same issue, related to colon.
 
janos said:
However, i guess there is some other place where this wrong, because i can start VM with qm commadn, but i can not from the web interface, its existing with the same issue, related to colon.
Click to expand...
I rebooted the node, after the web interface also working. I guess perl file cached somewhere.
 
You can apply the bugfix online, without rebooting the whole node. Just try to restart "pvedaemon.service" with following command "systemctl restart pvedaemon.service" - this solve the Problem in our Cluster.
 
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom
Back