RBD: error connecting: Operation not supported

J

janos

Well-Known Member
Aug 24, 2017
174
16
58
Hungary
www.awh.hu
I updated one of my node to the latest version, but on this node, KVM guest with Ceph RBD (we have external ceph cluster) not starting.

Code: 
pveversion -v
proxmox-ve: 5.1-43 (running kernel: 4.13.16-2-pve)
pve-manager: 5.1-52 (running version: 5.1-52/ba597a64)
pve-kernel-4.13: 5.1-44
pve-kernel-4.15: 5.1-3
pve-kernel-4.15.15-1-pve: 4.15.15-6
pve-kernel-4.13.16-2-pve: 4.13.16-47
pve-kernel-4.13.16-1-pve: 4.13.16-46
pve-kernel-4.13.13-6-pve: 4.13.13-42
pve-kernel-4.13.13-5-pve: 4.13.13-38
pve-kernel-4.13.13-3-pve: 4.13.13-34
corosync: 2.4.2-pve5
criu: 2.11.1-1~bpo90
glusterfs-client: 3.8.8-1
ksm-control-daemon: not correctly installed
libjs-extjs: 6.0.1-2
libpve-access-control: 5.0-8
libpve-apiclient-perl: 2.0-4
libpve-common-perl: 5.0-30
libpve-guest-common-perl: 2.0-15
libpve-http-server-perl: 2.0-8
libpve-storage-perl: 5.0-19
libqb0: 1.0.1-1
lvm2: 2.02.168-pve6
lxc-pve: 3.0.0-2
lxcfs: 3.0.0-1
novnc-pve: 0.6-4
openvswitch-switch: 2.7.0-2
proxmox-widget-toolkit: 1.0-15
pve-cluster: 5.0-26
pve-container: 2.0-22
pve-docs: 5.1-17
pve-firewall: 3.0-8
pve-firmware: 2.0-4
pve-ha-manager: 2.0-5
pve-i18n: 1.0-4
pve-libspice-server1: 0.12.8-3
pve-qemu-kvm: 2.11.1-5
pve-xtermjs: 1.0-3
qemu-server: 5.0-25
smartmontools: 6.5+svn4324-1
spiceterm: 3.0-5
vncterm: 1.5-3

However, manually i can open the image, even i can attach it (using rbd-nbd):

Code: 
~ # rbd-nbd  -m 172.24.2.31 --user proxmox -k /etc/pve/priv/ceph/rbd.keyring map proxmox/vm-211-disk-1
/dev/nbd0

This is the command what Proxmox try to start:

Code: 
'file=rbd:proxmox/vm-189-disk-1:mon_host=172.24.2.31,172.24.2.32,172.24.2.33:auth_supported=cephx:id=proxmox:keyring=/etc/pve/priv/ceph/rbd.keyring,if=none,id=drive-scsi0,discard=on,format=raw,cache=none,aio=native,detect-zeroes=unmap'

And looks like, this is the good, what need to be, so ceph mon hosts need to be separated by ; instead of ,

Code: 
'file=rbd:proxmox/vm-189-disk-1:mon_host=172.24.2.31;172.24.2.32;172.24.2.33:auth_supported=cephx:id=proxmox:keyring=/etc/pve/priv/ceph/rbd.keyring,if=none,id=drive-scsi0,discard=on,format=raw,cache=none,aio=native,detect-zeroes=unmap'

Server, what have pve-qemu-kvm_2.9.1-9 working without any issue with column separated monitoring hosts.
 
/etc/pve/storage.cfg part:

Code: 
rbd: rbd
        content images
        krbd 0
        monhost 172.24.2.31;172.24.2.32;172.24.2.33
        pool proxmox
        username proxmox
 
So, i found the issue.

In /usr/share/perl5/PVE/Storage/RBDPlugin.pm file, older version have the correct semicolon list for mon hosts, but the new version have simply .

Around line 63:

Code: 
$cmd_option->{mon_host} = $hostlist->($scfg->{monhost}, ',') if (defined($scfg->{monhost}));

I modified this line, and everything working without issue.
 
However, i guess there is some other place where this wrong, because i can start VM with qm commadn, but i can not from the web interface, its existing with the same issue, related to colon.
 
janos said:
However, i guess there is some other place where this wrong, because i can start VM with qm commadn, but i can not from the web interface, its existing with the same issue, related to colon.
Click to expand...
I rebooted the node, after the web interface also working. I guess perl file cached somewhere.
 
You can apply the bugfix online, without rebooting the whole node. Just try to restart "pvedaemon.service" with following command "systemctl restart pvedaemon.service" - this solve the Problem in our Cluster.
 
You must log in or register to reply here.
Top Bottom