PVE 9.2 + Ceph Tentacle 20.2.4: RBD operations fail because pve-storage passes removed --auth_supported option

UHL-Services

Member
Mar 15, 2022
7
1
21
40
Switzerland
Hi,

I’m running into what looks like a compatibility issue between current Proxmox VE 9.2 storage code and Ceph Tentacle 20.2.4.

Environment:

Proxmox VE: 9.2
pve-manager: 9.2.11
libpve-storage-perl: 9.1.8
qemu-server: 9.2.6

Ceph: 20.2.4-pve1 (Tentacle)
ceph-common: 20.2.4-pve1
librbd1: 20.2.4-pve1
Ceph is running locally on the Proxmox cluster.

The RBD storage configuration is very simple:

Code:
rbd: nvme
content images,rootdir
krbd 0
pool nvme
CephX is enabled normally:

Code:
[global]
auth_client_required = cephx
auth_cluster_required = cephx
auth_service_required = cephx

The cluster itself is healthy from a Ceph connectivity/authentication perspective:

Code:
3 MONs in quorum
3 MGRs
3 MDSs
6/6 OSDs up/in
193 PGs active+clean

We also completed the Ceph 20.2.4 AES-CBC migration and the cluster now uses:

Code:
auth_service_cipher   aes256k
auth_allowed_ciphers  aes256k
auth_preferred_cipher aes256k

All CephX entity keys were rotated to aes256k.

The problem appears when Proxmox performs user-space rbd operations.

For example, deleting a VM fails with:

WARN: Could not remove disk 'nvme:vm-112233-cloudinit', check manually:
rbd error: rbd: unrecognised option '--auth_supported'

WARN: Could not remove disk 'nvme:vm-112233-disk-0', check manually:
rbd error: rbd: unrecognised option '--auth_supported'

WARN: Could not remove disk 'nvme:vm-112233-disk-1', check manually:
rbd error: rbd: unrecognised option '--auth_supported'

WARN: Could not remove disk 'nvme:vm-112233-disk-2', check manually:
rbd error: rbd: unrecognised option '--auth_supported'

TASK ERROR: rbd error: rbd: unrecognised option '--auth_supported'
The same happens with:

pvesm list nvme
I reproduced the issue directly:

Bash:
/usr/bin/rbd \
-p nvme \
-c /etc/pve/ceph.conf \
-n client.admin \
--keyring /etc/pve/priv/ceph/nvme.keyring \
--auth_supported cephx \
ls

Result:

rbd: unrecognised option '--auth_supported'
The same command without --auth_supported works.

I traced this to the installed Proxmox RBD plugin:

/usr/share/perl5/PVE/Storage/RBDPlugin.pm
Around lines 128–129:

Code:
push @$cmd, '--auth_supported', $cmd_option->{auth_supported}
    if ($cmd_option->{auth_supported});
PVE::CephConfig::ceph_connect_option() derives:

Code:
$cmd_option->{auth_supported}
    = (defined $cmd_option->{keyring}) ? 'cephx' : 'none';
so when a keyring exists, Proxmox generates:

--auth_supported cephx
for standalone /usr/bin/rbd calls.

Interestingly, authsupported still exists in RBDPlugin.pm under properties():

Code:
authsupported => {
description => "Authsupported.",
type => 'string',
},

but it is no longer present in options():

Code:
sub options {
return {
nodes => { optional => 1 },
disable => { optional => 1 },
monhost => { optional => 1 },
pool => { optional => 1 },
'data-pool' => { optional => 1 },
namespace => { optional => 1 },
username => { optional => 1 },
content => { optional => 1 },
krbd => { optional => 1 },
keyring => { optional => 1 },
bwlimit => { optional => 1 },
};
}

Therefore this does not work:

pvesm set nvme --authsupported ""
and returns:

Code:
Unknown option: authsupported
400 unable to parse option

Older Proxmox documentation did expose --authsupported, but current PVE 9 no longer does.

I also verified there are no newer storage packages available in the configured repositories:

libpve-storage-perl 9.1.8
is currently the latest candidate.

So the issue appears to be:

Markdown (GitHub flavored):
PVE::Storage::RBDPlugin
        |
        | generates
        v
rbd ... --auth_supported cephx
        |
        v
Ceph Tentacle 20.2.4 rbd
        |
        +--> unrecognised option '--auth_supported'

The QEMU/librbd paths appear separate and still use modern constructs such as auth-client-required, so the issue seems specifically related to the standalone rbd CLI command builder.

I have intentionally not patched RBDPlugin.pm, replaced /usr/bin/rbd with a wrapper, downgraded Ceph, or disabled CephX.

Questions:
  1. Is this a known issue with libpve-storage-perl 9.1.8 and Ceph Tentacle 20.2.4?
  2. Is there already an upstream patch removing --auth_supported from standalone rbd CLI calls?
  3. Is there a supported configuration option in PVE 9 to suppress this argument that I’m missing?
  4. If not, what is the recommended temporary workaround until an updated libpve-storage-perl package is available?
I can provide additional debug output or test a proposed patch if useful.

Thanks
UHL Team
 
Last edited:
  • Like
Reactions: wellerbm