Please help: I can no longer clone a VM due to a keyring error, and I'm not sure what has gone wrong.
Has there been a change the location or method for the ceph client keyring storage?
Some time within the last month, during which I definitely did a few updates, I am no longer able to clone a VM because of a keyring error when accessing my Ceph volumes.
Specifically:
If I'm understanding this correctly, it appears that ptxpool01.keyring cannot be found. I have not (intentionally) changed any of my ceph config, which looks like this:
I've looked through the forums and seen references that ceph can't read /etc/pve/priv/, but then how was this configured this way in the first place? I set up ceph through the GUI, and everything was working fine last month, including cloning.
The permissions of ptxpool01.keyring are:
which looks suspicious because www-data is listed as the group but not allowed any permissions, so what is the point of that?
Just to test, I tried changing permissions to 640, but even as root changing permission on that file is not permitted.
Bottom line, how do I fix my config so that I can clone a VM again? Move the keyring to /var/lib/ceph/mds/ ?
Also wanted to bring this up here in case anyone else was having this problem, or there was an undocumented ceph config change. I"m running ceph reef 18.2.2.
Has there been a change the location or method for the ceph client keyring storage?
Some time within the last month, during which I definitely did a few updates, I am no longer able to clone a VM because of a keyring error when accessing my Ceph volumes.
Specifically:
Code:
TASK ERROR: clone failed: mirroring error: VM 201 qmp command 'drive-mirror' failed - Could not open 'rbd:ptxpool01/vm-128-disk-0:conf=/etc/pve/ceph.conf:id=admin:keyring=/etc/pve/priv/ceph/ptxpool01.keyring': No such file or directoryIf I'm understanding this correctly, it appears that ptxpool01.keyring cannot be found. I have not (intentionally) changed any of my ceph config, which looks like this:
Code:
[global]
auth_client_required = cephx
auth_cluster_required = cephx
auth_service_required = cephx
cluster_network = 10.10.200.201/24
fsid = ea5fc128-b7cc-4a88-8e7c-d73d7489f2e5
mon_allow_pool_delete = true
mon_host = 10.10.100.202 10.10.100.203 10.10.100.201
ms_bind_ipv4 = true
ms_bind_ipv6 = false
osd_pool_default_min_size = 2
osd_pool_default_size = 3
public_network = 10.10.100.201/24
[client]
keyring = /etc/pve/priv/$cluster.$name.keyring
[client.crash]
keyring = /etc/pve/ceph/$cluster.$name.keyring
[mds]
keyring = /var/lib/ceph/mds/ceph-$id/keyring
[mds.hv01]
host = hv01
mds_standby_for_name = pve
[mds.hv02]
host = hv02
mds_standby_for_name = pve
[mds.hv03]
host = hv03
mds_standby_for_name = pve
[mon.hv01]
public_addr = 10.10.100.201
[mon.hv02]
public_addr = 10.10.100.202
[mon.hv03]
public_addr = 10.10.100.203I've looked through the forums and seen references that ceph can't read /etc/pve/priv/, but then how was this configured this way in the first place? I set up ceph through the GUI, and everything was working fine last month, including cloning.
The permissions of ptxpool01.keyring are:
-rw------- 1 root www-data 151 Jan 30 19:00 ptxpool01.keyringwhich looks suspicious because www-data is listed as the group but not allowed any permissions, so what is the point of that?
Just to test, I tried changing permissions to 640, but even as root changing permission on that file is not permitted.
Bottom line, how do I fix my config so that I can clone a VM again? Move the keyring to /var/lib/ceph/mds/ ?
Also wanted to bring this up here in case anyone else was having this problem, or there was an undocumented ceph config change. I"m running ceph reef 18.2.2.
Last edited:
