I'm in the process of transferring VMs and disk images from an old Proxmox/Ceph cluster to new* ones, and I've come up against a surprisingly high amount of IOWait when the VMs are running in the new clusters. Wait - don't run off... I'm not complaining about *that*! 
Whilst looking into this, I found that the "old" cluster only has
In order to test if any of these extra features are causing my problem, I disabled
I've set
but also to no avail. So my question is - is it possible to disable these features on a single image "fully", for testing?
* new Ceph cluster:
new Proxmox cluster:
Whilst looking into this, I found that the "old" cluster only has
layering enabled on RBD images whilst the "new" one also has exclusive-lock, object-map, fast-diff, deep-flatten enabled.In order to test if any of these extra features are causing my problem, I disabled
object-map, fast-diff, deep-flatten via the rbd tool on the command line. However, on restarting the VM the object-map, fast-diff features are automatically re-enabled on the image. I've set
rbd_default_features = 1 in the [global], [client] and [client.admin] sections of the running ceph.conf for the Ceph cluster (which seems to stop images being *created* with deep-flatten), and the ${storage}.conf in /etc/pve/priv/ceph on the Proxmox VE clients. I've also tried commenting out the feature-enable lines in /usr/share/perl5/PVE/Storage/RBDPlugin.pm :
Code:
lines 159-169:
#if ($to_enable) {
# print "enable RBD image features this kernel RBD drivers supports: $to_enable\n";
# eval {
# my $cmd = $rbd_cmd->($scfg, $storeid, 'feature', 'enable', $name, $to_enable);
# run_rbd_command(
# $cmd,
# errmsg => "could not enable krbd-compatible image features '$to_enable' for rbd image: $name",
# );
# };
# warn "$@" if $@;
#}but also to no avail. So my question is - is it possible to disable these features on a single image "fully", for testing?
* new Ceph cluster:
Code:
root@ceph-sa-1-1:~# ceph version
ceph version 18.2.2 (e9fe820e7fffd1b7cde143a9f77653b73fcec748) reef (stable)
root@ceph-sa-1-1:~# pveversion
pve-manager/8.2.7/3e0176e6bb2ade3b (running kernel: 6.8.12-2-pve)
Code:
root@compute-m-c3n1:/etc/ceph# pveversion
pve-manager/8.2.7/3e0176e6bb2ade3b (running kernel: 6.8.12-2-pve)
root@compute-m-c3n1:/etc/ceph# ceph version
Error initializing cluster client: ObjectNotFound('RADOS object not found (error calling conf_read_file)')
root@compute-m-c3n1:/etc/ceph# apt-show-versions | grep ceph | head -n1
ceph:amd64/bookworm 18.2.2-pve1 upgradeable to 18.2.4-pve3