Hi,
after running the latest apt dist-upgrade and rebooting one of our storage servers, all OSD's on this node refuse to start.
After downgrading all CEPH packages the OSD's start immediately.
Any help appreciated
Setup: 7 x CEPH Storage nodes, 3 x PVE Compute Nodes
Log files of the nodes:
after running the latest apt dist-upgrade and rebooting one of our storage servers, all OSD's on this node refuse to start.
After downgrading all CEPH packages the OSD's start immediately.
Any help appreciated
Setup: 7 x CEPH Storage nodes, 3 x PVE Compute Nodes
Log files of the nodes:
Code:
2025-11-11T11:49:22.030+0100 7c8ce33e3940 0 set uid:gid to 64045:64045 (ceph:ceph)
2025-11-11T11:49:22.030+0100 7c8ce33e3940 0 ceph version 18.2.7 (4cac8341a72477c60a6f153f3ed344b49870c932) reef (stable), process ceph-osd, pid 8229
2025-11-11T11:49:22.030+0100 7c8ce33e3940 0 pidfile_write: ignore empty --pid-file
2025-11-11T11:49:22.032+0100 7c8ce33e3940 1 bdev(0x5dd69d9b1400 /var/lib/ceph/osd/ceph-81/block) open path /var/lib/ceph/osd/ceph-81/block
2025-11-11T11:49:22.032+0100 7c8ce33e3940 0 bdev(0x5dd69d9b1400 /var/lib/ceph/osd/ceph-81/block) ioctl(F_SET_FILE_RW_HINT) on /var/lib/ceph/osd/ceph-81/block failed: (22) Invalid argument
2025-11-11T11:49:22.032+0100 7c8ce33e3940 1 bdev(0x5dd69d9b1400 /var/lib/ceph/osd/ceph-81/block) open size 3200627245056 (0x2e934400000, 2.9 TiB) block_size 4096 (4 KiB) non-rotational device, discard supported
2025-11-11T11:49:22.033+0100 7c8ce33e3940 -1 bluestore(/var/lib/ceph/osd/ceph-81) _set_cache_sizes bluestore_cache_meta_ratio (0.8) + bluestore_cache_kv_ratio (0.2) + bluestore_cache_kv_onode_ratio (0.04) = 1.04; must be <= 1.0
2025-11-11T11:49:22.033+0100 7c8ce33e3940 1 bdev(0x5dd69d9b1400 /var/lib/ceph/osd/ceph-81/block) close
2025-11-11T11:49:22.306+0100 7c8ce33e3940 0 starting osd.81 osd_data /var/lib/ceph/osd/ceph-81 /var/lib/ceph/osd/ceph-81/journal
2025-11-11T11:49:22.322+0100 7c8ce33e3940 0 load: jerasure load: lrc
2025-11-11T11:49:22.322+0100 7c8ce33e3940 1 bdev(0x5dd69e761400 /var/lib/ceph/osd/ceph-81/block) open path /var/lib/ceph/osd/ceph-81/block
2025-11-11T11:49:22.322+0100 7c8ce33e3940 0 bdev(0x5dd69e761400 /var/lib/ceph/osd/ceph-81/block) ioctl(F_SET_FILE_RW_HINT) on /var/lib/ceph/osd/ceph-81/block failed: (22) Invalid argument
2025-11-11T11:49:22.323+0100 7c8ce33e3940 1 bdev(0x5dd69e761400 /var/lib/ceph/osd/ceph-81/block) open size 3200627245056 (0x2e934400000, 2.9 TiB) block_size 4096 (4 KiB) non-rotational device, discard supported
2025-11-11T11:49:22.323+0100 7c8ce33e3940 -1 bluestore(/var/lib/ceph/osd/ceph-81) _set_cache_sizes bluestore_cache_meta_ratio (0.8) + bluestore_cache_kv_ratio (0.2) + bluestore_cache_kv_onode_ratio (0.04) = 1.04; must be <= 1.0
2025-11-11T11:49:22.323+0100 7c8ce33e3940 1 bdev(0x5dd69e761400 /var/lib/ceph/osd/ceph-81/block) close
2025-11-11T11:49:22.596+0100 7c8ce33e3940 1 bdev(0x5dd69e761400 /var/lib/ceph/osd/ceph-81/block) open path /var/lib/ceph/osd/ceph-81/block
2025-11-11T11:49:22.596+0100 7c8ce33e3940 0 bdev(0x5dd69e761400 /var/lib/ceph/osd/ceph-81/block) ioctl(F_SET_FILE_RW_HINT) on /var/lib/ceph/osd/ceph-81/block failed: (22) Invalid argument
2025-11-11T11:49:22.597+0100 7c8ce33e3940 1 bdev(0x5dd69e761400 /var/lib/ceph/osd/ceph-81/block) open size 3200627245056 (0x2e934400000, 2.9 TiB) block_size 4096 (4 KiB) non-rotational device, discard supporte
d
2025-11-11T11:49:22.597+0100 7c8ce33e3940 -1 bluestore(/var/lib/ceph/osd/ceph-81) _set_cache_sizes bluestore_cache_meta_ratio (0.8) + bluestore_cache_kv_ratio (0.2) + bluestore_cache_kv_onode_ratio (0.04) = 1.0
4; must be <= 1.0
2025-11-11T11:49:22.597+0100 7c8ce33e3940 1 bdev(0x5dd69e761400 /var/lib/ceph/osd/ceph-81/block) close
2025-11-11T11:49:22.871+0100 7c8ce33e3940 1 mClockScheduler: set_osd_capacity_params_from_config: osd_bandwidth_cost_per_io: 499321.90 bytes/io, osd_bandwidth_capacity_per_shard 157286400.00 bytes/second
2025-11-11T11:49:22.871+0100 7c8ce33e3940 0 osd.81:0.OSDShard using op scheduler mclock_scheduler, cutoff=196
2025-11-11T11:49:22.873+0100 7c8ce33e3940 1 bdev(0x5dd69e761400 /var/lib/ceph/osd/ceph-81/block) open path /var/lib/ceph/osd/ceph-81/block
2025-11-11T11:49:22.873+0100 7c8ce33e3940 0 bdev(0x5dd69e761400 /var/lib/ceph/osd/ceph-81/block) ioctl(F_SET_FILE_RW_HINT) on /var/lib/ceph/osd/ceph-81/block failed: (22) Invalid argument
2025-11-11T11:49:22.873+0100 7c8ce33e3940 1 bdev(0x5dd69e761400 /var/lib/ceph/osd/ceph-81/block) open size 3200627245056 (0x2e934400000, 2.9 TiB) block_size 4096 (4 KiB) non-rotational device, discard supporte
d
2025-11-11T11:49:22.874+0100 7c8ce33e3940 -1 bluestore(/var/lib/ceph/osd/ceph-81) _set_cache_sizes bluestore_cache_meta_ratio (0.8) + bluestore_cache_kv_ratio (0.2) + bluestore_cache_kv_onode_ratio (0.04) = 1;
must be <= 1.0
2025-11-11T11:49:22.874+0100 7c8ce33e3940 1 bdev(0x5dd69e761400 /var/lib/ceph/osd/ceph-81/block) close
2025-11-11T11:49:23.147+0100 7c8ce33e3940 -1 osd.81 0 OSD:init: unable to mount object store
2025-11-11T11:49:23.147+0100 7c8ce33e3940 -1 ESC[0;31m ** ERROR: osd init failed: (22) Invalid argumentESC[0m
Last edited: