ceph

Hi everyone, I'm currently running a lab with four ESXi hosts. Only two of them have local storage capabilities, each with 2 x 4 TB drives and 2 x 128 GB drives. The other two hosts don't have a disk backplane. At the moment, storage is provided by two DataCore servers (in HCI mode) and...

Forgive me if this question is asked and answered often. I've tried reading the forum and manuals, and asking AI, and am more confused than ever. I come from a VxRail background. That technology uses vSphere and vSAN. Shutting down a node requires migrating running VMs elsewhere (either...

Hello, I am currently in the process of migrating a 5-node cluster running Proxmox 8 to version 9, and I am encountering some difficulties with Ceph. I am currently migrating the third node and am having trouble creating its monitor; across the entire cluster, Ceph remains stuck in a leader...

Hi Guys, on my pretty good Pi5 running Pi5OS Debian 1:6.18.50-1+rpt1 (2026-09-11) aarch64 , Kernel 6.18.50+rpt-rpi-2712 Ceph cannot create OSDs. Here is my lsblk: NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS loop0 7:0 0 2G 0 loop sda 8:0 0 1.8T 0 disk...

Hello, Anyone has some practical docs on network connectivity required for ceph rbd replication? Let's assume I have a dedicated lacp group for internal cluster communication (all the private traffic+ceph client net there) and another one for "production traffic and WAN" in two clusters in two...

With a Ceph cluster that has minimum client compatibility too new for krbd: root@pve01:~# ceph osd get-require-min-compat-client squid Booting a VM with a vTPM whose state is stored on rbd fails, even though PVE is configured to use librbd: Oct 06 17:23:47 pve03 kernel: libceph: mon2...

Hi, I'm following the instructions to make the aes -> aes256k ceph key migration. I've a 3 node cluster, with all the updates installed. Everything went fine until I had to run the last step: /usr/share/pve-manager/migrations/pve-cephx-rotate-service-keys INFO: This is a dry run. No Ceph key...

Hi! I'm facing an issue with mounting CephFS filesystems after the upgrade to latest version of Ceph Tentacle (20.2.4). From Proxmox perspective everything seems fine (on the UI and also manually):  root on vmserver01: /usr/share/pve-manager/migrations/pve-cephx-rotate-service-keys INFO: This...

I have a 6-node [pve2 ... pve9] PVE cluster [base versions: pvemgr 9.1.4, ceph 19.2.3] and I have started updating the nodes. After pve6 node upgrade two small OSDs failed to start back up, I destroyed them [as they would have been removed soon anyway]. I then upgraded pve9 [to...

When we first upgraded to Proxmox v9 and Ceph v19 we changed the fragmentation threshold from 8 to 9 to mute a bunch of health warnings. But now an OSD is actually over 0.9, which according to some sources could be considered serious. And of course it sets our entire cluster into HEALTH_WARN...

Hi, I’m running into what looks like a compatibility issue between current Proxmox VE 9.2 storage code and Ceph Tentacle 20.2.4. Environment: Proxmox VE: 9.2 pve-manager: 9.2.11 libpve-storage-perl: 9.1.8 qemu-server: 9.2.6 Ceph: 20.2.4-pve1...

Hi everyone, I would like to share a new project with you: Ceph Dashboard for Proxmox VE. As many of you know, Proxmox VE offers excellent integration for Ceph Storage. However, certain advanced features(S3) of the official Ceph Dashboard are not directly available within the Proxmox Web...

https://www.openwall.com/lists/oss-security/2026/08/19/4 https://www.openwall.com/lists/oss-security/2026/08/20/20 ;tldr There are flaws in the currently used authentication mechanism for CephX. Versions 19.2.6 and 20.2.4 introduce new algorithms to CephX that require a rotation of the...

Hi, I have a PVE cluster with 6 nodes : 2 regular nodes for virtualization (using their RAM/CPU for running VM), and 4 nodes for storage only (Ceph). I'd like to exclude the "ceph" nodes from the migration targets. I don't want any VM to run (RAM/CPU) on those nodes. Is there a way to disable...

Hi all, We are planning to set up a 4-node Proxmox cluster with Ceph across 2 datacenters. Our setup will be: DC1: Node 1 and Node 2 DC2: Node 3 and Node 4 Node 1 and 2 are using one network range. Node 3 and 4 are using a different network range. Both datacenters are connected through an...

What can I expect on a CEPH Datastore? - 5 DELL R740 Nodes - 60pcs 1,92 TB SAS SSD mixed-use - Dual 100Gbps Link exclusive for CEPH

Hallo, wir sind bei unserem Cluster auf ein Problem gestoßen. Mit dem neusten Sicherheitsupdate von Microsoft verlieren unsere Windows 2025 RDS Server ihr TPM Modul. TPM.MSC zeigt an, dass kein Modul verbunden wäre. Vor dem Update lief alles normal. Ich habe mit Hilfe von Claude eine Diagnose...

Hi Everyone After upgrading ceph from 20.2.1 to 20.2.2 I'm getting the following error. Ceph 20.2.1 was working fine. The error is preventing new drives from being recognised as available. Other than that, ceph appears to be working correctly. Proxmox 9.2.4 with all updates. Ceph was installed...

Hello, We are experiencing random fencing on our 4 PVE nodes + 1 node tie-breaker (stretched cluster). It happened for the third time. Our topology: - DC1: Node 1 and Node 2 - DC2: Node 3 and Node 4 - DC3: Node 5. This node acts as a tie-breaker. It is connected via a VPN because it is our...

Posting this in case it helps others on PVE 9.2 + Ceph Tentacle, and to ask whether the underlying behavior is a bug. Full disclosure, I heavily relied on AI to troubleshoot and resolve (albeit with a workaround) and draft this post. After a cluster-wide cold boot, all my LXC containers backed...