ceph

Hi Everyone After upgrading ceph from 20.2.1 to 20.2.2 I'm getting the following error. Ceph 20.2.1 was working fine. The error is preventing new drives from being recognised as available. Other than that, ceph appears to be working correctly. Proxmox 9.2.4 with all updates. Ceph was installed...

Hello, We are experiencing random fencing on our 4 PVE nodes + 1 node tie-breaker (stretched cluster). It happened for the third time. Our topology: - DC1: Node 1 and Node 2 - DC2: Node 3 and Node 4 - DC3: Node 5. This node acts as a tie-breaker. It is connected via a VPN because it is our...

Posting this in case it helps others on PVE 9.2 + Ceph Tentacle, and to ask whether the underlying behavior is a bug. Full disclosure, I heavily relied on AI to troubleshoot and resolve (albeit with a workaround) and draft this post. After a cluster-wide cold boot, all my LXC containers backed...

Hello Proxmox community, I would like to ask for a technical clarification regarding the use of Ceph in a Proxmox VE architecture with only two storage nodes. We are currently evaluating a virtualization infrastructure where the target architecture is based on: 2 Proxmox VE nodes used for...

I'm rebuilding my cluster and I'd like to take a stab at Ceph. Here's what I Have to work with: 6 servers, all dual-xeons with at least 6 cores (24 threads). min 128GB ram. Disks are where things get weird: 7 x 3TB platter (21tb) 5 x 2TB platter (10tb) 5 x 2TB platter (10tb) 1 500GB platter +...

After being advised to use the remote-migration feature to keep QEMU and LXC templates in sync between our three clusters, I have implemented additional remote-migration tasks in our Ansible playbook which is already capable of creating and deploying the OS templates within our primary cluster...

Our team has a 3 nodes Proxmox cluster. Every night we backup all of the VMs. Every 2-3 days the nightly backup completely freezes one of the nodes. Sometimes the freezing can be stopped by stopping the backup job from another node, but complete freezing is the usual, in that case the Backup...

I have a PVE 8.3 cluster with 3 nodes ( dell R450 ), using pveceph for storage. Recently one node was down( hardware issue) ,then I recovered the system and wanted to re-install the PVE8.3 but I don't find PVE8.3 ISO on the website. I just download a PVE8.4 ISO and installed. Now the new...

Bonjour all, I'll be retiring 3 Dell 650XS from their current jobs soon™, and I would like to put them in a CEPH HA NVMe backed cluster. Currently, those servers are running a SATA/SAS backplane with a PERC H755 running the show. My plan would be to replace that with a NVMe backplane and a...

Hi, i'm workin with the api in the proxmox to get the available space we had in differents pools, but i see some curious thing with the api I get 9.6 tb availble but in the web interface show me with a simple substraction the space availble is 10 tb approximately, with gemini explain that is...

Hi, After all of the CVE's and security disclosures over the past week or two, I thought it'd be useful to upgrade my Proxmox nodes to the latest. I recently updated everything to PVE 9 & Ceph 19 and it has been working without fault for a few weeks. Since todays package updates, I am getting...

Have a cluster where one of the nodes needed pve reinstalled. Have it rejoined to the cluster, SDN is all healthy, interfaces and pinning restored but now I just have ceph left. From the node menu, I install Ceph with the gui, using the same version; on the configuration page it stated...

I experience the same issue mentioned here: https://forum.proxmox.com/threads/ceph-client-still-using-luminous-while-on-squid.165370/ and here: https://forum.proxmox.com/threads/unable-to-mount-cephfs-when-set-require-min-compat-client-squid.159502/ (both of which seem to have remained...

Hi everyone, During testing we are seeing behaviour we haven't seen mentioned elsewhere, but hope somebody here might have ideas on how to solve it. Proxmox is connected to an external 5-node ceph cluster with each node having 3 HDD OSDs using DB/WAL on SSD. Proxmox is running version 8, while...

so...been using proxmox for many years (almost 20). have several existing server clusters. Just building a new server stack. 6x Dell R760 768G ram, dual 32 core cpu. each server with 11 850G 24GPS SA SCSI SSD, 4x 1G nic plus 2X dual 100G nic The drives are 3,400 MB/s capable Build cluster...

I want to connect my PVE datacenter to an external Ceph storage. On a pool on the external Ceph cluster, I created a namespace. Let's say my pool is called `testpool`, my namespace is called `testspace` and my user is called `testuser`. So what i did on my Ceph cluster is rbd namespace create...

Yesterday I was greeted with numerous unreachable services stemming from a Ceph health error on our VM cluster due to "1 full osd(s)" and "1 backfillfull osd(s)" resulting in "4 pool(s) full" and solved it. This was the ceph status panel: As a result, our Ceph ended up in read-only state. The...

Hello, I have an 3 node full mesh ceph cluster with 4x 100 GE NIC per Node. With the SDN stack, I use openfabric for the NICs and on top ceph. For corosync, there are 2 dedicated NICS via switch (no lacp, bonding/teaming), each corosync link is in his own small network. VM traffic and management...

Hi everyone, we are using Proxmox VE with Ceph and were wondering how much storage we should use from our available storage before ceph starts acting up. Our Ceph Cluster consists of three nodes that each contribute a capacity of 27,94TB, so we reach a total of 83,82TB We were wondering now...

Hi all, I’m putting together a Proxmox cluster with Ceph for HA and wanted to get some feedback before I go ahead and deploy everything. What I’m aiming for is fairly simple: I want proper HA with no data loss and automatic failover, but at the same time I’d still like one node (an R640) to...