ceph

After being advised to use the remote-migration feature to keep QEMU and LXC templates in sync between our three clusters, I have implemented additional remote-migration tasks in our Ansible playbook which is already capable of creating and deploying the OS templates within our primary cluster...

Our team has a 3 nodes Proxmox cluster. Every night we backup all of the VMs. Every 2-3 days the nightly backup completely freezes one of the nodes. Sometimes the freezing can be stopped by stopping the backup job from another node, but complete freezing is the usual, in that case the Backup...

I have a PVE 8.3 cluster with 3 nodes ( dell R450 ), using pveceph for storage. Recently one node was down( hardware issue) ,then I recovered the system and wanted to re-install the PVE8.3 but I don't find PVE8.3 ISO on the website. I just download a PVE8.4 ISO and installed. Now the new...

Bonjour all, I'll be retiring 3 Dell 650XS from their current jobs soon™, and I would like to put them in a CEPH HA NVMe backed cluster. Currently, those servers are running a SATA/SAS backplane with a PERC H755 running the show. My plan would be to replace that with a NVMe backplane and a...

Hi, i'm workin with the api in the proxmox to get the available space we had in differents pools, but i see some curious thing with the api I get 9.6 tb availble but in the web interface show me with a simple substraction the space availble is 10 tb approximately, with gemini explain that is...

Hi, After all of the CVE's and security disclosures over the past week or two, I thought it'd be useful to upgrade my Proxmox nodes to the latest. I recently updated everything to PVE 9 & Ceph 19 and it has been working without fault for a few weeks. Since todays package updates, I am getting...

Have a cluster where one of the nodes needed pve reinstalled. Have it rejoined to the cluster, SDN is all healthy, interfaces and pinning restored but now I just have ceph left. From the node menu, I install Ceph with the gui, using the same version; on the configuration page it stated...

I experience the same issue mentioned here: https://forum.proxmox.com/threads/ceph-client-still-using-luminous-while-on-squid.165370/ and here: https://forum.proxmox.com/threads/unable-to-mount-cephfs-when-set-require-min-compat-client-squid.159502/ (both of which seem to have remained...

Hi everyone, During testing we are seeing behaviour we haven't seen mentioned elsewhere, but hope somebody here might have ideas on how to solve it. Proxmox is connected to an external 5-node ceph cluster with each node having 3 HDD OSDs using DB/WAL on SSD. Proxmox is running version 8, while...

so...been using proxmox for many years (almost 20). have several existing server clusters. Just building a new server stack. 6x Dell R760 768G ram, dual 32 core cpu. each server with 11 850G 24GPS SA SCSI SSD, 4x 1G nic plus 2X dual 100G nic The drives are 3,400 MB/s capable Build cluster...

I want to connect my PVE datacenter to an external Ceph storage. On a pool on the external Ceph cluster, I created a namespace. Let's say my pool is called `testpool`, my namespace is called `testspace` and my user is called `testuser`. So what i did on my Ceph cluster is rbd namespace create...

Yesterday I was greeted with numerous unreachable services stemming from a Ceph health error on our VM cluster due to "1 full osd(s)" and "1 backfillfull osd(s)" resulting in "4 pool(s) full" and solved it. This was the ceph status panel: As a result, our Ceph ended up in read-only state. The...

Hello, I have an 3 node full mesh ceph cluster with 4x 100 GE NIC per Node. With the SDN stack, I use openfabric for the NICs and on top ceph. For corosync, there are 2 dedicated NICS via switch (no lacp, bonding/teaming), each corosync link is in his own small network. VM traffic and management...

Hi everyone, we are using Proxmox VE with Ceph and were wondering how much storage we should use from our available storage before ceph starts acting up. Our Ceph Cluster consists of three nodes that each contribute a capacity of 27,94TB, so we reach a total of 83,82TB We were wondering now...

Hi all, I’m putting together a Proxmox cluster with Ceph for HA and wanted to get some feedback before I go ahead and deploy everything. What I’m aiming for is fairly simple: I want proper HA with no data loss and automatic failover, but at the same time I’d still like one node (an R640) to...

Hallo zusammen, derzeit betreibe ich ein Proxmox Cluster mit Ceph als Shared Storage. Hardware pro Node: i5-12500 64GB DDR5 RAM 3x 1,92TB PM883 SSD LACP 2x 10Gbit für Cluster Network LACP 2x 10Gbit für VM Network Bei Benchmarks des Storages bekomme ich maximale Schreibwerte von rund...

Dear all, I am running PVE in a what i consider being a rather typical "IT nerd" setup, a 3-node-Cluster on consumer hardware. Private use only. 2 nodes are built identically: AMD 3200G (2C/4T) APU, 32GB DDR4 RAM, M2 NVMe for OS, 2x4TB HDD for storage (connected to SATA onboard), GbE NIC 1...

Hello, I have several Proxmox clusters of 3 dedicated servers each. Previously, I used GlusterFS to replicate the VM disks between the three servers in the cluster so that if one server went down, the VMs would restart on the two remaining servers without data loss. This method worked very...

I've got a working 6 nodes stretched Proxmox VE9.1 + Ceph cluster, with hosts split over two datacenters (3 in each). In a third datacenter I've got a Proxmox VE 9.1 virtual machine (on vsphere) which acts as proxmox + ceph tie-breaker. Each host has 4x25 Gbits/s interfaces in LACP bond defined...

Hello everyone, I have spent a lot of time to figure out what is provoking that IO wait. on this cluster VM who do a high amount of IO have a lot of IO wait ( like 30k I/O read, 50% IO wait) Summary of my setup: PVE proxmox-ve: 9.0.0 (running kernel: 6.17.2-2-pve) pve-manager: 9.0.18...