ceph

HI, Recently we performed an upgrade on our proxmox cluster and we faced an outage. we have 4 node clusters, after completed the upgrade in 3 rd node, we tried to reboot node 3, but what happened is the whole cluster, which means all nodes went rebooted. We suffered a lot due to the downtime...

As part of upgrading to proxmox 7 i done the folling steps: upgraded all nodes to latest 6.4-13 rebooting everything upgrading cepth to Octopus , based on (https://pve.proxmox.com/wiki/Ceph_Nautilus_to_Octopus) after running ( ceph osd pool set POOLNAME pg_autoscale_mode on) on a pool it...

Hello! I have a 10 nodes hyperconverged cluster. Each node has 4 OSDs (total 40 OSDs). I have 2 different questions: QUESTION 1 - OSD REPLACEMENT (with identical SSD) Since I need to replace an SSD (one OSD has crashed 3 times in latest months so I prefer to replace it with a brand new one)...

TL;DR - Upgrade from 16.2.5 to 16.2.6 - CEPHFS fails to start after upgrade, all MDS in "standby" - requires ceph fs compat <fs name> add_incompat 7 "mds uses inline data" to work again. Longer version : pve-manager/7.0-11/63d82f4e (running kernel: 5.11.22-5-pve) apt dist-upgraded, CEPH...

Hi. I am running Proxmox 7.0-11 with CEPH version 16.2.6. I have some trouble with the GUI Datacenter -> Ceph -> Services. I see a few services marked as unknown for some reason that is related to CEPH.. See screenshot below. I've tried systemctl restart pveproxy on all nodes and clearing my...

Hi, we are using CephFS on a 3-node proxmox cluster. We have mounted the CephFS to /home on several different Debian clients. All Debian clients (server) see the files of the other Debian clients in the cephfs mount (/home). It happens that client XY has services on Debian client 1 and Debian...

Hi, we would update our pve-cluster to pve7 and like to reduce the reboots to one for each node. Ceph is allready on 14.2.22. The howto say: "We assume that all nodes are on the latest Proxmox VE 6.3 (or higher)", which isn't on most nodes in our cluster. Is it still posible to upgrade ceph to...

Hi everyone! I'm having this strange issue with Ceph on my Proxmox cluster. The *ceph* nodes have OSD, mon, mds and mgr on each of them. The *pve* nodes have only OSD on them. Recently every Saturday exactly at 06:00 (±couple minutes) ceph goes to Warning state. Logs show that one of the tree...

Hi everyone, I have a simple 3 node cluster that has always worked for many years and successfully passed the updates starting from proxmox 4. After updating to version 7 of proxmox and pacific ceph, the system is affected by this issue: every time I reboot a node for any reason (ie updating to...

Hi everyone, I have setup a 3 nodes Proxmox 7.0-11 cluster, each node has 3 HDD drives, the first drive is used for Proxmox and the others are OSDs. I wanted to test HA by removing the last disk of the first node (which doesn't contain Proxmox), but when I remove it the node reboot. Is this...

I am trying to install the ceph in 2 nodes of the cluster to make a mirror node, and when the ceph finishes the OSD disk storage is not created, when creating it manually it gives me the following error, what can be happening? I need help please and thank you very much for your attention

Hello, I decided to use the functionality of proxmox to support only CEPH without any VE for RBD block. I installed the "Proxmox Virtual Environment" option for that experiment. I notice that the system run a lot of things responsible for e.g. VE replication (pvest). What do you recommend to...

I was setting up ceph on my dev cluster but I accidently set ceph to use nmbr0 instead of vmbr1. Is their a way I can change that? Is their a way I can confirm that I am using the wrong bridge instead of looking at the ips?

New to Ceph on PVE, have a couple of questions. Is it possible to setup one large Ceph cluster that is dedicated to just running Ceph, but using PVE to set it up and manage it? If so, can I then present this ceph cluster to multiple clusters of PVE? End goal here is one large ceph cluster...

Hi all, Today out of the blue my ceph cluster had all clients disconnected. Ceph dashboard still shows healthy (lies), but proxmox shows both my vm storage (based on rdb) and cephfs as in an "unknown state". When I started to dive in further, I found that on every node, ceph health hangs. osd's...

I have noticed that the CEPH public / private network weren't properly created during the execution of the Ceph installation wizard. Unfortunately reloading the page killed the process and I can't seem to find a way to relaunch the Ceph installation wizard ? Any idea how I could do that, I'd...

Hello, I'm looking for help on below points 1) Installing and configuring the RADOS Gateway 2) How to create a RADOS Gateway user for the S3 access 3) Configuring DNS & S3 client to access ceph object storage. Update: I followed steps given in below post but not able to access gateway on port...

Good morning everyone, I have a cluster of 3 Proxmox servers under version 6.4-13. Last Friday I updated Ceph from nautilus to octupus, since it is one of the requirements to upgrade Proxmox to version 7. At first everything worked wonders. But today when I check I find that it is giving me the...

Not maxing out ceph performance? I got a three node AMD epic 7702 installed in a Gigabyte R272-Z31 hyper converged proxmox cluster running. With a ceph storage cluster. The setup is as described in https://pve.proxmox.com/wiki/Deploy_Hyper-Converged_Ceph_Cluster. There is a separate ceph...

Hi We are in the process of migrating services from servers running a local ZFS-mirror on 4TB nVME SSD drives. We are migrating to new servers with the same specifications and hardware as what is currently running, but instead of spreading services over two nodes (a 3rd node runs some internal...