ceph cluster

I have a 3 node Ceph cluster running Proxmox 7.2. Each node has 4 x HDD OSDs and the 4 OSDs share an Intel Enterprise SSD for the Ceph OSD database (DB/WAL) on each node. I am going to be adding a 5th OSD HDD to each node and also add an additional Intel Enterprise SSD on each node for use with...

Hi All, I have been running a 5 nodes of proxmox for a while now, I have installed ceph and come to configure ceph pool i need two pool one for VMs and other for containers. i need help with this configuration for those pool size Min size num of PGs" as from pgcalc it said to put it 512 and from...

Hi all! I really hope someome could guide me with this issue because I do not know cepth much and I am not sure about the next steps.. So I have a 3 nodes cluster, same hardware. ceph was working without any issue until a reboot I had to do because of a network issue. Now on a node, ceph is...

Hello, i want to change the ceph config and use the bridge instead of the vlans I have created. But when I change the ceph.conf file with the IP network from bridge, my ceph doesn't work anymore. Any suggestions how I can do that? My main issue is that I am not getting enough read/write on the...

Hello, I'm looking over the Proxmox documentation for building a Ceph cluster here... https://pve.proxmox.com/wiki/Deploy_Hyper-Converged_Ceph_Cluster There is a small section entitled block.db and block.wal which says... I was wondering if anyone knows how much of a performance advantage...

Hallo zusammen, ich wollte mich nun einmal mit dem Thema Ceph in Proxmox vertraut machen und habe mir für diesen Zweck einmal einen kleinen 3 Node Cluster für Testzwecke gebaut. Auf diesem habe ich pro Node 2x SSDs für Ceph verbaut (gesamt 3x2). Das Ceph Netzwerk habe ich einmal in Public und...

Hi guys! I experimented with ceph behavior when 1 of the disks fails. The disk has been "hot" removed from the server. After a couple of days, the disc was inserted back. The disk status was down/out. I clicked the IN button, but the down status didn't change. The server has been rebooted, but...

Hello all, We are running a cluster of 5 nodes in our environment. We have a CEPH storage managed in the cluster , for each node we are having 4 OSD's so basically 20 OSD's Each node is a bare metal with proxmox ve-6.4-8 configured with CPU(s) 72 x Intel(R) Xeon(R) CPU E5-2699 v3 @...

Hi guys, During the free time I had to think how to extend and add new resources (storage) to our Cloud. For the moment I do have stroage base on Ceph - OSD only SSD type. I was reading docs about Ceph and I can say it's possbile I have even the plan. Problem is I have no idea if the actions I...

After spending some time learning that SAN is the traditional storage route and that Ceph + "hyperconverged" is actually encouraged now that we have the technology, I am a little lost on how to proceed with my home lab. I don't know if using Ceph would be like trying to shoe horn a new...

Hi, on my test-cluster i upgraded all my nodes from 7.0 to 7.1 CEPH went to pacific 16.2.7 (was 16.2.5?) Now the monitors and managers won't start. I had a pool and cephFS configured with MDS. I've read somewhere that a pool in combination with an old cephFS (i came from PVE6) it could...

Hello Everyone, For quite some time I have been looking into building myself a server cluster to run my Virtual Environment that will host various types of services both internally within my homelab and some that are externally accessible. I have been looking into using ProxMox to achieve this...

Hello I am new to using Proxmox and Ceph. I recently installed a homelab using 3 nodes. 3 cpu 4 cores each socket. It was nice to have a small cluster at my house. yay. So the ceph storage is distributed across all 3 nodes. I have set it up as RBD. so is RBD (block device) a way to spread the...

Good morning everyone, I have a cluster of 3 Proxmox servers under version 6.4-13. Last Friday I updated Ceph from nautilus to octupus, since it is one of the requirements to upgrade Proxmox to version 7. At first everything worked wonders. But today when I check I find that it is giving me the...

Hallo zusammen, ich lese mich gerade in das Upgrada PVE 6 nach 7 ein bzw. in die Vorbereitung des Ceph-Upgrades. Ich habe hier einen Cluster mit 3 Knoten und das ist jetzt mein erstes Proxmox/Ceph-Upgrade Kann ich das Ceph Upgrade wie im Wiki beschrieben bei laufenden VMs machen? Soweit ich...

Hello everyone, We have a proxmox cluster of 4 nodes with Ceph FS installed on all of them. Decided to upgrade all servers one-by-one to Proxmox 7 as well as Ceph Pacific. After upgrading the second one, Ceph became unresponsive (all ceph related commands get timed out on all nodes also not...

Hi Everyone, Getting into using the CEPH RBD command because I need to identify storage usage in my CEPH cluster. When issuing the command below, it almost appears as though my snapshots are using an immense amount of space: rbd -p pool du Excerpt of one VM...

Hello all, setting up a new 5 node cluster with the following identical specs for each node. Been using proxmox for many years but am new to ceph. I spun up a test environment and it has been working perfectly for a couple months. Now looking to make sure we are moving the right direction with...

Greetings Proxmox community, Since many of you are familiar with Ceph, I hope someone can help me with this issue. We have a 3 node Ceph cluster (ceph-node1, ceph-node2 and ceph-node3) as the main storage for our Proxmox VMs. We had a power outage a few days ago, and after that ceph-node1 and...

Hi, I am facing intermittent clock skew issues on my 3 node proxmox+ceph cluster. clock skew detected on mon.proxmox3 mon.proxmox3 clock skew 0.0609687s > max 0.05s (latency 0.000392331s) pasted few information about my current configuration below.. systemctl status...