ceph cluster

Hi guys! I experimented with ceph behavior when 1 of the disks fails. The disk has been "hot" removed from the server. After a couple of days, the disc was inserted back. The disk status was down/out. I clicked the IN button, but the down status didn't change. The server has been rebooted, but...

Hello all, We are running a cluster of 5 nodes in our environment. We have a CEPH storage managed in the cluster , for each node we are having 4 OSD's so basically 20 OSD's Each node is a bare metal with proxmox ve-6.4-8 configured with CPU(s) 72 x Intel(R) Xeon(R) CPU E5-2699 v3 @...

Hi guys, During the free time I had to think how to extend and add new resources (storage) to our Cloud. For the moment I do have stroage base on Ceph - OSD only SSD type. I was reading docs about Ceph and I can say it's possbile I have even the plan. Problem is I have no idea if the actions I...

After spending some time learning that SAN is the traditional storage route and that Ceph + "hyperconverged" is actually encouraged now that we have the technology, I am a little lost on how to proceed with my home lab. I don't know if using Ceph would be like trying to shoe horn a new...

Hi, on my test-cluster i upgraded all my nodes from 7.0 to 7.1 CEPH went to pacific 16.2.7 (was 16.2.5?) Now the monitors and managers won't start. I had a pool and cephFS configured with MDS. I've read somewhere that a pool in combination with an old cephFS (i came from PVE6) it could...

Hello Everyone, For quite some time I have been looking into building myself a server cluster to run my Virtual Environment that will host various types of services both internally within my homelab and some that are externally accessible. I have been looking into using ProxMox to achieve this...

Hello I am new to using Proxmox and Ceph. I recently installed a homelab using 3 nodes. 3 cpu 4 cores each socket. It was nice to have a small cluster at my house. yay. So the ceph storage is distributed across all 3 nodes. I have set it up as RBD. so is RBD (block device) a way to spread the...

Good morning everyone, I have a cluster of 3 Proxmox servers under version 6.4-13. Last Friday I updated Ceph from nautilus to octupus, since it is one of the requirements to upgrade Proxmox to version 7. At first everything worked wonders. But today when I check I find that it is giving me the...

Hallo zusammen, ich lese mich gerade in das Upgrada PVE 6 nach 7 ein bzw. in die Vorbereitung des Ceph-Upgrades. Ich habe hier einen Cluster mit 3 Knoten und das ist jetzt mein erstes Proxmox/Ceph-Upgrade Kann ich das Ceph Upgrade wie im Wiki beschrieben bei laufenden VMs machen? Soweit ich...

Hello everyone, We have a proxmox cluster of 4 nodes with Ceph FS installed on all of them. Decided to upgrade all servers one-by-one to Proxmox 7 as well as Ceph Pacific. After upgrading the second one, Ceph became unresponsive (all ceph related commands get timed out on all nodes also not...

Hi Everyone, Getting into using the CEPH RBD command because I need to identify storage usage in my CEPH cluster. When issuing the command below, it almost appears as though my snapshots are using an immense amount of space: rbd -p pool du Excerpt of one VM...

Hello all, setting up a new 5 node cluster with the following identical specs for each node. Been using proxmox for many years but am new to ceph. I spun up a test environment and it has been working perfectly for a couple months. Now looking to make sure we are moving the right direction with...

Greetings Proxmox community, Since many of you are familiar with Ceph, I hope someone can help me with this issue. We have a 3 node Ceph cluster (ceph-node1, ceph-node2 and ceph-node3) as the main storage for our Proxmox VMs. We had a power outage a few days ago, and after that ceph-node1 and...

Hi, I am facing intermittent clock skew issues on my 3 node proxmox+ceph cluster. clock skew detected on mon.proxmox3 mon.proxmox3 clock skew 0.0609687s > max 0.05s (latency 0.000392331s) pasted few information about my current configuration below.. systemctl status...

Hello, we have a fresh 3-node PVE-CEPH-Cluster running PVE 6.4-4 and we have done some redundancy Tests, yesterday. Each Node has the following separated Networks: 1GBit/s Port - > GUI VLAN (routed) -> 10.1.0.0/27 1GBit/s Port - > Corosync VLAN (only L2) -> 10.1.1.0/27 10GBit/s Port CEPH...

Hello, A few information about the System: Its a Hyperconverged Cluster of 5 Supermicro AS -1114S-WN10RT: 4 of the Servers have: CPU: 128 x AMD EPYC 7702P 64-Core Processor (1 Socket) RAM: 512 GB 1 of the Servers has: 64 x AMD EPYC 7502P 32-Core Processor (1 Socket) RAM: 256 GB Network: All...

We recently purchased three small servers and need advice on the network setup. What would be recommended with regard to distributing the network services amongst the adapter/ports? Both adapters are npar capable. 3x Dell 6515 with: 1x AMD 7542 128 GB memory 2x 1Gb nic 2x 25GbE sfp28 (Broadcom...

Hi All, I've a ceph Cluster with 3 nodes HPE each with 10xSAS 1TB and 2xnvme 1TB below the config. The replica and ceph network is 10Gb but the performance are very low... in VM I got (in sequential mode) Read: 230MBps Write: 65MBps What I can do/check to tune my storage environment? # begin...

This week we have been balancing storage across our 5 node cluster; Everything is going relatively smoothly but am getting a warning in CEPH: "pgs not being deep-scrubbed in time" This only began happening AFTER we made changes to the disks on one of our nodes; CEPH is still healing properly...

Hi, I plan to build my first ceph cluster and have some newbie questions. In the beginning I will start with 5 nodes, and plan to reach 50 nodes. Those nodes quite old (CPU E3,16GB RAM, 2x1Gbps network), so I think to gain the performance in adding more nodes but not upgrading RAM or CPU. I...