ceph

I'm trying to do a two ceph pools with 3 nodes (Site A) and 3 nodes (Site B) with two different rules. The first rule is a stretched Ceph with Site A and Site B and the second rule is just a standalone ceph rule on Site A. Each of the 6 nodes has 3 OSDs. So total there's 18 OSDs. I'm...

Hello, I am trying to create a high availability Proxmox cluster with three nodes and three SSD's however I can't seem to get it working. The monitoring on two of the three nodes isn't working and I cannot actually create the VM on the share

I'm operating a Dell PowerEdge R740 server as a Ceph node. The current system boot configuration utilizes a single SAS SSD for the OS drive. I'm planning to migrate to an Dell BOSS M.2 SATA drive via PCIe adapter (two drives as RAID 1) (https://www.ebay.com/itm/296760576190) to optimize the...

Hey everyone, I’m running into a serious issue while trying to install Ceph Reef (Windows x64) on my Windows laptop. The installation seems to go smoothly until it prompts me to restart the PC. However, after the restart, my computer goes into automatic repair mode, and I can't seem to get it...

I have 3 HPE Proliant DL360 Gen10 servers that I would like to configure as a 3 node Proxmox cluster. Each node has 6 x 2TB data drives, for a total of 12TB per node + 2 system drives in mirror. Most configurations I have seen relay on an external NAS as the shared storage space. I am specially...

Hi Community! The recently released Ceph 19.2 Squid is now available on the Proxmox Ceph test and no-subscription repositories to install or upgrade. Upgrades from Reef to Squid: You can find the upgrade how-to here: https://pve.proxmox.com/wiki/Ceph_Reef_to_Squid New Installation of Reef...

Howdy, I have a CEPH cluster built on a former vSAN cluster. Right now, there are three nodes, although I do have three more identical servers available. CEPH is slow and I know that I am not getting any new hardware, so I would like to make it run as well as possible. Hosts are BL460G9 with...

Hello, we have a proxmox server cluster with CEPH storage. Today one of the nodes (1/3) was crashed, we rebooted it physically but one of the VMs doesn't boot anymore because the disk is not visible in the CEPH pool. With the command rbd ls “poolname” the disk doesn't appear. Any ideas?

Hallo, ich bin gerade damit beschäftigt unseren ersten Cluster mit Proxmox zu installieren und hätte noch die ein oder andere Frage. Hardware Daten: 6 Server mit je 2x AMD Epyc 9124 768GB RAM 6x 3,84TB NVME SSD 2x 2Port 25G NIC Workload: ~100-120 Windows/Linux VMS Auf den Servern habe...

Hello, We recently built a 3-node PVE hyper-converged cluster w/ ceph and I was wondering about the following: For MTU size on interfaces, does it matter where that is applied? I believe I've found that on linux bridge it is unnecessary as it inherits it from the bond, but what about the...

I've just installed radosgw with apt install radosgw on my Proxmox Ceph 18.2.4 Reef cluster as well as the Ceph dashboard in the hopes of managing the object storage from it, however the dashboard throws a number of internal errors as it requires the rgw module to be enabled. When checking the...

Any guess for Ceph 19.2 implementation on the horizon?

Hi, i get the error message "595 Connection refused" if i try to manage 2 of a 4 nodes cluster. This is a production cluster and every node comes with 1 dedacted 10gig nic which one with 2 ports, one for HA and one for Ceph. The manage network is on default 1gig nic, checking on logs i'll see...

Hi yall, I'm thinking about creating a Ceph pool with a EC 2+4 scheme. Even though I did intensive Google reserch, I could not find any experience with that. My idea is this: The Ceph cluster is spread across two fault domains (latency < 1 ms, 40 disks on each side, all NVME SSDs, lots of CPU...

We have two data centers (streched cluster) with 4 servers each in a VMware vsan config ( with Raid 1 -mirroring) of each with 20 disks / 60TB total capacity ( RAW). We would like to convert this cluster into proxmox with ceph as a storage. I am listing out only the current VSAN storage layout...

Hi, We want to procure a 3-node Proxmox cluster with Ceph. The configuration of each node would be as follows: Memory: 384GB CPU: 40 core (80 Thread) OS: 2 enterprise SSD drives in RAID1 mode Ceph OSD: 5 x enterprise mixed used NVMe SSD drives. (All will be connected to the On-board storage...

Hello, I have 4 Proxmox nodes with 2x10G interfaces. I am testing out a configuration without a switch. I have followed the tutorial in the https://pve.proxmox.com/wiki/Full_Mesh_Network_for_Ceph_Server The only difference between my setup and the setup in the tutorial is the addition of the...

According to the Ceph docs, if a pool has less than min_size OSDs available, IO is blocked, that includes writes, but also reads. This seems counter-intuitive to me, does anyone know, why this is the case? Examples: Standard 3/2 pool, replicated with size 3, min_size 2: Can fully read and...

Hi, I have a test cluster of 3 nodes, each with 2 disks : one for boot + Proxmox, the other for storage. But I want to add a shared storage to make some tests. Before going to a NAS with iSCSI, I want to try something new (for me) : CEPH So, is it possible to dedicate the storage disk of each...

I've been messing with Proxmox on and off for years in homelab using old desktops as hosts and lately added a pair of second-hand enterprise servers. Just started messing with Ceph in virtualized hosts, had a lot of fun with that and I want to try running it on metal for home & lab use. However...