ceph

I need to build a 4 node Ceph cluster and I need an effective capacity of 30TB. Each node is configured like this: Supermicro 2U Storage Server 24 x NVME, X11DPU, Dual 1600W 2 x Intel Xeon Gold 6240 18 Core 2.6Ghz Processor 768G Ram 2x 100G Network 6 x Samsung PM9A3 3.84TB PCIe 4.0 2.5...

Users had been complaining about laggy VM performace on our HDD-based Ceph pool. With many users VMs having many applications that are logging to disk, I see a lot of small writes with write IOPS > 5x read IOPS. Reading up here and elsewhere, it seems that Write-back may yield better...

Hello community, My first post here. I have been searching on the subject and the posts I have seen on the forum and by search engines on the subject have not finished me clarify my doubt. To put in context. I have 3 PVE Nodes in Cluster with CEPH as backend storage for the VMs. In the ceph...

Hi All, I am using Proxmox 8.1.4 without license subscription. I have changed to no subscription repository. But it is still not possible to install Ceph. TT /etc/apt/sources.list deb http://ftp.debian.org/debian bookworm main contrib deb http://ftp.debian.org/debian bookworm-updates main...

Just ran into this in the lab, haven't gone digging in prod yet. pve-manager/8.1.3/b46aac3b42da5d15 (running kernel: 6.2.16-20-pve) Cluster is alive, working, zero issues, everything in GUI is happy, 100% alive -- however... the "ceph device" table appears to have NOT updated itself for a...

Hey guys, in the latest Ceph blog a post was published on 13.02 regarding a CMAKE_BUILD_TYPE bug, which apparently affects Ceph Ubuntu packages. I know that we use Debian, but in a github pull request Proxmox is mentioned with a significant improvement in latency. Can we assume that we are...

Hi, when I try to attach a new osd to my ceph cluster, I get an error regarding the link https://quay.io/v2/ I would like to know where this error comes from and why ? And what is the real use of the quay.io/v2/, does ceph retrieve information on the remote server ? Thanks in advance Error...

Hi, i have a cluster of 3 compute nodes and 3 storage nodes. I wanted to upgrade to pve 7.4 and ceph quincy. Followed the official documentation https://pve.proxmox.com/wiki/Ceph_Pacific_to_Quincy All went ok until i restarted the osds on one of the storage nodes. some are upgraded and some...

We're succesfully using Ceph on Proxmox, and have started to attempt to use CephFS. We are able to mount, and create a file, but can then not write to the file, it shows the below error: root@<redacted>:/mnt/ceph# echo "test" > /mnt/ceph/testfile -bash: echo: write error: Operation not permitted...

Hello guys, I have a server that is now set as: 2 disk Mirror with zfs (SSD). for boot and 4 disk for ceph. I have an option to install nvme so my plan was to do the following: brake the mirror, use the nvme as mirror and then use the SSD drive to enlarge the ceph pool. Any suggestions on that...

Hi, I plan to set up a 2-node cluster, with each node having exactly the same configuration: 1 CPU with 12 cores, 64 GB RAM, 5 x GbE, 2 TB NVMe SSD, and 500 GB SATA-6 SSD. Network: One GbE will be dedicated to Proxmox Virtual Environment (PVE) administration (on a dedicated VLAN) and...

Hi, to set the context, I want to connect my ceph cluster to my proxmox cluster, I did the same thing as leihnix Post reference: the https://forum.proxmox.com/threads/using-external-ceph-cluster.93453/ I also see that my pool is visible with a question mark but the size is not available, do you...

Hello all, i just noticed, that one of my Cluster Nodes shows following log every 1-2 minutes: FYI: The Cluster works and Ceph is showing all OK Feb 08 01:18:52 pve-2 pvedaemon[3570]: <root@pam> successful auth for user 'monitoring@pve' Feb 08 01:18:54 pve-2 ceph-crash[2062]...

Hello, I have 3 identical servers running identical version of Proxmox with all of therm fully upgraded. Each has 2 network cards installed, one is on-board quad 10gb and another is PCie quad port SPF+ 10gb. I have created two bridges, each bridge having all 4 ports assigned. Both bridges has...

Hello, to set the context, I have a proxmox cluster with 3 nodes, I also have a ceph cluster with 3 nodes, everything is fine and everything works well independently. But, before linking the ceph cluster and the proxmox cluster (CephFS), I want a linux/windows client to use the ceph cluster as...

Hello, I have 3 servers later planning to add 3 nodes more. Lenovo sr570 CPU: 2x Xeon Silver 4208 2.1ghz 8c 16t OS: 2x sata 480gb m.2 ssd mirroring kit Network: 4x10GBase-T 2x lacp for frontend and cluster and 2x Lacp for ceph cluster and public every where will be individual Vlan. Switch: 2x...

Just written down for your amusement, on a lazy, dark and rainy Sunday afternoon: Someone (me) might try to extend a Ceph cluster by adding NVMe (or other SSDs) via USB3 Adapters. For a small homelab this should be feasable, isn't it? My three PVE nodes already run a single Ceph OSD on an...

Hi, I have a probleme with my proxmox cluster, I have a bond for CEPH derectly connected betwen the nodes in 10Gb and a bond for public network with 10Gb link to. But when i do an iperf with public network I have my 10Gb/s but not with my CEPH network. Before this configuration I haved...

For my testing/development environment, I am looking to configure a cluster of Proxmox servers using a Ceph storage pool so I can implement HA. My goals are high availability and having the ability to bring down a host for maintenance and patching. Also, to handle the occasional fault. I...

Hey everyone, I am running my kubernetes nodes on my Dell R730 At the moment they are using a bridged Network on a 10Gbe NIC…But I only have a 1Gbe switch.\ That said, the server has a second (unused 10Gbe Nic) eno2 I was wondering if It would be possible to create a “logical” inter-vm network...