ceph

  1. Adding a 3rd manager hangs ceph

    When I add a 3rd monitor to CEPH the 3rd node times out and gives me timeout error 500 on the crush map and database. Also navigating to any of the ceph tabs on the 3rd node shows no data. However ceph continues to work. I’m not in front of the cluster at the moment but when I am tomorrow, I...
  2. How to move ceph mgr to a specific mgr node

    Hi, I have a ceph cluster with 6 nodes, and 3 nodes has mgr installed, and one (Proxmox2) is active and 2 are standby. I had an active zabbix agent installed on proxmox2, so it will send ceph info to zabbix server and it is working fine. But, every time when I have problem with proxmox2, ceph...
  3. micro server Cluster with Ceph

    Hi All I am working on a proof of concept for a micro hosting requirement. The client requires hosting of many small VM's, about 30-50 of 2GB of RAM each and 30GB of storage each. CPU and disk Load generated by these VM's are very low. They are looking to retire their two old Dell R540...
  4. Proxmox VE 2-Node Cluster and Ceph Lab Environment

    Hello guys out there, I am new to Proxmox - used to work with a VMWare 2-Node Cluster and VSAN with Core-i7 12th generation, 64 GB RAM, 2,5 Gbit/s NIC for VM traffic, 1x NVME and 2x SSD on each host over 2x10 Gbit/s direct attached HCI network as my lab environment at home. I know the Proxmox...
  5. Kernel Crashes on IO error on dev dm-X

    Hey People, I have some devastating issues with my Backup Ceph Filesystem. It started with a failed disk some days ago, which is set it into recovery/rebuild mode. Its based on two erasure coding pools, one being 8/7 and the other 12/11 with a total of 16 disks. I didn't worry about anything as...
  6. Ceph HCI - loaker Zugriff?

    hi folks, habe bloß mal schnell 'ne Frage zu ceph, wenn man 3 Node Proxmox als HCI betreibt mit Mirror 3.. Alle Knoten beherbergen ja immer alle Daten lokal. Man nutzt das RBD Device. Man brauch ja beim Einrichten des Storage keine Monitore angeben, weil Proxmox prüft, ob HCI da ist, und alles...
  7. Read-only filesystem after failed backup

    Hi, some guests become readonly after the snapshot backup (performed overnight by PBS) fails. Curiously, I happen to see the very same backup logs on different guests, even with the same guest OS, but they don't go readonly. The backup task fails, and they go on like it was nothing. Can you...
  8. Compute and Ceph Storage Cluster

    Hi Everyone I want to ask best practice for 4 nodes that i have node-1 DL380 dual proc xeon gold with 256gb ram node-2 DL380 dual proc xeon silver with 96gb ram node-3 DL380 single proc xeon silver with 32gb ram, 3x1.92tb ssd node-4 DL380 single proc xeon silver with 32gb ram, 4x1.92tb ssd...
  9. CEPH: osd_pool_default_size > # nodes possible?

    Hello All, I have allocated two nodes in my PVE cluster for storage. The idea is to use cephfs wherever possible, and possibly set up ganesha NFS or samba file share export in containers on the storage nodes for applications that need those. I don't particularly trust the disks I have now...
  10. Ceph performance advices

    Hi, We are on the point of changing our current IT infrastructure with a small Proxmox cluster of 5 nodes : Dell Precision R7920 Xeon Gold 5118 or Xeon Silver 4112 128 GB ECC RAM Data/VM disks (Ceph) on 5x3 16 TB Exos HDD. System/ISO disk on NVMe SSD via a PCIe adapter. Currently these are...
  11. Ceph LXC Volume Migration in anderen Pool

    Hallo, ich habe einen Linux Container mit einem großen Ceph RBD Volume das ich in einen anderen Pool verschieben soll. Aus der GUI ist das aber nur mit heruntergefahrenem Container möglich. Das Kopieren würde ca. 2 Tage dauern, daher habe ich mich nach Alternativen umgesehen und bin auf den...
  12. Proxmox VE Ceph/RBD Usage

    Hello, Currently we are using NFS storage for our VMs. However, there are some performance issues. We want to try Ceph/RBD storage. We have three proxmox nodes (2 physical proxmox VE + 1 tiebreaker). We have LVM(Block) storage with SAN configuration. There is a NVMe storage and two SAN...
  13. Proxmox Cluster mit CEPH Storage und OPNsense

    Hallo zusammen, ich bin derzeit dabei, ein Homelab zu planen. Es gibt jedoch noch einige Bereiche, bei denen ich unsicher bin, ob sie umsetzbar sind. Meine Idee ist es, ein HA-Cluster mit 3x U59, jeweils mit 16 GB RAM und 500 GB SDD, zu betreiben. Nun habe ich zwei Fragen zum Setup: Ist es...
  14. Ceph completely broken - Error got timeout (500)

    I have a test cluster of 3 VMs in virtualbox on my pc. Their IPs changed as follows: A: 172.16.0.150 > 192.168.100.1 (NOT a gateway) A: 172.16.0.151 > 192.168.100.2 A: 172.16.0.152 > 192.168.100.2 Now I get the following message when i run 'service ceph-mon@local status': Processor -- bind...
  15. CEPH or Cluster first

    I am trialing 3 nodes PVE with cluster and CEPH. Should I setup the cluster first or CEPH first.
  16. Proxmox HA storage and performances

    Hi. I have multiple proxmox 7 clusters for different needs. On a "staging" cluster (for staging / preproduction infrastructure) I don't have any HA. But on the production cluster, I configure two ceph storages (SSD and HDD). Developers have always pointed out to me that deploying code is...
  17. Diverse Kernel Probleme mit > 6.1.2.1

    Hallo Zusammen, seit der Kernel Version 6.1.X haben wir diverse Stabilitätsprobleme in unserem Proxmox/Ceph Cluster erfahren. Wir nutzen Ceph seit Version 17.2.4 und dem Kernel 5.15, dann mit 5.19 und jetzt 6.1.10-1. Bis Kernel Version 6.1.2.1 lief alles Problemlos, mit 6.1.6-1 hatten wir 5-10...
  18. RadosGW - S3 - Performance Expectations

    I have a relatively small nvme Ceph cluster running on a dedicated 10gb network. RBD and CephFS performance seems to be pretty good at around 500MB per second in various synthetic benchmarks. Performance uploading a 16gb test file to S3 (RadosGW) from a VM is terrible at only 25MB or so...
  19. [solved] Help with removing node(+ceph) from cluster

    Hi to all i am very new in proxmox environment, so maybe it is a basic question but i am unsure how to proceed there is a cluster with 6 node call it sm7,8,9,10,11,12 well, when i assembled the cluster, every node was in good shape, but... last pc start to behave oddly so i want to remove it...
  20. PVE - Remove one node from ceph cluster

    Hello, I have a PVE cluster with ceph installed. My ceph cluster is so composed: -node1: ceph manager, ceph monitor and osd -node2: ceph manager, ceph monitor and osd -node3: ceph manager, ceph monitor and osd -node4: ceph manager, ceph monitor and osd I need to remove one node(either ceph and...

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get your own in 60 seconds.

Buy now!