ceph

  1. F

    Cluster 3 hosts datastore CEPH with HA at OVH Cloud

    Hello, I would like to set up a Proxmox cluster at OVH. 3 Hosts with CEPH for HA. I was thinking of using the Advance 3 offer from OVH. I've mapped out a solution in my lab with 3 machines, each with a 1TB SSD. After configuring a 1TB CEPH datastore including the SSD for each host. Hot VM...
  2. K

    CEPH installation requires the mon-address

    I was trying to install Ceph on a new 3-node Proxmox cluster. Therefore, I created two extra network bridges for my two 10G ports. The installation of Ceph was successful. However, when I wanted to configure the public and cluster networks, I encountered the following error: Multiple IPs for...
  3. P

    Bluestore erroring opening db

    Hello, I'm managing a cluster of 4 nodes using proxmox 7.4-17 with CEPH. After a messy shutdown caused by a long power outage, a couple of VM images were corrupted, but we could restore them from backup. However, two of the OSD services refuse to come back, showing this kind of error: Apr 18...
  4. F

    proxmox with external ceph cluster and EC pool

    I have set up a Proxmox 8 ceph cluster with an EC pool called tsPool1. Now I want to connect an independent standalone proxmox 8 host to that cluster using rbd. I used the GUI Datacenter->Storage-> Add feature to setup the storage. The storage is created but I only get a connect when I use...
  5. F

    Backup and restore cluster and Ceph

    Hi, i must restore an old 3 node cluster. Which dirs do i have to backup for the cluster restores exactly like was before (configuration, files...). Which is the correct procedure to recover it? Thank you
  6. B

    Install specific version of ceph reef

    I have an existing ProxMox Ceph cluster with ceph v18.2.1. I want to add a new host to the cluster but the ceph version it installs is 18.2.2 so I get warnings about a mismatch. How do I install the 18.2.1 version on my new host? The command I'm using is: pveceph install --repository...
  7. I

    Ceph - Module 'restful' has failed dependency PyO3 modules may only be initialized once

    Proxmox noob here - I've been asked to setup a 3 node PVE 8.1.10 cluster specifically to investigate options for monitoring with our NMS. First install, I hit the (apparently) well known CEPH v18.2 (Reef) dashboard compatibility problem, and subsequently rebuilt the cluster and installed CEPH...
  8. herzkerl

    Ceph Object Gateway (RadosGW), multi-part uploads and ETag

    We'd been having issues with a particular (Synology) Hyper Backup quite some time. Synology support analyzed the log files and told us that there's no ETag returned after a multi-part upload: 2023-11-21T13:20:55+01:00 NAS img_worker[20149]: (20149) [err] transfer_s3.cpp:355 ETag is empty...
  9. J

    PMX 8.1.4 - Post upgrade - Network throughput problem - Iperf3 0.00 bits/s

    Hi, Long shot.... Trying to repair the questionable decision to reinstall for upgrade from 7.4.x and to 8.1.4. Question: Why is the iperf3 showing these issues? Not necessarily even a problem, but still very strange results. Background: 4 host machines 256 core in total 2TB RAM, all...
  10. D

    Importing a Ceph disk

    Doing this all on the command line to create a VM on a Ceph storage. Hoping this is a simple fix. qm create ... qemu-img create rbd:Ceph/vm-998-disk-0 41G Formatting 'rbd:Ceph/vm-998-disk-0', fmt=raw size=44023414784 When I run the following command, I get this error: # qm disk import 998...
  11. K

    [SOLVED] Ceph and Storage Networking Configuration

    I have a three node cluster. Each node is an HP 600 G6 DM with the built in 1GbE port and a 10GbE add on through the Flex IO port. Each nodes gigabit port is connected to my UniFi Dream Machine on the 10.0.2.0/24 (VLAN2) network. The three 10GbE connections are connected to a USW-Flex-XG. I plan...
  12. K

    [SOLVED] Ceph and erasure code profile

    So I have some servers where I am trying to setup the ssd to utilize erasure-coding. Problem is there are hdd's on as well and I needed to set the device class to ssd.. So using the instructions I was able to create the profile I wanted using the following command ceph osd erasure-code-profile...
  13. R

    Smartctl Offline_Uncorrectable in SSD

    Hello ! I am receiving a lot of email about Offline_Uncorrectable in my SSD, in different servers and multiple devices. The following warning/error was logged by the smartd daemon: Device: /dev/sdc [SAT], 16 Offline uncorrectable sectors Device info: HFS1T9G32FEH-BA10A...
  14. S

    Ceph - 1 subtrees have committed pool target_size_bytes

    Hello everyone We have gotten this failure on our 3 nodes cluster and I really don't know how to solve it. I would really appreciate the help. Thanks in advance
  15. I

    Ceph configuration - best practices

    Long story short... not possible. I'm planning to install and use ceph and HA in my working cluster enviroment. I have 4 nodes with 256GB RAM each and 2x10G NICs dedicated for ceph cluster network traffic. I already know this may be not enough for perfect performance so I'm planning to swap NICs...
  16. D

    Command Line - Create a disk on Ceph storage

    I need to create a new VM (250+ actually) that uses Ceph storage. To create a disk on non-Ceph storage: echo "----- Creating the disk -----" qemu-img create -f qcow2 $DISK $STORAGE_SIZE How do I create a disk on Ceph storage?
  17. B

    [SOLVED] Import KVM Snapshots to Proxmox

    Hello, I'm excited to share that I've successfully migrated and imported two virtual machines from an existing KVM ovirt environment to a new hyperconverged Proxmox setup, utilizing an external Ceph cluster, which also operates as a KVM environment. Leveraging the Proxmox migration guidelines...
  18. B

    [SOLVED] Thin Provisioning in a Ceph Cluster managed by Proxmox

    Hello, I have a system landscape where I have set up a ProxmoxVE instance and an external Ceph cluster in Quorum 3. From there, I migrated two VMs, along with their disks, from an already existing oVirt KVM landscape to ProxmoxVE for testing purposes, using the CLI utilities provided by...
  19. C

    Cephs not working and rbd locked problem

    Hello! We have recently installed a PVE cluster with three nodes and want to use Cephs. The hosts are (almost) identical. Host 3 has a different network interface. We have a public network channel and an exclusive cluster network channel for Ceph. Each host has four discs for OSDs. Creating...
  20. A

    [SOLVED] Ceph OSD adding issues

    Greetings community! After few month of using ceph from Proxmox i decided to add new disk and stuck with this issue. ceph version 17.2.7 (2dd3854d5b35a35486e86e2616727168e244f470) quincy (stable) Running command: /usr/bin/ceph-authtool --gen-print-key Running command: /usr/bin/ceph --cluster...

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!