1. I

    Ceph - Module 'restful' has failed dependency PyO3 modules may only be initialized once

    Proxmox noob here - I've been asked to setup a 3 node PVE 8.1.10 cluster specifically to investigate options for monitoring with our NMS. First install, I hit the (apparently) well known CEPH v18.2 (Reef) dashboard compatibility problem, and subsequently rebuilt the cluster and installed CEPH...
  2. herzkerl

    Ceph Object Gateway (RadosGW), multi-part uploads and ETag

    We'd been having issues with a particular (Synology) Hyper Backup quite some time. Synology support analyzed the log files and told us that there's no ETag returned after a multi-part upload: 2023-11-21T13:20:55+01:00 NAS img_worker[20149]: (20149) [err] transfer_s3.cpp:355 ETag is empty...
  3. J

    PMX 8.1.4 - Post upgrade - Network throughput problem - Iperf3 0.00 bits/s

    Hi, Long shot.... Trying to repair the questionable decision to reinstall for upgrade from 7.4.x and to 8.1.4. Question: Why is the iperf3 showing these issues? Not necessarily even a problem, but still very strange results. Background: 4 host machines 256 core in total 2TB RAM, all...
  4. D

    Importing a Ceph disk

    Doing this all on the command line to create a VM on a Ceph storage. Hoping this is a simple fix. qm create ... qemu-img create rbd:Ceph/vm-998-disk-0 41G Formatting 'rbd:Ceph/vm-998-disk-0', fmt=raw size=44023414784 When I run the following command, I get this error: # qm disk import 998...
  5. K

    [SOLVED] Ceph and Storage Networking Configuration

    I have a three node cluster. Each node is an HP 600 G6 DM with the built in 1GbE port and a 10GbE add on through the Flex IO port. Each nodes gigabit port is connected to my UniFi Dream Machine on the (VLAN2) network. The three 10GbE connections are connected to a USW-Flex-XG. I plan...
  6. K

    [SOLVED] Ceph and erasure code profile

    So I have some servers where I am trying to setup the ssd to utilize erasure-coding. Problem is there are hdd's on as well and I needed to set the device class to ssd.. So using the instructions I was able to create the profile I wanted using the following command ceph osd erasure-code-profile...
  7. R

    Smartctl Offline_Uncorrectable in SSD

    Hello ! I am receiving a lot of email about Offline_Uncorrectable in my SSD, in different servers and multiple devices. The following warning/error was logged by the smartd daemon: Device: /dev/sdc [SAT], 16 Offline uncorrectable sectors Device info: HFS1T9G32FEH-BA10A...
  8. S

    Ceph - 1 subtrees have committed pool target_size_bytes

    Hello everyone We have gotten this failure on our 3 nodes cluster and I really don't know how to solve it. I would really appreciate the help. Thanks in advance
  9. I

    Ceph configuration - best practices

    Long story short... not possible. I'm planning to install and use ceph and HA in my working cluster enviroment. I have 4 nodes with 256GB RAM each and 2x10G NICs dedicated for ceph cluster network traffic. I already know this may be not enough for perfect performance so I'm planning to swap NICs...
  10. D

    Command Line - Create a disk on Ceph storage

    I need to create a new VM (250+ actually) that uses Ceph storage. To create a disk on non-Ceph storage: echo "----- Creating the disk -----" qemu-img create -f qcow2 $DISK $STORAGE_SIZE How do I create a disk on Ceph storage?
  11. B

    [SOLVED] Import KVM Snapshots to Proxmox

    Hello, I'm excited to share that I've successfully migrated and imported two virtual machines from an existing KVM ovirt environment to a new hyperconverged Proxmox setup, utilizing an external Ceph cluster, which also operates as a KVM environment. Leveraging the Proxmox migration guidelines...
  12. B

    [SOLVED] Thin Provisioning in a Ceph Cluster managed by Proxmox

    Hello, I have a system landscape where I have set up a ProxmoxVE instance and an external Ceph cluster in Quorum 3. From there, I migrated two VMs, along with their disks, from an already existing oVirt KVM landscape to ProxmoxVE for testing purposes, using the CLI utilities provided by...
  13. C

    Cephs not working and rbd locked problem

    Hello! We have recently installed a PVE cluster with three nodes and want to use Cephs. The hosts are (almost) identical. Host 3 has a different network interface. We have a public network channel and an exclusive cluster network channel for Ceph. Each host has four discs for OSDs. Creating...
  14. A

    [SOLVED] Ceph OSD adding issues

    Greetings community! After few month of using ceph from Proxmox i decided to add new disk and stuck with this issue. ceph version 17.2.7 (2dd3854d5b35a35486e86e2616727168e244f470) quincy (stable) Running command: /usr/bin/ceph-authtool --gen-print-key Running command: /usr/bin/ceph --cluster...
  15. E

    [SOLVED] Issues Moving CEPH Traffic to Fiber Cards on Proxmox - RBD Unavailable After Switch

    Hi everyone, long time lurker but hey, there's a first time at anything right? So i have posted about a problem i'm facing on the proxmox reddit, and someone PM'd me to post it also here, so here's a copypasta of the original post with added all the things they told me to try over there, hope...
  16. M

    [SOLVED] Proxmox mount Cephfs on linux VM

    root@pve1:~# sudo cat /etc/ceph/ceph.conf [global] auth_client_required = cephx auth_cluster_required = cephx auth_service_required = cephx cluster_network = fsid = 5dcbd126-d511-4e12-a653-fee67d7ec467 mon_allow_pool_delete =...
  17. A

    [SOLVED] Proxmox ceph network configuration

    Hi, I read this sentence on ceph hardware-recommendations : "Provision at least 10 Gb/s networking in your datacenter, both among Ceph hosts and between clients and your Ceph cluster" This is my ceph configuration: [global] auth_client_required = cephx auth_cluster_required = cephx...
  18. M

    [TUTORIAL] What are the steps to create and mount CephFS on a Linux VM?

    It will help if it is GUI-based steps to create CephFS. I currently have configured Ceph as shared storage on all 3 nodes. Also, Linux VMs are on local storage. I have to use CephFs to create a shared folder between 2 VMs on different nodes.
  19. S

    Migration from RAID to Ceph without outage

    OK, so I’m fairly new to Proxmox and Ceph. I need some advice on how to migrate to Ceph. Current Layout: SERVER01: (Same exact hardware as SERVER02) Production - hosting 20-30 VM’s Storage Hardware RAID 2 - SSD’s setup as RAID1 6 - SSD’s setup as RAID5 This storage is hosting the VM’s Needs...
  20. L

    New build: storage optimization for variety of workloads

    Storage noob here. I am building a new single node proxmox server on a 2U server that has 9 3.5in 4TB HDDs and 6 2.5in 800GB SSDs beyond what is used for Proxmox's boot image. The server will be running a mix of stateful container workloads, databases for stateless containers, and VMs for...


The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!