Proxmox Ceph Cluster for prodution service

May 28, 2020
South Africa
Hi Proxmoxers,

Im looking at setting up a storage solution for a production service to store archival video data for a managed service

I am looking to start out this service using 3 x Poweredge R740/750's with later model dual socket Xeon cpus (64 vCPU) and 128GB ram each. For storage each box will have a PCIE 4.0 SSD/flash accelerater and a bunch of high end, high capcity SATA surveillance disks. Hosts will be connected to the network with 2 x 10GB SFP+ bonded using LACP to give 20GB throughput capacity between the hosts.

This will give me roughly 1,5PB RAW of spinning storage to start to build my ceph cluster (500TB per host) plus 3 x 1.6TB flash accelerator.

Ceph will be built inside proxmox cluster.

The proxmox cluster will essentially only host workloads to expose the storage to the client, the rest of the compute will be available to ceph services.

Each node will have a monitor, manager and OSD installed on it.

Once the storage reaches 60% capacity we will then add an additional node to the cluster adding anoither chunks of 500TB raw.

We would then intend to keep scaling as above by adding an addtional 500TB plus compute efvery time we hit that 60% capacity mark.

Will this work?

How do i make use of the flash accelerators to improve performance in cluster?

What am i missing?

What (if any) is the limitation on how many nodes i could add in?

Looking forward to your input


The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get your own in 60 seconds.

Buy now!