Anderson Alves

New Member
Aug 28, 2017
4
1
3
41
Hi guys,

I'm planing to build a 3 node Proxmox + Ceph with 6 Samsung 850 PRO 1TB, 2 SSDs (OSD) per node. I know that these SSDs are not good for Journaling, but in case of ALLFLASH setup I'm not gonna use Journaling. There will be 20 VMs running, 1 of these is a Graylog+ElasticSearch and 5 of these are Galera Cluster.

What do you think, is it a good plan?

Thanks in advance.
 
No, this is NOT a good plan, never use consumer/workstation SSDs.
this 850 has only 300TB TBW, not suites for Ceph.

Better use enterprise class SSD (Intel or Samsung).

A reasonable priced Intel SSD: DC S3520 1.2TB, 2.5", SATA (SSDSC2BB012T701)

But before you make a final decision, do a detailed calculation of your total writes for the next years.

And use Ceph Luminous with Bluestore in this case, will be production ready in a few weeks. Small setups with SSDs benefits a lot from bluestore.
 
Hi guys,

I'm planing to build a 3 node Proxmox + Ceph with 6 Samsung 850 PRO 1TB, 2 SSDs (OSD) per node. I know that these SSDs are not good for Journaling, but in case of ALLFLASH setup I'm not gonna use Journaling. There will be 20 VMs running, 1 of these is a Graylog+ElasticSearch and 5 of these are Galera Cluster.

What do you think, is it a good plan?

Thanks in advance.

you still have journal on each osd, so it'll be slow.
Maybe a small fast ssd for journals could be a better solution, even if you have slow ssd for datas.
 
But before you make a final decision, do a detailed calculation of your total writes for the next years.
That's a very good advice when planning a SSD-backed ceph infrastructure. But how to estimate the Ceph overhead? Let's say that my VMs write 100 GB of effective data per day and my pool has size=2 and uses Ceph 12 with bluestore, with data, WAL and DB on the same device ...how many GBs will be written on every SSD per day?
 
How's Samsung 960 PRO though in terms of value for money? I know it doesn't give optimal performance but if its half i suppose its good enough.
 
Could you please give me some more details as far as not being reliable?
Would it be due to low TBW?
We write almost 13GB per day...I could put Graylog+EL on another local storage with an HDD, so that my writes goes down to 5GB per day. If these Samsung 850 PRO last for 1 year, that'd be more than enough for me. That would give me the time I need to get better SSDs. What do you say?

Another question, I have 3 SSDs of 200GB from HP, they are rated as Class C for Endurance, with 3650 TBW. Will it be any good if I use these 200GB for journaling of 2 OSD of 1TB HDD 7200RPM?

By the way, here are the pieces that I already have played with:
3 x Dell R430 - 1x Xeon 2650v2 - 96Gb of RAm
3 x SSD 200Gb HP class C
6 x SSD 1Tb Samsung 850 PRO
6 x HDD 1Tb Dell SATA 7200RPM

Please help me to decide, I have to go in production ASAP...
Thank you in advance for any advices.
 
Could you please give me some more details as far as not being reliable?
Would it be due to low TBW?
We write almost 13GB per day...I could put Graylog+EL on another local storage with an HDD, so that my writes goes down to 5GB per day. If these Samsung 850 PRO last for 1 year, that'd be more than enough for me. That would give me the time I need to get better SSDs. What do you say?

Another question, I have 3 SSDs of 200GB from HP, they are rated as Class C for Endurance, with 3650 TBW. Will it be any good if I use these 200GB for journaling of 2 OSD of 1TB HDD 7200RPM?

By the way, here are the pieces that I already have played with:
3 x Dell R430 - 1x Xeon 2650v2 - 96Gb of RAm
3 x SSD 200Gb HP class C
6 x SSD 1Tb Samsung 850 PRO
6 x HDD 1Tb Dell SATA 7200RPM

Please help me to decide, I have to go in production ASAP...
Thank you in advance for any advices.

The reason i believe is the 1 job speed as illustrated in the job blog above. I'd be satisfied with 250MB/sec for 1 job speeds if its more cost effective. I think Samsung 960 PRO and beyond should do slightly better although there are no tests out there so its only an assumption.
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!