ideal storage for proxmox os and ceph mon (not to overspend it)

[ilia987]

hii all. i am in the process of getting new hw and go to ceph storage

i think my hardware will be based on 5 nodes each with with 3x pcie-nvme and 10x 2.5 ssds each

the ssds 2.5 will be used for mostly read intensive access
and the pcie-nvme will be used for os\sqldb and lxc containers

1. two 2.5 ssds (for example in raid 1 (clone ) )
2. one pcie-nvme will be better?
3. i can get 1u with 8x2.5(sata\sas) and 2x NVMe orange color caddies )

and if so what capacity is enough for the task ?

what is the best option ?

 

[Alwin]

Have you seen our Ceph Benchmark Paper and the corresponding thread?
https://forum.proxmox.com/threads/proxmox-ve-ceph-benchmark-2018-02.41761/

Plus our and Ceph's hardware recommendation/precondition.
https://pve.proxmox.com/pve-docs/chapter-pveceph.html#_precondition
https://docs.ceph.com/docs/nautilus/start/hardware-recommendations/

 

[ilia987]

Have you seen our Ceph Benchmark Paper and the corresponding thread?
https://forum.proxmox.com/threads/proxmox-ve-ceph-benchmark-2018-02.41761/

Plus our and Ceph's hardware recommendation/precondition.
https://pve.proxmox.com/pve-docs/chapter-pveceph.html#_precondition
https://docs.ceph.com/docs/nautilus/start/hardware-recommendations/

i have seen it , all i found is

Important
Since Ceph has to write all data to the journal before it can send an ACK (for XFS at least), having the journal and OSD performance in balance is really important!

and i did not find an answer to my question

 

[Alwin]

Well, this note is more for filestore backends than bluestore and more historical. But did you go through our preconditions? These give a rough guide on what to look for in hardware.

But didn't we go through this already in this thread?
https://forum.proxmox.com/threads/extending-proxmox-cluster-and-moving-to-ceph.57836/

 

[ilia987]

Well, this note is more for filestore backends than bluestore and more historical. But did you go through our preconditions? These give a rough guide on what to look for in hardware.

But didn't we go through this already in this thread?
https://forum.proxmox.com/threads/extending-proxmox-cluster-and-moving-to-ceph.57836/

not exactly,

this post is generally to find the optimal storage to be used as proxmox os and ceph (jornal,mon...)