Correct initial installation w/ CEPH

Ma907xb

Well-Known Member
Dec 26, 2018
71
1
48
USA
Hello,

I have 2 SATA drives both 1TB. What is the best option for a proxmox installation with CEPH + cluster. Which option runs the most stable?

Both drives are setup on my server as AHCI with hardware raid disabled. Would zfs (RAID 1) be the best option?


proxmox install.JPG

Do any of these advanced configurations matter?


proxmox install 2.JPG

Thanks for the help
 
Last edited:
I have 2 SATA drives both 1TB. What is the best option for a proxmox installation with CEPH + cluster. Which option runs the most stable?

Both drives are setup on my server as AHCI with hardware raid disabled. Would zfs (RAID 1) be the best option?

RAID1 would mirror the data to both disks, if one fails normally nothing is lost and you can just replace the disk and "resilver" it. So yes, for a two disk setup it'd be the most stable, regarding hardware failures.
But you'll natrually then have only the space of 1 disk available, as their mirrored.

Do any of these advanced configurations matter?

No normally not. Single thing which could matter is that we currently do not create a SWAP automatically on ZFS volumes as there where issues with that in the past. But, if you reduce "hdsize" parameter for the amount of SWAP desired you can then add a SWAP devices on that remaining space, after the installation finished.

If you id not already see and read it, our docs may help to answer a few questions too: https://pve.proxmox.com/pve-docs/pve-admin-guide.html#chapter_zfs

For more suggestions we'd need to know what's important for that setup, and what your requirements are.
 
RAID1 would mirror the data to both disks, if one fails normally nothing is lost and you can just replace the disk and "resilver" it. So yes, for a two disk setup it'd be the most stable, regarding hardware failures.
But you'll natrually then have only the space of 1 disk available, as their mirrored.

Hardware/drive failure is not so important for my installation. Initializing clustering + CEPH will give me the redundancy I need. The data on the VMs is not critical. CEPH provides the redundancy through OSDs, correct? It's more vital that I have more storage.

Would RAID 0 be adequate? Should I use ext4? If a drive does fail CEPH will be able to rebuild it from other OSDs?

CEPH recommendations
https://docs.ceph.com/docs/hammer/rados/configuration/filesystem-recommendations/

No normally not. Single thing which could matter is that we currently do not create a SWAP automatically on ZFS volumes as there where issues with that in the past. But, if you reduce "hdsize" parameter for the amount of SWAP desired you can then add a SWAP devices on that remaining space, after the installation finished.

If you id not already see and read it, our docs may help to answer a few questions too: https://pve.proxmox.com/pve-docs/pve-admin-guide.html#chapter_zfs

For more suggestions we'd need to know what's important for that setup, and what your requirements are.

We have no plans to implement SWAP on our hosts.
 
Last edited:
Isnt there no need to use ZFS when configuring CEPH? I could use ext4 and that would be adequate? I would then be able to utilize each drive and the OSDs will provide the redundancy. Is this correct?
 
Isnt there no need to use ZFS when configuring CEPH? I could use ext4 and that would be adequate? I would then be able to utilize each drive and the OSDs will provide the redundancy. Is this correct?

Yes, I have to admit that I initially read your question wrong and missed the ceph part and thought this all was about a single node local ZFS only, sorry about that.

So, what's your (future) ceph setup would look like? How many nodes with how many disks?
As two disks ain't that much, so a smaller extra one for the OS to free up both disks fully for use as OSD with ceph could help.
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!