Correct initial installation w/ CEPH

[Ma907xb]

Hello,

I have 2 SATA drives both 1TB. What is the best option for a proxmox installation with CEPH + cluster. Which option runs the most stable?

Both drives are setup on my server as AHCI with hardware raid disabled. Would zfs (RAID 1) be the best option?




Do any of these advanced configurations matter?




Thanks for the help

 

[t.lamprecht]

I have 2 SATA drives both 1TB. What is the best option for a proxmox installation with CEPH + cluster. Which option runs the most stable?

Both drives are setup on my server as AHCI with hardware raid disabled. Would zfs (RAID 1) be the best option?

RAID1 would mirror the data to both disks, if one fails normally nothing is lost and you can just replace the disk and "resilver" it. So yes, for a two disk setup it'd be the most stable, regarding hardware failures.
But you'll natrually then have only the space of 1 disk available, as their mirrored.

Do any of these advanced configurations matter?

No normally not. Single thing which could matter is that we currently do not create a SWAP automatically on ZFS volumes as there where issues with that in the past. But, if you reduce "hdsize" parameter for the amount of SWAP desired you can then add a SWAP devices on that remaining space, after the installation finished.

If you id not already see and read it, our docs may help to answer a few questions too: https://pve.proxmox.com/pve-docs/pve-admin-guide.html#chapter_zfs

For more suggestions we'd need to know what's important for that setup, and what your requirements are.

 

[Ma907xb]

RAID1 would mirror the data to both disks, if one fails normally nothing is lost and you can just replace the disk and "resilver" it. So yes, for a two disk setup it'd be the most stable, regarding hardware failures.
But you'll natrually then have only the space of 1 disk available, as their mirrored.

Hardware/drive failure is not so important for my installation. Initializing clustering + CEPH will give me the redundancy I need. The data on the VMs is not critical. CEPH provides the redundancy through OSDs, correct? It's more vital that I have more storage.

Would RAID 0 be adequate? Should I use ext4? If a drive does fail CEPH will be able to rebuild it from other OSDs?

CEPH recommendations
https://docs.ceph.com/docs/hammer/rados/configuration/filesystem-recommendations/

No normally not. Single thing which could matter is that we currently do not create a SWAP automatically on ZFS volumes as there where issues with that in the past. But, if you reduce "hdsize" parameter for the amount of SWAP desired you can then add a SWAP devices on that remaining space, after the installation finished.

If you id not already see and read it, our docs may help to answer a few questions too: https://pve.proxmox.com/pve-docs/pve-admin-guide.html#chapter_zfs

For more suggestions we'd need to know what's important for that setup, and what your requirements are.

We have no plans to implement SWAP on our hosts.

 

[Ma907xb]

Isnt there no need to use ZFS when configuring CEPH? I could use ext4 and that would be adequate? I would then be able to utilize each drive and the OSDs will provide the redundancy. Is this correct?

 

[t.lamprecht]

Isnt there no need to use ZFS when configuring CEPH? I could use ext4 and that would be adequate? I would then be able to utilize each drive and the OSDs will provide the redundancy. Is this correct?

Yes, I have to admit that I initially read your question wrong and missed the ceph part and thought this all was about a single node local ZFS only, sorry about that.

So, what's your (future) ceph setup would look like? How many nodes with how many disks?
As two disks ain't that much, so a smaller extra one for the OS to free up both disks fully for use as OSD with ceph could help.