Some advice on new cluster install

[pashadee]

Hi guys,

Need the more seasoned Proxmox users to advise please.

I am setting up a new cluster that I would like to go live with in the next couple weeks.

Setup is as following: 5 Servers, all identical, each host has

32GB RAM
sda: 120GB SSD read optimized
sdb: 1TB WD Red NAS Edition

My question is in regards to partitioning. I intend on using the SSD for the Proxmox install and journal for the Ceph OSDs. The 1TB reds to be used as Ceph OSD, add more if needed later.

What would your recommendation be for the root partition / swap / pve-data partition? use ext4? Any thoughts really on this setup would be greatly appreciated!

Thanks in advance.

Paul

 

[mir]

If you use SSD your only option is ext4.
Use these mount options for /: relatime,data=ordered,barrier=0,errors=remount-ro
Use these mount options for /var/lib/vz: relatime,data=ordered,barrier=0,errors=remount-ro

Partition size SSD:
/: 10 GB
swap: 16 GB
/var/lib/vz: rest of disk

 

[pashadee]

Thanks for the quick response. Much appreciated.

I will give this a go... the only other thing, I will do some doc reading as far as what the recommended journal size is for my setup (for ceph cluster) and I assume there should be no issue with leaving that available instead of using the remainder of the SSD for /var/lib/vz?

Paul

 

[pashadee]

Quick follow up note. I got my hosts installed with the structure I wanted, bud had to do it twice as one of the commands did not work as expected.

DID NOT WORK: linux ext4 maxroot=10 swapsize=16 minfree=12

What WORKED instead is: linux ext4 maxroot=10 swapsize=16 hdsize=100

Since my SSD gave me about 112.7 GB of usable space, the second attempt gave me the 12GB of space I wanted at the end of the disk.

Hope this helps someone out.

Paul

 

[danielmash]

What about networking?

You need to think about how many NIC adaptors. 1Gb/10Gb/InfiniteBand? Switches, teaming, redundancy.

Would you mind to setup HA by any chance? In that case what hardware you think to setup fencing? DRAC, APC, Switch?