Three Node Cluster Setup recommendations

[Alfio]

Hi, I’m planning to deploy a three node cluster setup with Proxmox 4.3, Basically I want to use the following servers:


Proxmox Servers


Two Dell R710 with 128GB RAM, two xeon quad core CPU, and two sas 300gb 15k in Raid 1.

One Dell R710 with 64GB Ram, two xeon quad core CPU, and two sas 300gb 15k in Raid 1.

Those servers are for the cluster node members.


NFS Servers

I’m Planning to use a NFS server to storage the virtual machines, backups, Containers, some ISO files etc.

I will use a Dell R720 with 32GB RAM, two xeon quad core CPU and ten sas 300gb. I will use Centos 7 in this server, I will use two disk for Centos OS and the other eight disk for Raid 5+Parity spare for 1.8 TB available for free use.

I will use a second server Dell R610, 16gb RAM, three SATA 1TB disk in raid 5, and two xeon quad core CPU, this server will be used for backup for my NFS server data.


I will use bonding for the servers in the following way:

Each proxmox server will have two gigabit Ethernet card attached for summary directly connected to the NFS server, the NFS server has eight gigabit network card.

I will be hosting at the beginning like six Windows 2008 and 2012 servers VM, in the two big node servers and four Windows VM in the less resource server (16 VM for the complete cluster).

My question is if this setup will work fine or do I have to change something or some recommendations for optimization or tips.


Thanks in advanced

 

[hansm]

If your setup will work fine depends on the usecase of your Windows servers. SQL is more resource intensive than serving a remote desktop for 1 Word user. I think your storage will suffice, but try to calculate the necessary IOPS and your workload, than calculate the IOPS your storage can deliver, this will be somewhere between 500-800 IOPS.

If you need HA, please mind that your storage server is your SPOF, you can have a three node Proxmox cluster, but if your NFS server is unavailable, all your VM's will be unavailable. If HA is needed, than think about adding your R720 to the Proxmox cluster and use Ceph for your storage. You will need more disks to get the same amount of usable storage and speed. Eg. 3 disks in your 4 servers with a replication size of 2, this means that all your data will be stored twice, on 2 different servers. Get more LAN ports for your R710's, bonded (LACP) for public network (used for communication between Proxmox and Ceph storage), and bonded (LACP) for the cluster network (used for replication and recovery). Network seperation gives you more performance and security.

Ceph gives you full HA functionality, if one Proxmox node crashes, your storage is still available and VM's can be started (automatically) on other nodes. More disks = more speed. If you need more storage speed, buy some Intel DC S3710 200GB SSD's as journal disks. In your setup I recommend 1 SSD per 3 SAS disks. Your R710's probably have 8 2,5" hdd bays where 2 are used for OS disks, the other 6 could be filled with 2 SSD's and 4 SAS disks for best performance. With 4 nodes you have 4x 4 SAS disks = 16x 300GB = 4800GB. Replication size of 2 means 4800GB / 2 = 2400GB of usable capacity. This is without reservations and not taking in account that disks can fail. When 1 disk fails it will be recovered on the remaining disks. So, if 1 node with 4x 300GB fails, the 1200GB need to be recovered on the remaining disks. This will change the calculation of total space to 12x 300GB = 3600GB / 2 = 1800GB. Than you need to take in account the full ratio of 95% of the Ceph cluster.. 95% of 1800GB = 1710GB. If you fill above 1710GB and 1 of your nodes is offline to long and a recovery takes place your cluster will be full and no writes are allowed on your storage.

You have many options for disk configurations for Ceph, probably it's OK for you to use 3 SAS disks per server (12 total) with rep size of 2. Expansion is easily done by adding 4x 300GB and add it to your Ceph cluster. Your data will be rebalanced between all disks and you gain more speed because all IOPS are spread across all disks. Your disk configuration depends on needed capacity for now and the (near) future, and the demanded speed.