Some Advice needed on adding new node or san/nas

[gerdnl]

Hi all,

i currently have 3 node running on proxmox 3.4

node1: 8x intel xeon cpu e5405 2.00ghz 2 sockets (2 network ports)
node2: 8x intel xeon cpu e5405 2.00ghz 2 sockets (2 network ports)
node3: 24x intel xeon cpu e52620 v2 2.10 ghz 2 sockets (4 network ports)

I buyed a new server wich i want to use to make HA working
- 8x intel xeon 2.9ghz 8 mb cache, 4x 240gig ssd with lsi megaraid + lsi cachevault (4 network ports)

Can i do it like this? what can i best install on the new server? san/nas?
Does it even gona work with node3 not being the same as node1/2, node3 already retransmits alot to 1/2

What would be your advice?
Go on with making HA happen by making the new server a san or nas or dont go for it and just make it node 4

i already decided to stay with 3.4 for now, can always go 4.* later on

 

[LnxBil]

Where is HA in your setup? One machine cannot be considered HA. You can of course use the new server as a NAS (via NFS/Samba) or SAN (iSCSI), and have a working Cluster with live migration, but HA needs two storage servers, or distributed shared storage via glusterfs.

 

[gerdnl]

thanks, so different servers is not a problem in the cluster

 

[Ashley]

thanks, so different servers is not a problem in the cluster

Correct, main importance is that if one fails (biggest) the smaller ones can handle the load of the extra VM's.

 

[gerdnl]

i gues 4x gigabit network has a limit to right in read/write access
im not experienced with that do, im not doing extreem work on my nodes do

 

[Ashley]

Are you looking to just use the 4th server as storage? and not spread across all servers?

 

[gerdnl]

eu i kinda need to for live migration?

 

[Ashley]

I mean you looking to use a single node with storage presented via NFS for example, or using storage across all servers using CEPH for example.

 

[gerdnl]

i would like to use it across all nodes, i want vm's to trans automatically from 1 node to another when 1 node goes down.

 

[gerdnl]

i didnt read much about seph yet, could that be a solution to, create a ceph from node storage and use it as HA

 

[Ashley]

If you use CEPH and have 4 nodes you can lose any without issue, if you use one for storage if you lose that node no matter what you will loose the VM's.

 

[gerdnl]

ashley, i will read more about seph, so i got this right? i can use seph on the nodes i already have. can i set this up without messing with the existing proxmox on it, so beside it. i will read more else i probaly ask to much stupid questions

 

[gerdnl]

I think this answeres my question

 

[Ashley]

With CEPH you need to have free storage / disks available on each node you can convert into OSD's.

You can then setup a storage pool across multiple servers.

 

[gerdnl]

node 1 and 2 only have 2 gig ports (these dont have a expansion slot) SATA
node 3 has 4 gig ports (this on has a expansion slot) SATA
node 4 has 4 gig ports (this on has a expansion slot) SSD

2 node ceph wont work right? i think node1/2 arent good enough for the task so i kinda have 2 good nodes

any ideas?

 

[LnxBil]

Ceph on all servers will technically work, but you will not have fun with it. I'll to slow due to limited network bandwidth and local storage speed - Depending on your needs, 2 or 4 GBE would be ok if you do not write a lot.

With your given hardware, it is hard to create a HA setup. One way to go would be DRBD on two nodes and a third as quorum. then you'll have HA where 1 of 3 nodes can fail and everything will still work. Please refer to https://pve.proxmox.com/wiki/DRBD
Another way would be glusterfs and replicate to each node. This will be more network intensive, but also work.

 

[gerdnl]

so if i want to go seph, i would need to upgrade 1 node to sdd also, get 1 more ssd node. add more gig ports to each, lets say 8 on each to avoid having to buy a 10g switch, remove node1/2 pretty old server, or use it for something else, somehow DRBD doesnt appeal to me

why is it btw that 2 node ceph doesnt work?

 

[Ashley]

so if i want to go seph, i would need to upgrade 1 node to sdd also, get 1 more ssd node. add more gig ports to each, lets say 8 on each to avoid having to buy a 10g switch, remove node1/2 pretty old server, or use it for something else, somehow DRBD doesnt appeal to me

why is it btw that 2 node ceph doesnt work?

A 2 node CEPH cluster does work, however its recommended to always use a replication of 3 on CEPH.

Something you can't really do too well with 2 nodes, and will always require a 3rd small node for quorm.