Setup Proxmox help

[Cerberus]

Hello, we are new in community and we need some help and some advices about Proxmox.

We have 2 same Servers both with 2x Ten Core, 128GB RAM, 1 SSD (250GB) and both with 4 (6TB) HDD.

On SSD proxmox is installed only.

Can we have RAID for hard drives between first server and second server? In order that we can have with this setup (RAID) the best performance, for free space and high availability. So if some server closed down from an accident the other server must take all (data,VM,etc..). How can we do that ?

What are you suggest for synchronize our hard drives from one server to other ? We have on optical fiber card 4+4G and we have 4 ports of ethernet except 1 port because of network connectivity with datacenter.

We have already a proxmox server in another datacenter.. Can we import this server into the cluster for backup use. We want the "Ceph" to work propertly with other 2 servers. If yes how can we do that? And what is the best solution?

After one year example we gonna add one new server but maybe he is not in the same datacenter just like the previous two servers. what do you suggest for this?

And one last question we have one qnap with 6 x 4 TB HDD in another datacenter and our ping between first datacenter and second is about 3 to 13ms. We just want to make some backups in qnap, we do not want to match one disk from Proxmox with iSCSI but maybe with some other protocol with better immediate response. What can we use here?

Thanks we are waiting your replies

 

[LnxBil]

Two nodes are not enough for HA and not enough for CEPH, so you need at least a third node. Cluster between datacenters are tricky because of the delay. If it would be 3 ms, it's ok, 13ms can be too much. You also do not want to run CEPH on 1 GBE, i would not want to use it without 10 GBE.

 

[Cerberus]

Two nodes are not enough for HA and not enough for CEPH, so you need at least a third node. Cluster between datacenters are tricky because of the delay. If it would be 3 ms, it's ok, 13ms can be too much. You also do not want to run CEPH on 1 GBE, i would not want to use it without 10 GBE.

Thanks for your replay, i understand what are you talking about, so we can't have 2 Servers on HA with CEPH? Is there anything else that I can do with 2 Servers? Or it is not safe ?

We have already for CEPH 4+4G Fiber = 8GB if it is not enough we could use some ethernet ports to catch 10GB.

 

[LnxBil]

Thanks for your replay, i understand what are you talking about, so we can't have 2 Servers on HA with CEPH? Is there anything else that I can do with 2 Servers? Or it is not safe ?

You won't be able to use HA because you cannot have quorum, Cluster (as in one gui to administer both) itself is no problem.

https://pve.proxmox.com/wiki/High_Availability

We have already for CEPH 4+4G Fiber = 8GB if it is not enough we could use some ethernet ports to catch 10GB.

Determine the IO/delay and throughput. Both are important for a fast setup.

 

[Cerberus]

Yesterday we try to setup the nodes with this configuration:
ZFS 1 software RAID in 4HDD x6TB and we take after that 15.31TB space.

I add ZFS 15.31TB space to each node from disk manager on web interface

I create 2 LXC Containers and 2 Virtual Machines to each node.

After that i create a group inside cluster and i named that “rack1” and i add in there (pm1 and pm2).

I go to menu HA and i add all LXC containers and VMs.

After that i login node 2 with SSH and i shut down the server 2. I wait few minutes to pm1 server takes the LXC Containers and VMs from pm2 but nothing happend. And quorum was down.

Any solution about this configuration?

**Also if i go to migrate a VM from pm1 to pm2 that doesnt work. (Task error: Migration aborted).

***extra information: in the same network we have a shared file via SMB. Also in another datacenter we have one more proxmox server with lower specifications and in the future in this server we gonna end this subscription. In pm1 and pm2 we gonna buy subscription soon. Is it possible to add this quorom into SMB shared file? And not on 3rd proxmox server in the other datacenter?

 

[LnxBil]

After that i login node 2 with SSH and i shut down the server 2. I wait few minutes to pm1 server takes the LXC Containers and VMs from pm2 but nothing happend. And quorum was down.

Any solution about this configuration?

No, HA is not possible with two machines, quorum can never be reached. In a two node setup (as well as in a three node setup) you need two votes, so it only works (= you have quorum) if two machines are up.

**Also if i go to migrate a VM from pm1 to pm2 that doesnt work. (Task error: Migration aborted).

Sure, you have a share-nothing architecture. In a "real" cluster and HA environment you need shared storage for easy online migration, failover evt.