The new servers for work system

[ProxmoxBoy1]

I am currently thinking about a new server system at my company.
There are currently separate old servers running under Esxi and Nutanics.
The idea is to purchase 3 new servers and perhaps 1 backup server.
2 new servers are in building A
1 server and the backup in building B.
The buildings are interconnected with a fiber optic ring.
Ring 1 1GBring
Ring 2 10GBring for data link between the servers ( copy vm's to another server backups')

The servers will be equipped with
AMD processors 3.0 GHz 16C/32T 64M Cache
96GB Ram 4800SR
PERC H355 with rear load bracket
3 x 1,92GB read intensive SSD ( Raid 1 + 1 Hotspare)
Broadcom 5719 Quad Port 1GbE BASE-T Adapter
Broadcom 57416 Dual Port 10GbE BASE-T Adapter

Which I have already found out

Disable Raid controller and set BIOS to HBA mode for the disks
(maybe change to no raid controller but ik can not deselect this on the Dell website)


Whats is best practice ?
Add extra disks for the Proxmox operating system like small SSD 120GB ?

I already found a good video about linking 3 servers

The backup server maybe we do not need now.
We also need HA i think. On a failure in 1 server restore a backup from the shared storage is no problem.
The Synology rackstation with 20tB space is also connected to the 10GBE network.

What are we running
2 Production systems server 2019 2 Processor 16GB
1 InfluxDB server Ubuntu 2 Processor 16GB
1 Historian server Server2019 2 Processor 16GB
2 very Small windows10 1 Proccessor 12GB

 

[bbgeek17]

Just to clarify, as its not stated explicitly in your description, you are planning to use ZFS replication for failover? If that is the case, keep in mind that replication is asynchronous and can be behind from few IOs to many minutes. Make sure your applications (databases, etc) are able to recover from such state.

Few thoughts:
- Splitting the cluster across multiple sites will only provide reliable setup when the network latency is reliably low.
- Keep in mind that a split in link between sites (for any reason) would put your cluster in in RO mode for configuration changes and may require manual override for recovery.
- It appears you will not be using shared storage for disk images. If your method of recovery is restoration from backup, that would mean your business is ok with data loss as your RPO could be hours, or even days old.
- Its always best to have your root disk protected. The most common way to protect the root disk is a mirror, either hardware or software.

---

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox

 

[ProxmoxBoy1]

Sorry for the delay, i have been seek the whole week.

The good news is that my servers are ordered we have added 2 smal disks for the proxmox install on ZFS

@bbgeek17

We don't plan on hot migration, since the Scada systems use database's for almost everything.

The other vm's will not be problematic.

The Scada systems we now also have 2 vm's on different locations.
1 running the other standby. The data part from the scada is backed up separately. By a script. On failure we copy with another script the backup to the other scada vm. This the only solution witch has always worked for us.
If the new system is ready we will test the live failover just to now for the future.

For now we let the 3 servers just run with ZFS each. If copying is fast enough we can always change to ceph
The 10GB connection between the 3 servers we can test, the bigger VM's are 300 GB . The whole copy is about 5 minutes i think.

 

[LnxBil]

Are you trying to run a cluster? If so, you should have read up on how clusters work: If the servers in datacenter A fail e.g. due to a power outage, your will not be able to run the VMs in datacenter B. It has no quorum and only 1 out of 3 votes.

 

[ProxmoxBoy1]

Are you trying to run a cluster? If so, you should have read up on how clusters work: If the servers in datacenter A fail e.g. due to a power outage, your will not be able to run the VMs in datacenter B. It has no quorum and only 1 out of 3 votes.

We first we setup a cluster, after this we going to tests.
Like mentioned earlier, moving vm's in the cluster test.
If 1 server fails we can restore the backup.

We will decide later whether to switch to CEPH, but first we need to install more disk capacity