I need to setup a two node cluster with a third PBS node. I wanted to understand how the cluster works, and the services to configure to activate it. In essence, does the two-node cluster work with replicas? Can you tell me the documentation to follow for testing? In this configuration, I thought of putting two SAS disks on the PBS to reduce costs and putting three SSD disks in the two nodes respectively, of which two per node in ZFS RAID mode. Can you tell me the documentation to follow?
CLUSTER 2 NIDE
- Thread starter Fabio
- Start date
Hi, I found a documentation that explains the difference between the 2-known ZFS cluster and the 3-known cluster with Chep. For my configuration I use the 2 node cluster with replicas and zfs. I would like to add a third node made with PBS for backups. At this point I ask you for advice. The known with PBS for backups do I necessarily have to use SSD disks or can I also use SAS disks in raid formatted with ZFS? This is because, to optimize costs I would like to have two disks in RAID with ZFS and only one smaller disk for the operating system for each known PVE. Then manage all the replicas with the volume in RAID. Basically, am I forced to use SSD disks in PBS or can I also use disks with SAS technology?
A cluster needs 3+ servers. Either 3x PVE nodes or 2x PVE +1x something else (could be your PBS) as a qdevice to act as the third voter for quorum.
See here for cluster basics: https://pve.proxmox.com/wiki/Cluster_Manager
PBS needs IOPS performance so SSDs are recommenden. I wouldn't use HDDs if you want to store more than maybe 1 or 2 TBs of backups.
So a raidz1 aka raid5 and then using ZFS relication? Then keep in mind that you have to increase the block size of the ZFS storage to at least 16K before creating your first VM or you will waste a lot of space. And ZFS isn't a shared storage. If you care about HA then keep in mind that ZFS only syncs every 1+ minutes. So you will always lose the data since the last sync in case a node fails.
I opted for this solution: two PVEs and a PBS as quorum controller as putting 3 PVEs + 1 PBS would have been too expensive. So I thought about managing virtual machine replicas with PBS incremental backups. I would put a replication time at 1 minute, in this system I will have a maximum of 3/4 virtual machines, one of which only with windows and sqlserver db. The rest of the other cars will be CENTOS. I have a 1GB switch, the server lans are 10GB. To have a 3 node cluster I should also have a switch of at least 10GB.... Do you follow me in my reasoning? At this point I format the disks in ZFS, but what were you referring to when you talked about 16K, perhaps in the part where the RAID is created during the installation of proxmox?
Search this forum for "padding overhead". Every Zvols gets created with a volblocksize that can't be changed later. What volblocksize will be used, for creating new zvols, is defined in PVE by the "block size" field of your ZFS storage. It defaults to 8K and that is always bad when running any kind of raidz. Minimum volblocksize to not waste tons of capacity in a 3-disk raidz1 would be 16K when using a pool with the default ashift=12.