2 PVEs and a PBS - sanity checking my plan

mgaudette112

Member
Dec 21, 2023
39
2
8
Hi,

I'm planning a 3 server install - 2 PVEs in a cluster (with a direct SPF+) between them - I know it takes 3 to build a cluster, hence my first questions below.

The third machine will be a PBS install. I am hoping to use it as a qdevice to build the necessary quorum (this works right?). Also, in an effort to make this as easy as possible, I want to connect the PBS to the two PVEs using my second SPF+ port of each PVEs.

Here are my relevant question:

1) The dedicated PBS as a qdevice - good plan?

2) To avoid making things overly complicated (mesh network comes to mind as a working but complicated solution) I am simply planning to make a bridge of 2 SPF+ ports on the PBS, and connect each SPF+ port to one of the PVE. Does this make any sense? I'm not very familiar with using two ports as a switch on a Linux device, but since none of the PVEs will communicate to each other on this network (in theory) the issues of lesser performance vs a switch is not that important (I think).

The point is that the cluster network and the PVE-to-PBS network will render the need for a switch unnecessary - my stacked/redundant switches do not have enough SPF+ ports and with this plan I don't need it any, as long as the PBS server has one SPF+ port per PVE (right?).

3) As a bonus, the PBS server might have a separate ZFS pool of disks (not the one where the backups reside) used for an NFS share (provided to PVEs as VM storage) - I understand the danger of making backups on the same server as the source data, and I plan to mitigate that anyways by having a remote PBS server anyways. But, besides this caveat, are there any reasons not to do this?

4) I could instead a PVE on the third server and make it a PBS too - while I don't plan on needing it as a VM host, I guess it couldn't hurt to have it available in a pinch - but I did read that sometimes there are conflict when major upgrades are released, and PVE/PBS have different prerequisites in term of kernel versions etc. Is this still current as an issue?

Mostly looking for reassurance here that I understand the basics. I am running a single PVE right now with very basic settings, as a bit of a test.
 
2) To avoid making things overly complicated (mesh network comes to mind as a working but complicated solution) I am simply planning to make a bridge of 2 SPF+ ports on the PBS, and connect each SPF+ port to one of the PVE. Does this make any sense? I'm not very familiar with using two ports as a switch on a Linux device, but since none of the PVEs will communicate to each other on this network (in theory) the issues of lesser performance vs a switch is not that important (I think).
Keep in mind that corosync wants <1ms latency. Would be a good idea to have a dedicated Gbit connection for corosync so migrating or backing up VMs won't increase the latency when saturating the 10Gbit connections.
 
Keep in mind that corosync wants <1ms latency. Would be a good idea to have a dedicated Gbit connection for corosync so migrating or backing up VMs won't increase the latency when saturating the 10Gbit connections.

I guess I could use the main network (the one connected to the switches, that I would use to access the web UI.).

But isn't this also the case for the cluster network, and the same issue applies during migrations? But default I understand the migration uses the same link as corosync.

Edit - I reread your reply in which you mention this exact issue. Thank you, will rework my plan accordingly.
 
Last edited: