2 PVEs and a PBS - sanity checking my plan

mgaudette112

Member
Dec 21, 2023
30
1
8
Hi,

I'm planning a 3 server install - 2 PVEs in a cluster (with a direct SPF+) between them - I know it takes 3 to build a cluster, hence my first questions below.

The third machine will be a PBS install. I am hoping to use it as a qdevice to build the necessary quorum (this works right?). Also, in an effort to make this as easy as possible, I want to connect the PBS to the two PVEs using my second SPF+ port of each PVEs.

Here are my relevant question:

1) The dedicated PBS as a qdevice - good plan?

2) To avoid making things overly complicated (mesh network comes to mind as a working but complicated solution) I am simply planning to make a bridge of 2 SPF+ ports on the PBS, and connect each SPF+ port to one of the PVE. Does this make any sense? I'm not very familiar with using two ports as a switch on a Linux device, but since none of the PVEs will communicate to each other on this network (in theory) the issues of lesser performance vs a switch is not that important (I think).

The point is that the cluster network and the PVE-to-PBS network will render the need for a switch unnecessary - my stacked/redundant switches do not have enough SPF+ ports and with this plan I don't need it any, as long as the PBS server has one SPF+ port per PVE (right?).

3) As a bonus, the PBS server might have a separate ZFS pool of disks (not the one where the backups reside) used for an NFS share (provided to PVEs as VM storage) - I understand the danger of making backups on the same server as the source data, and I plan to mitigate that anyways by having a remote PBS server anyways. But, besides this caveat, are there any reasons not to do this?

4) I could instead a PVE on the third server and make it a PBS too - while I don't plan on needing it as a VM host, I guess it couldn't hurt to have it available in a pinch - but I did read that sometimes there are conflict when major upgrades are released, and PVE/PBS have different prerequisites in term of kernel versions etc. Is this still current as an issue?

Mostly looking for reassurance here that I understand the basics. I am running a single PVE right now with very basic settings, as a bit of a test.
 
2) To avoid making things overly complicated (mesh network comes to mind as a working but complicated solution) I am simply planning to make a bridge of 2 SPF+ ports on the PBS, and connect each SPF+ port to one of the PVE. Does this make any sense? I'm not very familiar with using two ports as a switch on a Linux device, but since none of the PVEs will communicate to each other on this network (in theory) the issues of lesser performance vs a switch is not that important (I think).
Keep in mind that corosync wants <1ms latency. Would be a good idea to have a dedicated Gbit connection for corosync so migrating or backing up VMs won't increase the latency when saturating the 10Gbit connections.
 
Keep in mind that corosync wants <1ms latency. Would be a good idea to have a dedicated Gbit connection for corosync so migrating or backing up VMs won't increase the latency when saturating the 10Gbit connections.

I guess I could use the main network (the one connected to the switches, that I would use to access the web UI.).

But isn't this also the case for the cluster network, and the same issue applies during migrations? But default I understand the migration uses the same link as corosync.

Edit - I reread your reply in which you mention this exact issue. Thank you, will rework my plan accordingly.
 
Last edited:

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!