best practices in network configuration

[ITKnow]

I have two dedicated servers at OVH, each one of them has two network adapters, one for Internet access and the other connected to the vRack through which the two servers are connected.
One of the servers in production with VM, the other recently installed in order to set up a high availability cluster.
My question is the following: what would be the best practice to configure the different networks of the cluster?

Right now I have the two adapters configured on the two servers, the one connected to the internet with the configuration and public IP provided by OVH (linus bridge) and the internal adapter with another Linux Bridge and the same segment on the two servers.

What would be the best configuration to add the following networks?

• Cluster Management Network
• Storage management network
• Management Network
• Common services network
• Work network client 1

Different clients will work on the same server in isolation, but I would like them to be able to access a specific network where shared services can be located.
I have a PfSense installed on the server in production.

Thanks for your advice.

 

[Dunuin]

First, you need at least 3 servers to form a cluster. Otherwise there is no quorum in case a node fails and the second node will stop working too.
And in an ideal case you got at least 3 NICs on each server. Corosync needs low latency so you best give it dedicated NIC (or at least a NIC that is never heavly used). Then a public NIC to the internet. And a dedicated NIC just for the storage.

 

[ITKnow]

I did not know that it could not be done if it was not with 3 nodes.
Are the 3 nodes really necessary to mount the cluster?

I mean if I could set up the cluster with only 2 nodes but without high availability, to make backups from one node to another.

I currently do not plan to expand to a third node.

As for the physical network adapters, they are what they are and I cannot change them, one for internet traffic and another for internal traffic, so if I have to mount adapters they have to be on this internal physical one.

I don't know if linux bridge is better, OVS...
That's why I was asking what the recommendation was.

 

[Dunuin]

I did not know that it could not be done if it was not with 3 nodes.
Are the 3 nodes really necessary to mount the cluster?

I mean if I could set up the cluster with only 2 nodes but without high availability, to make backups from one node to another.

I currently do not plan to expand to a third node.

No, you really need at least 3 servers to form a cluster...using HA or not. Otherwise, the whole cluster will fail if one node goes down. You either need at least 3 PVE nodes or 2 PVE nodes + additional host that is a qdevice and acting as a third voter (this also could be a cheap virtual server in the same datacenter).

 

[ITKnow]

OK I understand
As you can see I'm new to Proxmox
Assuming I have 3 nodes, what about the network adapters? How can I manage the different networks with only two physical adapters in each server?

 

[Dunuin]

First question would be what storage you want to use, as you need replication or a shared storage.

NFS as a shared storage got the problem of single point of failure. When the node with the NFS server goes down all 3 nodes will stop working.

ZFS with replication is only synced local storage. So not that great for HA as it is never perfectly synced so you will lose the last couple of seconds or minutes of data when a node goes down. But at least it will work with just 2 nodes + qdevice.

Ceph would be the best choice as it is a real shared storage with redundancy. But ceph needs at least 3 nodes and multiple fast NICs (10Gbit or higher).

So depending on what storage you use there are different network requirements.

 

[ITKnow]

I see that I am limited, although the machines that are in production are not high performance, I only have 1Gb adapters.
As for storage, I have a Raid 1 with two 500Gb SSD drives and then a ZFS-mounted raid 1 with two 6Tb drives.

I'm currently backing up to a remote NAS via NFS, local production machines.
But I would like to make backup copies from one Proxmox server to another, in order to be able to raise the copy on the second server if there is a problem with the first server, their specs are the same.

I had thought about the CEPH option you mentioned, hiring a third server, but initially discarded it due to the issue of 1 Gb network adapters.

Options?

 

[Dunuin]

With just 2 nodes you could run them unclustered with a proxmox backup server (PBS) on each node. Then you could setup a sync tasks so the datastore of the primary PBS gets synced every hour/day to the secondary PBS. If then a server fails you at least got a recent backup on the remaining node you could restore.

 

[ITKnow]

Thank you very much for the information, as I had ruled out the third node for the moment, just today I had started to study the Proxmox backup, I did not know that it could synchronize the datastores between the two nodes, that is very interesting since even if it grows to a third node It will be very difficult to implement CEPH due to the speed of the network adapters.
Does Proxmox Backup need a network for these synchronizations?
I suppose that being a virtual machine, an adapter like in any other machine, correct?

 

[Dunuin]

You don't need to run PBS as a VM. It can be VM, LXC or even bare metal next to PVE.
And yes, would be good to have a dedicated NIC for the PBS sync. But not that important as low latency isn't an requirement. But network performance of your guests might be bad while PBS is doing it's sync pushing a full 1 Gbit of data over the NIC leaving no bandwidth for other stuff. So you might want to limit the PBS bandwidth to something like 500Mbit/s or similar.