Cluster creation (no bond supported?)

[jompsi]

Hi all

I have a question about creating a cluster.

We use two servers, which have each four 1GB NICs. We work with local storage. The most importent VMs are Actice Directory und fileserver. These VMs are redundant. AD1 is on hypervisor1 and AD2 is on hypervisor2.

Now I just wanted to make a bond(802.3ad/LACP) of this four NICs. Create a cluster and all good. But I read on the page: https://pve.proxmox.com/wiki/Network_Configuration the following:

If you intend to run your cluster network on the bonding interfaces, then you have to use active-passive mode on the bonding interfaces, other modes are unsupported.

I am not sure if I should make a LACP bond with two NICs. Use this as a bridge and let all VMs communicated over this and create one Actice-Backup bond for the cluster communication. But I have the feeling this isnt optimal.

How would you do this?

Thanks and kind regards
jompsi

 

[Klaus Steinberger]

LACP works well, of course the switches have to be configured appropriately.

We run two clusters with LACP without any quirk.

(what is not working: any bonding types other than 1 or 4 -> depending on switches you can run into any network hell you can think of)

Some more points:

At least if your VM's have to talk via a router, use xmit_hash_policy = layer3+4 (with layer2 they talk always via the same port due to the MAC of the router)

Use xmit_hash_policy = layer2 only if most of traffic is local to the subnet.

Instead of classic Linux bridge you can also use OpenVswitch with LACP, it works very well with the packages from the Proxmox Repo (the original Debian Stretch packages are crap!). We first run our clusters with bonding type 4 (LACP), and then migrated to OpenVswitch as it has advantages with many VLAN's (no need for a bridge for every VLAN, we have a real large number of VLAN's)

 

[jompsi]

Switches support LACP and will be configured.

We have no VLANs, so I think I will stay with bonding type 4(LACP).

As hash policy I use layer2+3 as recommended in this book: https://www.packtpub.com/big-data-and-business-intelligence/mastering-proxmox-third-edition

This policy creates the transmission hash based on the combined MAC and IP addresses. These are also the layer 2 and layer 3 protocols of the network layer. This policy also sends the network traffic to a destination on the same slave interface. However, it provides more balanced network transmission than just using the layer 2 policy. For best performance and stability, use this policy.

Thanks for your tips. I will try it with one bond which holds all four NICs.