multihomed qdevice not supported?

[Paul B. Henson]

I'm setting up a proxmox cluster with two nodes and a qdevice. Each node has a separate physical connection for two cluster networks, each pair going to a different switch. I'm running the qdevice in a container on a synology nas providing NFS shared storage to the cluster. The nas also has two physical interfaces for the two cluster networks (these are all 1Gbs ports, there are also 10Gbs ports for NFS and migration traffic).

I have everything set up, but it appears the qdevice only supports one IP address in the cluster configuration? The qnetd process is listening on both IP addresses, one on each cluster network, but based on the documentation the backup interface can not be used.

Why is that? Having the qdevice only available on one of the cluster networks reduces the availability of the cluster. For example, if the qdevice is on cluster network A, and that network fails, the two nodes still have quorum with 2 votes on cluster network B. However, if there is then a cascading failure that removes one of the nodes from the network, the remaining node will lose quorum and fail. If the remaining node was able to communicate with the qdevice on cluster network B, it would still have 2 votes and service would continue.

Thanks...

 

[waltar]

See this for your wished config https://forum.proxmox.com/threads/configuring-dual-corosync-networks.104991/

 

[SteveITS]

I don’t see a reference to Qdevice in that thread…I was under the impression it could have only one IP?

 

[Paul B. Henson]

Yes, I was going to say the same thing. My actual proxmox nodes are properly connected via two separate corosync links with physically divergent switch connections.

But from looking at the documentation for the qdevice, it only supports a single IP address, not multiple. That seems like a single point of failure for high availability. While the loss of the qdevice itself won't break anything, the lack of redundancy in its connection leaves failure modes that will result in a split brain condition that would be avoided if it was connected to all of the networks used by corosync.

 

[SteveITS]

I think the answer is that it doesn’t use the same mechanism it doesn’t need to be a local IP, but it still assumes it’s reachable multiple ways.

 

[Paul B. Henson]

Well, yes, the documentation is fairly clear that the qdevice doesn't communicate via the corosync protocol, just a standard tcp connection.

Theoretically the single IP address could provide some form of redundancy using bonded interfaces, anycast routing, or something. But given a deployment where two physically divergent network links have already been created for the nodes and a qdevice dedicated to this cluster is running locally, it makes a lot more sense and is a better use of resources to allow the qdevice to be multihomed and avail of the network infrastructure already purpose build for that need.