Help to setup cluster

[juju01]

Looking to create a cluster with two existing standalone nodes ( with VMs on them) and a new third node. The second node has a truenas vm, using 4 pci passed through disks. I am not entirely sure how to handle this when adding the node to the cluster. I can't migrate it because the disks are on the node. Ideally, I'd like to exclude this truenas vm from the cluster management functions. How will this work?

I also have a question about how to setup a dedicated network for the cluster functions. I have seen tutorials where some folks even setup a separate dedicated nic for migration only, and another separate nic for cluster functions. On all my three nodes, I have just one spare unused nic. What is the best way to use this? VXLAN? EVPN?

 

[aaron]

Looking to create a cluster with two existing standalone nodes ( with VMs on them) and a new third node. The second node has a truenas vm, using 4 pci passed through disks.

A node on which the cluster is initially created can have guests. Any node that is joining the cluster needs to be empty.
Therefore create the cluster on the node with the PCI passthrough TrueNAS VM.

The other node needs to be freed. Either by using a remote-live migration or backup->restore.
You might want to set it up from scratch anyway after you moved all guests away.

I also have a question about how to setup a dedicated network for the cluster functions. I have seen tutorials where some folks even setup a separate dedicated nic for migration only, and another separate nic for cluster functions. On all my three nodes, I have just one spare unused nic. What is the best way to use this? VXLAN? EVPN?

Configure another IP subnet those second NICs and then you can use that as additional Corosync network and specify it as migration network in the DC options.

VXLAN and EVPN is for guests. But from what you told, I don't see the need for that.

 

[juju01]

@aaron Node 1 is the main workhorse - with tons of VMs on it. It also has another truenas vm running there too. It will be much simpler for me to clear the VMs on node 2. How will the backup and restore work in this case?

Update:
--
It seems I am not using PCI to passthrough the disks ( I am doing that on node1 ). On node 2, I am passing through each drive. I have unchecked "backup" in each drive setting and backed up the truenas vm using pbs. But what is the process of adding this to the cluster? Wondering if perhaps I can use this opportunity to passthrough the PCI controller instead?

Will these disks be wiped during the cluster creation?