Proxmox cluster getting wrong network

[esprox]

Creating a cluster via GUI gets the wrong IP for the cluster.

I have a node I'm already using and now I want to add a second one and form a cluster. The second one is freshly set up and has 0 guests.

First node has 2 NICs, one in a 192.168 net and the other one in 10.x and is running opnsense which acts as DHCP server in the 10. net.

➜  ~ cat /etc/hosts
127.0.0.1       localhost
192.168.188.20  microx.fritz.box        microx
10.13.37.20     microx.opnsense         microx

➜  ~ cat /etc/network/interfaces
# network interface settings; autogenerated
# Please do NOT modify this file directly, unless you know what
# you're doing.
#
# If you want to manage parts of the network configuration manually,
# please utilize the 'source' or 'source-directory' directives to do
# so.
# PVE will preserve these directives, but will NOT read its network
# configuration from sourced files, so do not attempt to move any of
# the PVE managed interfaces into external files!

source /etc/network/interfaces.d/*

auto lo
iface lo inet loopback

auto eno1
iface eno1 inet dhcp

iface eno1 inet6 auto

iface eno2 inet manual

auto vmbr0
iface vmbr0 inet static
        address 192.168.188.20/24
        gateway 192.168.188.1
        bridge-ports eno1
        bridge-stp off
        bridge-fd 0

auto vmbr1
iface vmbr1 inet static
        address 10.13.37.20/24
        bridge-ports eno2
        bridge-stp off
        bridge-fd 0

Second node is connected to the 10.x nic and has a 10.x ip itself (always issued the same).

cat /etc/hosts
127.0.0.1       localhost
10.13.37.21     thinkpox.thinkpox.local thinkpox

# The following lines are desirable for IPv6 capable hosts
::1     localhost ip6-localhost ip6-loopback
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters

I created a cluster on microx in the 10. net:


And it's showing my node in the cluster list, but the join info shows the 192. IP:


Can I change that? Why does it choose that IP?

 

[alexskysilk]

that IP is not important, it can be any ip that pveproxy listens to. the next screen will prompt you to enter the relevent corosync ip.

btw, why are you obtaining dhcp for eno1 AND setting a static on its child bridge?

 

[esprox]

It did not work with the default joining method because after pasting the Join Information, it set the IP and didn't let me change it, but unchecking the "Assisted join" box let me set the ip in the joining dialogue. Thanks!

As for the interface config, I wasn't sure if the router accepts my static IP (or how that specific step works in general tbh) when I set that up and just took the bridge config from the installation manual (manual proxmox on (encrypted) Debian) and I ended up with that config which worked. eno2 wasn't used then and I added it later. Guess I can switch the dhcp setting to manual but one thing at a time.

 

[esprox]

While they form a cluster now and both are green in the ui, the second node can see some resources, but not the content of directories/zfs of the other host and also not starting guests.
I get Connection error 595: Connection timed out
Checking /etc/pve/.members file on both machines shows that the first host has the 192.168 ip. Traffic from 10 to 192 net is blocked, but when I lift the firewall rule, the second node can access everything. Should I just edit the .members file on both nodes or what is the proper way to fix this?