Creating a separate LAN for new cluster - and feeling foolish

[holckj]

Hello

I'm feeling foolish, I really believe this should be quite simple. I have 2 Proxmox hosts, connected to the same LAN with a gateway to the world outside. This works fine.
Now I want to create a separate LAN to use for the cluster I want to set up. Each hosts has 4 ehternet connections. Below is the contents of /etc/network/interfaces for the two hosts. Both hosts are connected to a simple switch for the local LAN, and a router for the "standard" LAN. But they can't ping each other in the 10.10.10.* network.

Please help

Host 1

auto lo
iface lo inet loopback
iface eno4 inet manual
iface eno1 inet manual
iface eno2 inet manual
auto eno3
iface eno3 inet manual


auto vmbr0
iface vmbr0 inet static
        address 192.168.1.232/24
        gateway 192.168.1.1
        bridge-ports eno4
        bridge-stp off
        bridge-fd 0


auto vmbr1
iface vmbr1 inet static
        address 10.10.10.232/24
        bridge-ports eno3
        bridge-stp off
        bridge-fd 0

Host 2

auto lo
iface lo inet loopback
iface eno1 inet manual
iface eno2 inet manual
iface eno3 inet manual
iface eno4 inet manual

auto vmbr0
iface vmbr0 inet static
        address 192.168.1.167/24
        gateway 192.168.1.1
        bridge-ports eno1
        bridge-stp off
        bridge-fd 0

iface usb0 inet manual

auto vmbr1
iface vmbr1 inet static
        address 10.10.10.167/24
        bridge-ports eno3
        bridge-stp off
        bridge-fd 0

 

[HLPCLC]

Personally, I have a pfSense router at home, and it has a trunked VLAN with multiple DHCP servers for each VLAN, with the trunked VLAN connecting to a managed switch. This setup lets me create new subnets pretty quickly. It is a bare-metal pfSense router, but I have heard of people virtualizing a pfSense machine, and using the machine to network all of their devices more quickly.

People have reported being able to network their nodes and whatnot through pfSense on proxmox (which is why I'm here today). If you are able to buy a managed switch you can use 802.1Q to quickly deploy subnets in pfSense, and even set up firewall rules to secure them from eachother; if one cluster or node is hacked, it can be recorded or prevented from communicating with other nodes.

I am using a laptop as a proxmox server x_x and have a very nice desktop PC I am using to run pfSense baremetal, but I really need to switch over to a virtualized pfSense to save on monthly electric bills. My bare metal pfSense can draw about 300 watts but it can route VPN at gigabit speeds which is really cool.
Edit: I use TP-Link TL-SG105e and TP-Link TL-SG108e for some pretty versatile and secure networking, but don't have a CLI yet Hope this helps.

 

[HLPCLC]

Here is where I learned to manage this network setup and more. This guy is really cool.

https://www.youtube.com/watch?v=5ohLAFHnOHg

 

[HLPCLC]

Here is another video

https://youtu.be/lUzSsX4T4WQ