Proxmox network configuration issue and best practise

[nissens10]

Hi,

I've recently started working with Proxmox and built a 3-node cluster.

My nodes have the following addresses:
node1 - 10.140.128.1/9
node2 - 10.140.128.3/9
node3 - 10.140.128.5/9
Additionally, I have a special subnet for VMs: 10.140.134.0/24.

Currently, my network configuration looks like this:

auto lo
iface lo inet loopback

# Physical if
auto eno8303
iface eno8303 inet manual

# brone
auto brone
iface brone inet static
    address 10.130.128.1/24
    bridge-ports eno8303
    bridge-stp off
    bridge-fd 0
    up ip route add default via 10.130.128.254 metric 300
    dns-nameservers 8.8.8.8

# VLAN conf
auto eno8303.2
iface eno8303.2 inet manual
    vlan-raw-device eno8303

# brtwo
auto brtwo
iface brtwo inet static
    address 10.140.128.1/9
    bridge-ports eno8303.2
    bridge-stp off
    bridge-fd 0
    up ip route add default via 10.140.0.121 metric 200
    dns-nameservers 8.8.8.8

Additionally, I created a new bridge (vmbr0) on each node via the Proxmox GUI with the following addresses:
node1 - 10.140.134.1/24
node2 - 10.140.134.2/24
node3 - 10.140.134.3/24

I want to enable HA in the cluster, be able to SSH into my VMs and allow VMs to connect to the Internet.
What is the best practice for configuring networking in this case? Should I configure nodes as routers? Would it be better to create a dedicated VM as a router on each node? Are there any alternative solutions I should consider?

Thanks in advance!

 

[UdoB]

Not an answer to your question but just a remark: both networks 10.130.128.1/24 + 10.140.128.1/9 are overlapping as 10.130 is included in /9.

If this is intended: fine. If the implications are not clear...

To clarify:

~$ ipcalc 10.140.128.1/9
Address:   10.140.128.1         00001010.1 0001100.10000000.00000001
Netmask:   255.128.0.0 = 9      11111111.1 0000000.00000000.00000000
Wildcard:  0.127.255.255        00000000.0 1111111.11111111.11111111
=>
Network:   10.128.0.0/9         00001010.1 0000000.00000000.00000000
HostMin:   10.128.0.1           00001010.1 0000000.00000000.00000001
HostMax:   10.255.255.254       00001010.1 1111111.11111111.11111110
Broadcast: 10.255.255.255       00001010.1 1111111.11111111.11111111
Hosts/Net: 8388606               Class A, Private Internet

 

[nissens10]

Not an answer to your question but just a remark: both networks 10.130.128.1/24 + 10.140.128.1/9 are overlapping as 10.130 is included in /9.

If this is intended: fine. If the implications are not clear...

I'm aware of that and the overlapping is intended.

 

[ghusson]

Nodes shouldn't do any function other than virtualization ( and maybe CEPH storage ).
Thus one or several VMs should do routing / firewalling.
Be careful with your overlapping networks as it can be dangerous (security) and prone to mistakes in network flow understanding.

 

[nissens10]

Nodes shouldn't do any function other than virtualization ( and maybe CEPH storage ).
Thus one or several VMs should do routing / firewalling.
Be careful with your overlapping networks as it can be dangerous (security) and prone to mistakes in network flow understanding.

This is exactly what I started doing, but I'm afraid of reliability and security. So, I'm searching Internet for some examples and explaination how to do it, but I haven't found anything.

However, thank you so much for the advice.

 

[ghusson]

My ideas :
Reliability : use several firewalls/routers, with VRRP protocol for example in order to have Active/Passive firewalls.
Security : use dedicated cluster servers with low level hardware in order to avoid lateral attackers moves.
You are welcome.