Proxmox Networking Issue: Creating 02 different networks

[proxnoober]

Dear all,
I am trying to setup 02 different isolated networks to basically simulate 2 different locations and create VMs in these separate networks.


My problem
Although in theory these VMs should be in different subnets hence should not be able to communicate with each other. But in my case VMs can ping each other. I am still trying to figure why this is happening and what did I do wrong. Your kind help would be highly valuable Below are the steps I followed so far. I have only 1 node in proxmox.

Networking part
I referred to https://pve.proxmox.com/pve-docs/chapter-pvesdn.html
Create 02 Zones of type simple inside Datacenter > SDN > Zones (eg; zone1, zone2)
Create 02 VNets (eg; net1, net2) 01 in each zone
create 1 subnet in each network.

• net1 > subnet = 10.0.1.0/24. Gateway = 10.0.1.1 SNAT=yes
• net2 > subnet = 10.0.2.0/24. gateway = 10.0.2.1 SNAT = yes

In the node (pve1) when I check /etc/network/interfaces it shows as below
auto lo
iface lo inet loopback
iface eno1 inet manual
iface eno2 inet manual
iface eno3 inet manual
iface eno4 inet manual
auto vmbr0
iface vmbr0 inet static
address 172.22.xx.xx/20
gateway 172.22.xx.x
bridge-ports eno1
bridge-stp off
bridge-fd 0

proxmox management
iface idrac inet manual
source /etc/network/interfaces.d/*

VM Part
While creating the VM I assigned 1 VM to net1 and other device to net2. When I check the VM hardware is correctly shows it (eg; bridge=net1, firewall=1, bridge=net2, firewall=1).
Then I created VMs without setting up IPs initially because it kept saying whatever IP I gave is not belongs to the /24 subnet. After booting up the VM, I manuallt edited the /etc/netplan default configuration file as below (config for the VM attached to net1). OtherVM also same except for the IP being 10.0.2.10/24.
network:
ethernets:
ens18:
dhcp4: false
addresses: [10.0.1.10/24]
gateway4: 10.0.1.1
nameservers:
addresses: [8.8.8.8]
version: 2
Then I ran sudo netplan apply. The IP is correctly configured when checked with IP a command.

Thank you!

 

[shanreich]

As long as SNAT is active, those VMs will be able to reach each other, since the connections between those subnets will be routed via the host. You can explicitly deny this by configuring a respective firewall rule that forbids traffic between those two subnets.

 

[proxnoober]

Dear Stefan, Thank you for the reply. I now got some idea of SNAT. So, I have disabled the SNAT (unchecked the box) and tried but still VMs in different network subnets can talk to each other and can access internet. I am not sure how it's happening in the background. If I am to add any firewall rules can I do it within Proxmox.

UPDATE: When SNAT is disabled the internet access was removed. It took sometime but cannot ping google.com. That is expected. But still the VMs can reach each other despite being in different networks.

UPDATE 2: I created 02 Linux bridge networks (node > system > network) names vmbr1, vmbr2 without any IP configurations. Then, I attached 03 VMs into vmbr1 and other 3 to vmbr2. I configured IPs manually inside the VMs. VMs connected to vmbr1 have 192.168.100.0/24 where as VMs connected to vmbr1 have IPs in subnet 192.168.200.0/24. Now, each VMs inside same network can ping each other and cannot ping between the VMs in other network which is expected. However, I have completed 1 part of the task. Now, I have to enable internet access from the VMs individually. Could you please help me on this. Thank you!



My node is accessible through 172.22.xxx.xx/20. However, when I add a Linux Bridge (Node > System > Network ) as vmbr1, vmbr2 and connect VM to these and try to ping between networks it doesn't work as expected. Also, cannot access internet. vmbr0 is created when I installed Proxmox. When I access the node through SSH I can ping to internet. But, I cannot ping to 192.168.100.10 and 192.168.200.10. However, can ping to gateways (192.168.100.1 and 192.168.200.1).


I hope to have internet access to each VM. But, each set of VMs (03 per network) should not have access to other set of VMs simulating that these VMs are in 2 different sites. I am sorry I tried my best to explain the situation and hope your kind advice.

vNets created in SDN



Thank you!

 

[shanreich]

UPDATE 2: I created 02 Linux bridge networks (node > system > network) names vmbr1, vmbr2 without any IP configurations. Then, I attached 03 VMs into vmbr1 and other 3 to vmbr2. I configured IPs manually inside the VMs. VMs connected to vmbr1 have 192.168.100.0/24 where as VMs connected to vmbr1 have IPs in subnet 192.168.200.0/24. Now, each VMs inside same network can ping each other and cannot ping between the VMs in other network which is expected. However, I have completed 1 part of the task. Now, I have to enable internet access from the VMs individually. Could you please help me on this. Thank you!

Hi! The issue here is that as soon as you use the host as gateway or use the host to do NAT you will get connectivity between those two networks. This is due to the fact that the packets get routed via the host and the host can reach both networks.

The easiest way to go about this would be your initial setup with a simple zone + SNAT. Additionally to the simple zones, you will have to create firewall rules that allow local traffic inside the subnet. The most convenient way to accomplish this would be to additionally create a security group for each simple zone on the datacenter layer like this:

[group simple1]
IN ACCEPT -source 192.168.100.0/24 -dest 192.168.100.0/24 -log nolog

[group simple2]
IN ACCEPT -source 192.168.200.0/24 -dest 192.168.200.0/24 -log nolog

Then you will need to enable the firewall for your guests and add the respective security group to the firewall rules of every guest. Sadly this is currently a bit cumbersome, but I am currently working on improving this.