Help with VM isolation via VLANs

Venumo

New Member
Nov 15, 2022
2
0
1
I'm trying to setup networking to where I can just tag a vm with a vlan so that it can only communicate out to the internet and to other VM's inside that vlan. These VM's are in a private network as well, so I know some form of masquerading will be necessary.
I first tried using a vlan aware bridge, but a tagged vm can't access the gateway and therefore can't reach the internet. So I've started looking into openvswitch however I can't figure it out. Any advice? (And I have read this post: https://forum.proxmox.com/threads/isolating-virtual-machines-with-vlan.92797/ but using firewall rules based on ip won't work for my situation without a lot of extra work)

More detail on my situation if it helps:
I'm currently working on a project for my school where cyber security labs are to be created automatically inside Proxmox, these labs can be 1 to 3 VMs each. I've already got automation using the API going pretty well, my only task I'm stuck on is keeping labs isolated as they can be running malicious code (the rest of the network at our campus is already isolated). I also have no access to a physical switch, it is under lock and key and getting any changes made takes months here.

Any help would be appreciated thanks.
 
If you create a 'virtual network' for your vm's inside proxmox - they will be able to communicate with each other easily and without needing VLAN's. The trick is then allowing the VM's to reach the internet without reaching elsewhere and using VLAN's alone would not achieve this in any case.

If we assume your 'real' LAN is using 172.16.100.0/24 and your proxmox host is on 172.16.100.50 - for example your network config might look like this
Code:
iface lo inet loopback

auto ens18
iface ens18 inet manual

auto vmbr0
iface vmbr0 inet static
        address 172.16.100.50/24
        gateway 172.16.100.1
        bridge-ports ens18
        bridge-stp off
        bridge-fd 0

This gives both your proxmox host and any VM's attached to vmbr0 access to your 'real' LAN

If you add the following
auto vmbr1 iface vmbr1 inet static address 192.168.10.1/24 bridge-ports none bridge-stp off bridge-fd 0

Note that vmbr1 is not connected to any physical network port (bridge-ports none) so the VM's attached to vmbr1 can talk to each other but cannot reach out to the internet - so you have your isolated network.

So that VM's on vmbr1 can reach the internet, there will need to be some form of network routing. Either via VM with two network interfaces, one on vmbr0 and one on vmbr1 (something like pfsense or opnsense would do), or some iptables rules in your network config - e.g
post-up echo 1 > /proc/sys/net/ipv4/ip_forward post-up iptables -t nat -A POSTROUTING -s 192.168.10.0/24 -o vmbr0 -j MASQUERADE post-down iptables -t nat -D POSTROUTING -s 192.168.10.0/24 -o vmbr0 -j MASQUERADE post-up iptables -t raw -I PREROUTING -i fwbr+ -j CT --zone 1 post-down iptables -t raw -D PREROUTING -i fwbr+ -j CT --zone 1

If you're using iptables, I'm not sure how you would go about preventing the VM's reaching your main LAN as they need to use that to reach the internet - in that case, you may find it easier to connect your vm's to the internet using a firewall VM such as pfSense as you will be able to define access rules using a GUI

HTH
 
If you create a 'virtual network' for your vm's inside proxmox - they will be able to communicate with each other easily and without needing VLAN's. The trick is then allowing the VM's to reach the internet without reaching elsewhere and using VLAN's alone would not achieve this in any case.

If we assume your 'real' LAN is using 172.16.100.0/24 and your proxmox host is on 172.16.100.50 - for example your network config might look like this
Code:
iface lo inet loopback

auto ens18
iface ens18 inet manual

auto vmbr0
iface vmbr0 inet static
        address 172.16.100.50/24
        gateway 172.16.100.1
        bridge-ports ens18
        bridge-stp off
        bridge-fd 0

This gives both your proxmox host and any VM's attached to vmbr0 access to your 'real' LAN

If you add the following
auto vmbr1 iface vmbr1 inet static address 192.168.10.1/24 bridge-ports none bridge-stp off bridge-fd 0

Note that vmbr1 is not connected to any physical network port (bridge-ports none) so the VM's attached to vmbr1 can talk to each other but cannot reach out to the internet - so you have your isolated network.

So that VM's on vmbr1 can reach the internet, there will need to be some form of network routing. Either via VM with two network interfaces, one on vmbr0 and one on vmbr1 (something like pfsense or opnsense would do), or some iptables rules in your network config - e.g
post-up echo 1 > /proc/sys/net/ipv4/ip_forward post-up iptables -t nat -A POSTROUTING -s 192.168.10.0/24 -o vmbr0 -j MASQUERADE post-down iptables -t nat -D POSTROUTING -s 192.168.10.0/24 -o vmbr0 -j MASQUERADE post-up iptables -t raw -I PREROUTING -i fwbr+ -j CT --zone 1 post-down iptables -t raw -D PREROUTING -i fwbr+ -j CT --zone 1

If you're using iptables, I'm not sure how you would go about preventing the VM's reaching your main LAN as they need to use that to reach the internet - in that case, you may find it easier to connect your vm's to the internet using a firewall VM such as pfSense as you will be able to define access rules using a GUI

HTH
Thank you for your reply, but sorry I should have explained better. I had already basically gotten to this step, my issue is with keeping groups of VM's isolated from one another. I'm spawning VM's from templates dynamically and was hoping I could do something easy like just set the vlan tag through the api. However, when I do this it can't even ping the bridge because it's tagged as a different vlan. To accomplish that I did have to set the vlan aware option on the bridge, so I have tried that.
Is there any way to have translation like NAT but for vlans?


My current fallback idea I've come up with is to just predefine a 50 to 100 bridges with iptable masquerading like your reply, then assign groups of VM's to each bridge and manage which are in use with a database. But this seems dirty and that there should be a better way.

Also, I'm not worried about VM's being able reaching my main LAN, the server I'm working on is inside what is almost a DMZ.
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!