VM in a DMZ

[mladen popov]

Hey guys,

Im trying to implement a VM with SecureLink which should be residing in the DMZ zone. The proxmox server has both nic bonded and connected to our main network 192.168.10.x which is connected to port 4 of our firewall/gateway. On port 5 i have the DMZ/network 192.168.42.1, nothing is physically connected port 5. The interface is pingable from the proxmox server and from everywhere, as the DMZ is not enforced yet. But when i setup statick IP on the VM 192.168.42.5 with gateway .1 i cant access anything. What am I missing here?

Thanks,
Mladen

 

[Adayge]

Generally speaking, unless you need management on an interface I would recommend against binding IP addresses for Proxmox on bridge interfaces other than your main one. Proxmox works quite will with non-IP'd bridge interfaces given to VMs. In fact, on DMZ networks you would specifically want Proxmox to NOT be manageable on this subnet. Removing the IP allows Proxmox to 'stand back' and allow for the bridge to function as a 'switch' for VMs on it without the potential for passthrough traffic.

The 'tcpdump' command line program will demonstrate whether or not traffic is coming to or from your interface. You can also validate that your physical interface is actually associated with the bridge. For example:

root@prox2:~# brctl show

bridge name	bridge id	STP enabled	interfaces
fwbr100i0	8000.9a6b3ec29b61	no	fwln100i0
			tap100i0
fwbr100i1	8000.222e0c4690cc	no	fwln100i1
			tap100i1
vmbr0	8000.08f1eaeba82c	no	enp3s0f0
			fwpr100p0
vmbr1	8000.08f1eaeba82d	no	enp3s0f1
			fwpr100p1

If your interface is not on your bridge, it will fail to egress. If it is on your bridge, you can use 'tcpdump' on either the bridge and/or on the interface. For example:

tcpdump -n -I vmbr1

This will tell you if you are getting info or not through the interface. Simply swap out the physical interface to see the same traffic in the data path.

 

[mladen popov]

Hi Adayge,

I can ping the DMZ interface(192.168.42.1) from the proxmox(ip addr 192.168.10.34) server itself but i cant ping anything from the VM(192.168.42.5)

edit:
for people to get a better idea what the current setup is im attaching a quick networking diagram

 

[Adayge]

Your Proxmox server needs a network card that is exclusively for the 192.168.42.x network. It should be a bridge and it should be plugged into the DMZ switch or router.

For the 192.168.10.x network, it seems that you are using the 192.168.10.34 as your management address. It is likely true that this interface is also on a bridge so that any VMs that are configured for the 192.168.34.x network. You seem to also have an interface for 192.168.42.x network. You should remove any IP address assigned to Proxmox for this interface. VMs can still use it if they have addresses in the DMZ but Proxmox should not have any interface bound to this. You will plug this Proxmox interface into the same switch as all your other 192.168.42.x devices. That way they can talk to each other but not to Proxmox management.

I have something similar configured for my firewall which is virtual:

Internet
^
v
Switch1 port 1
Switch1 port 2 <> PVE Server 1 (prox1) <> Switch 2 port 1
Switch1 port 3 <> PVE Server 2 (prox2) <> Switch 2 port 2

Each one of my PVE servers has 2 NICs. One is plugged into my public internet facing switch and the other is plugged into my LAN facing switch. I configure my PVE servers to only have 1 IP address on them on the LAN side of the switch. I virtualize my firewall which is the default gateway of my PVE servers. The virtual firewall has both interfaces configured and the two servers are in a cluster and can migrate the firewall between them.

If I create additional VMs connected to the public switch, it is not necessary for them to be able to ping the Proxmox VE server so long as they are able to ping the firewall and other things on the network. I did have some issues when I first set this up because the interface was not bound to the vmbr0 device properly (I didn't properly save the configuration and it vanished on reboot).

Double check your bridge to interface assignments and remove extra IPs from your Proxmox server, especially on DMZ networks. Another good troubleshooting tool is to set up multiple VMs so that you can do simple ping tests without firewalls.

 

[mladen popov]

So I scrapped out an old server so i dont play around with a live system. Dropped proxmox and on one lan i have 192.168.10.10 as an ip to connect to our main network. And for the second one i have 192.168.42.3 and for the VM i have .42.5. Which interface should i bridge?

 

[Adayge]

Bridging is what your switch does. Imagine that you have a server with 48 network cards on them. You could configure that server to bridge various physical ports together or you could configure a bridge with zero ports on them. Not really useful but the concept of a bridge means that physical interfaces start working collectively together. Some wireless routers, for example, are great illustrators of how this works. Internally, the wireless component is its on network interface, but when bridged with the other 4 ports on the wireless router, it becomes a cohesive layer2 network and traffic can pass from host to host plugged in to it regardless of whether or not the IP is configured for the wireless router.

The same thing goes with Proxmox. Adding physical interfaces to the bridge just means that there are additional physical ports on this bridge/switch. For VMs, plugging them into a bridge is as if there is a virtual cable plugging the vm into the network attached.

This is useful because you can have two VMs talk to each other on a bridge that has zero physical interfaces. Imagine in Proxmox, that you have a server and a backup server. You could make it so that the backup server only has a connection to the other server and doesn't have an internet connection. On your proxmox server, this network would consume no physical interfaces unless you wanted to add them. But for the VMs, they can talk to each other because the bridge is essentially a switch which contains a number of ports equal to the things you plug into it. For two VMs, it would be a 2 port switch.

When you assign an IP address to a vmbrN interface, you are merely adding the physical machine to the same bridge/switch.

 

[Sprinterfreak]

I have to disagree, that an interface/bridge wich just has no IP given, is not listening or reachable via its connected L2 network.

Why?
By default, the kernel always sticks his plug into a linux-bridge.

- Interfaces (vmbrX) still pick up ipv6 adresses if the network it switches announces one
- the vmbr (the host) can still be reached by pinning its mac address
- the host kernel can be reached via non-ip services in general

Currently there is no elegant way of proper isolating a bridge from the kernel. Hint: Feature request "isolated [x]" on interfaces config
Read more: e.g. https://vincent.bernat.ch/en/blog/2017-linux-bridge-isolation

As a measure of precoution, we just can enable Firewall on Datacenter level and create a rule for each untrusted bridge interface like "in, DROP"

 

[Sprinterfreak]

I've recently learned the solution to this.

If VMs have to be configured to multiple vlans, don't create a bridge vor every vlan on the host. A linux bridge is by design an interface for the host to be reachable in.

Just use one bridge with "vlan aware" enabled and only use this one bridge as host interface for VM's. If VM NIC's have to be configured into vlans, use the VLAN-Tag-field in the VM's network card settings instead of creating a bridge for every vlan.

The downside of this is, that there seem to be no way to name/describe things like it would be if a bridge is created for each vlan.