[SOLVED] Management VLAN

[trigg3r]

I'm new with VLAN and there is one thing I don't understand in the Proxmox documentation about creating the managment VLAN: Network Configuration - Proxmox VE

From what I understand, the purpose of the management VLAN is to separate the subnet used to access the admin interface from the "physical" subnet.

The examples in the documentation do not create this separation, but simply move all traffic into a tagged subnet.

Can anyone help me how this can serve to increase security?

 

[B.Otto]

Hello,

I don't really get what you mean by 'physical subnet'? VLANs (Layer2) and Subnets (Layer3) are different kind of structures. VLANs makes traffic controllable by the network administrator while subnets/IPs are a soft way to segregate traffic which can easily be spoofed by clients in the same network.

The examples put the Proxmox 'management interface' in a different VLAN and that makes it controllable by the network administrator.

Kind regards,
Benedikt

 

[trigg3r]

The examples put the Proxmox 'management interface' in a different VLAN

Not just the 'management interface', but all traffic has to go through that vlan ... so , no traffic segmentation.

I expected to see at least 2 IP addresses configured: one for the untagged subnet to be used in the VMs and one for tagged subnet dedicated to accessing the administration interface.

For example, let's say the Proxmox server interface is connected to my router's DMZ interface: On which subnet should the IP address of the router's DMZ interface be located?

As I understand it, the router's DMZ interface should have one address on the untagged subnet, which acts as a gateway to the VMs, and then a second address on the subnet tagged with the management VLAN ID, to access the VMs interface. administration.

Am I wrong?

 

[Dunuin]

As I understand it, the router's DMZ interface should have one address on the untagged subnet, which acts as a gateway to the VMs, and then a second address on the subnet tagged with the management VLAN ID, to access the VMs interface. administration.

Jup.

I expected to see at least 2 IP addresses configured: one for the untagged subnet to be used in the VMs and one for tagged subnet dedicated to accessing the administration interface.

A IP on the untagged inteface isn`t required for the guests to work (except they need to access some services running on the PVE host itself...for exaple if your PVE host would also work as a SMB/NFS server). Assiging a IP there would also allow to access SSH, webUI and API from the guests...which you normally don't want.

If you assign a IP on the bridge for the guests, then you would need to create custom firewall rules to block port 22 and 8006 to overwrite the hidden anti-lockout rules, so that SSH and webUI/API wouldn't be accessible.

 

[vesalius]

I find it easier to think about a linux bridge (vmbr#) as a switch with unlimited ports. When you define an IP/Gateway at the proxmox host level (with or without vlan) this is like plugging the proxmox host into the switch and not the Guest (vm/LXC). So adding a second IP address in a different Subent/VLan at the proxmox host level would be giving proxmox 2 IP addresses (which is fine if that is what you want to do).

All other Guest traffic are not forced over the management vlan (just because this has an IP/Gateway defined). Going back to the switch analogy. When you set a guest up to use vmbr#, it's like connecting that guest to the vmbr# switch with a virtual ethernet cable. You can manage what subnet/vlan the VM/LXC expects to use within the Guest (like a bare metal device) or in guest (VM/LXC) network hardware proxmox settings.

 

[trigg3r]

@Dunuin @vesalius : thanks so much to both of you: I've read several posts in the forum, but nothing so clear!

For testing I set up the configuration I used for my example: on vmbr0 (set with IP and gateway, as after standard Proxmox installation, but enabling the "VLAN aware" flag) I configured a "Linux VLAN" interface, so I expected Proxmox to respond on this interface as well. However, I realized that the traffic arrives tagged from the router's DMZ up to Proxmox, but then the replies come back untagged because they pass through the gateway set to vmbr0 ...

Thanks again!

 

[trigg3r]

I can't get the VLAN to work between pfSense DMZ and Proxmox ...

The pfSense DMZ interface is directly connected to the Proxmox eth0 interface bridged with vmbr0. I already have some active servers using vmbr0, so to test the VLAN I did this:

• on pfSense I configured VLAN 10 on the DMZ interface assigning the IP address 192.168.10.1.
• on Proxmox I configured a Linux VLAN with TAG 10 on the vmbr0 bridge assigning IP address 192.168.10.253.

So, ping doesn't work.

I used tcpdump to monitor the interfaces on both hosts, trying to ping (both directions between 192.168.10.1 and 192.168.10.253): I see the tagged ARP request, but it remains unanswered (it doesn't even reach the recipient host).

Ping from Proxmox:

ethertype 802.1Q (0x8100), length 46: vlan 10, p 0, ethertype ARP (0x0806), Ethernet (len 6), IPv4 (len 4), Request who-has 192.168.10.1 tell 192.168.10.253, length 28

Ping from pfSense:

ethertype 802.1Q (0x8100), length 46: vlan 10, p 0, ethertype ARP, Request who-has 192.168.10.253 tell 192.168.10.1, length 28

 

[vesalius]

Can you copy your /etc/network/interfaces file here? After making the changes did you reload the Network ifreload -a in a few rare instances people have had to reboot the Proxmox node to get Network changes such as vlans to come up correctly.

 

[trigg3r]

Further reading the Proxmox admin guide I saw that I was doing things the wrong way. So I tried strictly following the official documentation to move Proxmox webGUi to a management VLAN.

First on pfSense I created VLAN 10 (192.168.10.1 ) based on the DMZ interface (192.168.3.1) connected to Proxmox.

Then on Proxmox I went from this configuration (working):

auto lo
iface lo inet loopback

iface eno1 inet manual

iface enp2s0f1 inet manual

iface eth0 inet manual

iface enp5s0 inet manual

auto vmbr0
iface vmbr0 inet static
    address 192.168.3.253/24
    gateway 192.168.3.1
    bridge-ports eth0
    bridge-stp off
    bridge-fd 0

auto vmbr1
iface vmbr1 inet static
    address 192.168.6.253/24
    bridge-ports enp2s0f1
    bridge-stp off
    bridge-fd 0

auto vmbr3
iface vmbr3 inet static
    address 192.168.5.253/24
    bridge-ports eno1
    bridge-stp off
    bridge-fd 0

to this:

auto lo
iface lo inet loopback

iface eno1 inet manual

iface enp2s0f1 inet manual

iface eth0 inet manual

iface enp5s0 inet manual

auto vmbr0.10
iface vmbr0.10 inet static
    address 192.168.10.253/24
    gateway 192.168.10.1

auto vmbr0
iface vmbr0 inet static
    bridge-ports eth0
    bridge-stp off
    bridge-fd 0
    bridge-vlan-aware yes
    bridge-vids 2-4094

auto vmbr1
iface vmbr1 inet static
    address 192.168.6.253/24
    bridge-ports enp2s0f1
    bridge-stp off
    bridge-fd 0

auto vmbr3
iface vmbr3 inet static
    address 192.168.5.253/24
    bridge-ports eno1
    bridge-stp off
    bridge-fd 0

and after reboot:

ip -br a

lo               UNKNOWN        127.0.0.1/8 ::1/128
eno1             UP             
enp2s0f1         UP             
eth0             UP             
enp5s0           DOWN           
vmbr0            UP             xxxx::xxxx:xxxx:xxxx:xxxx/64
vmbr0.10@vmbr0   UP             192.168.10.253/24 xxxx::xxxx:xxxx:xxxx:xxxx/64
vmbr1            UP             192.168.6.253/24 xxxx::xxxx:xxxx:xxxx:xxxx/64
vmbr3            UP             192.168.5.253/24 xxxx::xxxx:xxxx:xxxx:xxxx/64

ip route

default via 192.168.10.1 dev vmbr0.10 proto kernel onlink
192.168.5.0/24 dev vmbr3 proto kernel scope link src 192.168.5.253
192.168.6.0/24 dev vmbr1 proto kernel scope link src 192.168.6.253
192.168.10.0/24 dev vmbr0.10 proto kernel scope link src 192.168.10.253

ip neigh

192.168.5.254 dev vmbr3 lladdr xx:xx:xx:xx:xx:xx DELAY
192.168.10.1 dev vmbr0.10  FAILED
192.168.5.11 dev vmbr3 lladdr xx:xx:xx:xx:xx:xx REACHABLE
192.168.6.120 dev vmbr1 lladdr xx:xx:xx:xx:xx:xx REACHABLE
xxxx::xxxx:xxxx:xxxx:xxxx dev vmbr3 lladdr xx:xx:xx:xx:xx:xx STALE

Thus, the pfSense interface is unreachable ...

 

[trigg3r]

One more thing: even if I get the management VLAN to work, how can I route the traffic of the VMs now running on the 192.16.3.0/24 subnet if the default gateway forwards the traffic on the tagged port 192.168.10.1?

Sorry, you already explained this to me ... "All other Guest traffic are not forced over the management vlan (just because this has an IP/Gateway defined). "

 

[vesalius]

The only error I see is under vmbr0 the line iface vmbr0 inet static should no longer be static but should now be manual iface vmbr0 inet manual. Not sure if that will impact vmbr0.10 though. Change that then ifrelaod -a and see if you can get a connection. Have you double-checked that your pfsense vlan10 firewalls are in line to allow 2-way traffic? The pfsense box is plugged directly into the proxmox node over eth0 correct? no switch in between the 2?

Once you get a connection over 192.168.10.0/24 working will need to check into changing your proxmox nano /etc/hosts file to reflect the new 192.168.10.253/24 primary proxmox address.

 

[trigg3r]

The pfsense box is plugged directly into the proxmox node over eth0 correct? no switch in between the 2?

You nailed it! I built the rack myself so I was sure the two devices connected directly, in fact I spent a whole day on the issue, thinking about what the problem could be but having no doubts about the wired connection between the two devices. But at this point I went to check, and... the connection goes through a switch!!

I don't remember doing it, but in the face of evidence ... I configured the VLAN in the switch ports and everything was fine.

Thank you so much!