Understanding VLANs in Proxmox

[afhelton]

Hi All,

New to Proxmox here. I've read through the Network Configuration doc and have even successfully implemented Proxmox VE management IP with VLAN aware Linux bridge as per the docs. I'm simply trying to understand how this works 'behind the scenes'

auto lo
iface lo inet loopback

iface eno1 inet manual


auto vmbr0.5
iface vmbr0.5 inet static
        address  10.10.10.2
        netmask  255.255.255.0
        gateway  10.10.10.1

auto vmbr0
iface vmbr0 inet manual
        bridge_ports eno1
        bridge_stp off
        bridge_fd 0
        bridge_vlan_aware yes

In the example above, is the vmbr0.5 simply being mapped to the vmbr0 bridge interface based on the 0.X format? Is this effectively creating a 'sub-interface' on the virtual bridge to be use by VLAN 5 (in this case)? What exactly is happening here?

Also, how does the Proxmox host know that the IP address and DG entered in the vmbr0.5 configuration is intended for it? If I were create another Linux VLAN, say vmbr0.10 for example, and entered an IP address and DG what would happen?

 

[spirit]

In the example above, is the vmbr0.5 simply being mapped to the vmbr0 bridge interface based on the 0.X format? Is this effectively creating a 'sub-interface' on the virtual bridge to be use by VLAN 5 (in this case)? What exactly is happening here?

vlan tag, on vlan-aware bridge, create a virtual interface in the bridge. (like a vlan admin interface in a physical swich)
Note that this is only for admin vlans interface of your proxmox host, not vms vlans. for vm vlans, you need to defined vlan on the vm nic.

you can verify vlan with command "bridge -c vlan show", it'll display all vlans for each "virtual" ports of the bridge

Also, how does the Proxmox host know that the IP address and DG entered in the vmbr0.5 configuration is intended for it? If I were create another Linux VLAN, say vmbr0.10 for example, and entered an IP address and DG what would happen?

you need to defined differents subnet on each vlans, then you need to declare theses subnet on differents services.
(Like corosync links for example, ceph.conf, datacenter.cfg for live migration links, ....). By default, all services are main hostnames dns resolution ip.

 

[afhelton]

vlan tag, on vlan-aware bridge, create a virtual interface in the bridge. (like a vlan admin interface in a physical swich)
Note that this is only for admin vlans interface of your proxmox host, not vms vlans. for vm vlans, you need to defined vlan on the vm nic.

Ok. This is what I thought was happening. So basically I can create other virtual interfaces and 'map' them to the vlan-aware bridge. Then I can assign those virtual interfaces to my vms under the vms hardware config (or doing vm creation) and assign correct vlan tag. Is that correct?

 

[afhelton]

Ah. After messing with it a bit more I think I understand your response better. When I create a Linux VLAN, I am effectively creating a virtual vlan interface like I would on, say, a Cisco Catalyst switch. I can create as many of these as I want and assign certain services on the Proxmox host to use these VLANs.

However, for my actual VMs themselves, I go into the VM's network configuration an tell it to use a particular VLAN. I just need to make sure that the vmbr0 bridge is set to be vlan-aware. At least I hope I'm understanding this correctly!

 

[spirit]

Ah. After messing with it a bit more I think I understand your response better. When I create a Linux VLAN, I am effectively creating a virtual vlan interface like I would on, say, a Cisco Catalyst switch. I can create as many of these as I want and assign certain services on the Proxmox host to use these VLANs.

Yes, when you configure a linux vlan interface with ip address, it's like a vlan admin interface in cisco.

However, for my actual VMs themselves, I go into the VM's network configuration an tell it to use a particular VLAN. I just need to make sure that the vmbr0 bridge is set to be vlan-aware. At least I hope I'm understanding this correctly!

When you configure a vlan on the vm, proxmox will configure a virtual port on the bridge with this vlan tag. (Like an access port on a cisco switch).

 

[afhelton]

Got this setup yesterday. Everything is working great.

 

[zhoid]

Got this setup yesterday. Everything is working great.

Hey Man do you mind sharing your network configuration?

Thanks

 

[afhelton]

Hey Man do you mind sharing your network configuration?

Thanks

Sure thing.

Right now I've got an HP Proliant DL360P running Proxmox. I installed a PCIe adapter card that supports a single SSD and am using it as my boot drive / Proxmox drive. My server is hardwired (naturally) into a Cisco Catalyst 2960G switch and that in turn runs into a Cisco 2811 router which ultimately connects to my 'real' network. I've got my Cisco 2811 set up to NAT so that there's some segmentation from my 'real' network and to allow for my Proxmox VMs to connect to the Internet. I also had to setup static addressing on my actual production router so that it knows how to route stuff to my lab network if need be.

As far as Proxmox goes, I have a very simple set up as I am just learning...and honestly, I like to keep things straight forward anyways. On my Proxmox install I just have a single bridge network adapter vmbr0 (the default one). I created a virtual vlan interface and mapped it to the bridge interface (you do this using the Linux VLAN option). This virtual vlan interface acts as my management vlan for Proxmox itself. It has the default gateway and ip address configured on it. The bridge interface (vmbr0) has no actual ip configuration.

I'm teaching myself Windows Server as I need to learn it for my business, so I've got a couple of domain controllers setup and replicating. They are setup on their own vlan and subnet. I then have another vlan / subnet set up for virtual workstations (windows 10 clients in this case). In order for everything to communicate properly, I set the port on my Cisco switch that connects to the HP Proliant server as a trunk port, and then configured the appropriate sub-interfaces on the Cisco 2811 router to support routing between the vlans.

And that's honestly about it at this point. I do plan to figure out the whole nic-teaming thing at some point and then maybe learn about live migrations and such.

Oh, and one other thing I've found super helpful. I ultimately installed a single Windows server, configured a base setup (Windows updates and a few other settings), and then sysprepped it. I then converted it to a template which allows me to provision linked clones. This allows for super rapid deployment which I find particularly useful in a lab environment. I just give my template VM a little extra storage space to allow for the fact that the linked clones share storage with it (at least that's how I understand it). This is so much faster than doing a full clone. You can do the same thing for workstations as well. I've got a Windows environment, but I am sure the same is true for a Linux environment as well!