[SOLVED] Vlan Tag?

killmasta93 · Nov 12, 2017

Hi, I was wondering if someone else has accomplish this. Not sure why im struggling, I have done it before when pfSense was on bare metal but no sure whats the issue.

Currently pfSense is virtualized inside of Proxmox 5.0. Which is connected to a netgear smart switch which i configured the tag ports 2 and 4 (p2 is proxmox and p4 is unifi ap). And the AP unifi created another guest network with vlan 50 see picture.

I was looking around saw few users saying to connect a linux bridge or bond but not sure.

Thank you

killmasta93 · Nov 13, 2017

SOLVED: i needed to check vlan aware tab on the LAN tab of the bridge for the VLAN tag to go though

superawesomeusername · Nov 13, 2017

Anytime vlans, bonding and bridging are involved I always use openvswitch right away:

The following assumes you're using Proxmox version 5

Add proxmox repo:

Code:

wget -q -O- 'http://download.proxmox.com/debian/pve/dists/stretch/proxmox-ve-release-5.x.gpg' | apt-key add -

Add Proxmox repo:

Code:

echo "deb http://download.proxmox.com/debian/pve stretch pve-no-subscription" > /etc/apt/sources.list.d/pve-no-subscription.list

Update apt

Code:

apt update

Install openvswitch (it MUST be the Proxmox version, from the repo above)

Code:

apt install openvswitch-switch

Then configure the networking for your hypervisor, something like this (if you're using VLAN 5 and a subnet like 10.1.5.0/24 for example):

Code:

auto lo
iface lo inet loopback


allow-vmbr0 bond0
iface bond0 inet manual
    ovs_bridge vmbr0
    ovs_type OVSBond
    ovs_bonds eth0 eth1
    ovs_options bond_mode=balance-tcp lacp=active other_config:lacp-time=fast

auto vmbr0
allow-ovs vmbr0
iface vmbr0 inet manual
    ovs_type OVSBridge
    ovs_ports bond0 vlan5 vlan99

allow-vmbr0 vlan5
iface vlan5 inet static
    ovs_type OVSIntPort
    ovs_bridge vmbr0
    ovs_options tag=5
    ovs_extra set interface ${IFACE} external-ids:iface-id=$(hostname -s)-${IFACE}-vif
    address 10.1.5.XXX
    netmask 255.255.255.0
    gateway 10.1.5.1
    mtu 1500


allow-vmbr0 vlan99
iface vlan99 inet static
    ovs_type OVSIntPort
    ovs_bridge vmbr0
    ovs_options tag=99
    ovs_extra set interface ${IFACE} external-ids:iface-id=$(hostname -s)-${IFACE}-vif
    address 10.1.99.XXX
    netmask 255.255.255.0
    mtu 9000

Then reboot!

killmasta93 · Nov 13, 2017

Thanks for the reply and great guide, for the openvswitch isnt that mostly for Vms to do VLANS within the proxmox? Something like Hyper V using the Virtual Switch? and would i be able to do this on the webgui? or is this in /etc/network/interfaces?

superawesomeusername · Nov 13, 2017

Sure, no problem

Openvswitch is a very powerful upgrade to standard Linux networking. It provides a ton of features, options and enhancements.

The reason I started using Openvswitch was due to a weird situation where a VM guest would share a VLAN with the hypervisor itself. In my prior situation, once the VM guest came online it would steal the VLAN from the hypervisor everytime. Once I started using Openvswitch that problem went away.

Personally, I don't use most of the features of Openvswitch but it does provide cleaner logic, more closely matching the underlying network concepts (and switch-level configuration commands), IMO, so it's easier for me to read and troubleshoot.

Openswitch isn't just for internal Proxmox networking; it's an important enhancement to Linux networking itself. It sort of works in parallel with standard Linux networking and as you can see above, you need to add some special syntax to enable Openvswitch on an interface or connection (things like "allow-ovs vmbr0", "ovs_type OVSBridge", "ovs_ports bond0 vlan5", etc).

I suggest everyone spend a little time with it in the lab to understand it's syntax and features, it's well worth the time.

killmasta93 · Nov 13, 2017

wow thanks for the reply, i did not know that as currently still running proxmox test lab for the past few months. when you say

VM guest came online it would steal the VLAN from the hypervisor everytime.

do you mean when you reboot proxmox and the vm would start up for a moment proxmox IP would die then come back one line?

and can the openVswitch be configured thought the WebGui on the network page?

superawesomeusername · Nov 13, 2017

Personally I configure Openvswitch in /etc/network/interfaces but you can also do that in the GUI (see attached sample from my lab)

killmasta93 · Nov 13, 2017

Thanks for the reply, out of curiosity, looking at your picture, so you have 2 nics physical on your proxmox setup. You then bonded both nics using Vmbr0 and then you have proxmox the VLAN 5 and all your other Vms get the vlan 99? and what about the eno1-4 those are physical nics?

superawesomeusername · Nov 13, 2017

Regarding the VM stealing the VLAN, that was a situation in Proxmox 4. I had a VM which needed to be on the same VLAN as the hypervisor itself. Logically this should be fine, but if you use standard Linux networking, it's not..

I would setup the VM to avoid starting on start up. Everything worked fine until I started the VM. Once I started the VM, the VLAN would get moved away for use by the VM itself. I could connect to the VM fine, but not the hypervisor (Proxmox)

killmasta93 · Nov 13, 2017

wow very interesting first time i heard about this, in the future will use your guide for the VLANS for internal VLAN networking

Thank you again

superawesomeusername · Nov 13, 2017

killmasta93 said:
Thanks for the reply, out of curiosity, looking at your picture, so you have 2 nics physical on your proxmox setup. You then bonded both nics using Vmbr0 and then you have proxmox the VLAN 5 and all your other Vms get the vlan 99? and what about the eno1-4 those are physical nics?

My hypervisor is on VLAN 5 and has an IP on VLAN 99 as well. The NICs "enp65s0f0" and "enp65s0f1" are 10G NICs used for bond0. The bond0 interface carriers all the traffic. The other Onboard NICs "eno1", "eno2", "eno3" and "eno4" are not being used in this example.

In this example I am allowing Proxmox have IP addresses on both vlan5 and vlan99, so vmbr0 bridges over both of them.

The really nice thing about doing it this way is that you can give ALL of your VM guests "vmbr0" as their network device and then specify a VLAN in each of their network interfaces (in the GUI).

I updated the /etc/network/interfaces config above to more closely match the screenshots, to avoid any confusion there.

paraguy · Jul 6, 2018

I have tried for hours to follow the example as seen above but I am getting no vlan traffic on my vm. The only way I can get out from the vm is to take away vlan 100 and put it on the default vlan.

auto lo
iface lo inet loopback

auto eno1
iface eno1 inet manual

auto eno2
iface eno2 inet manual

auto eno3
iface eno3 inet manual

auto eno4
iface eno4 inet manual

allow-vmbr0 bond0
iface bond0 inet manual
ovs_bonds eno1 eno2 eno3 eno4
ovs_type OVSBond
ovs_bridge vmbr0
ovs_options bond_mode=balance-tcp lacp=active

auto vmbr0
iface vmbr0 inet static
address 10.44.1.5
netmask 255.255.255.0
gateway 10.44.1.1
ovs_type OVSBridge
ovs_ports bond0 lab100

allow-vmbr0 lab100
iface lab100 inet static
address 10.44.100.2
netmask 255.255.255.0
ovs_type OVSIntPort
ovs_bridge vmbr0

killmasta93 · Jul 6, 2018

dont complicate, on proxmox on the vmbr0 check that box for the VLAN tag, if your running a dumb switch it will go though easily and on pfSense create the Vlans, if your running smart switch you need to tag the port on which your proxmox LAN is connected to the smart switch

kcallis · Jul 9, 2018

I am glad that I ran into the thread, because I have been running to issue a week or so and not getting anywhere. I have Proxmox running on a laptop with a single interface. I run externally pfSense and have several VLANs defined, and I thought I would place the Proxmox box on my VLAN05_MGMT (192.168.5.0/24) segment. I figured that setting my interface on the PM on my MGMT VLAN would be simple and secure.

I immediately started to create KVM images and lxc containers and gave static IP address to other VLANs. Needless to say, that did not go well for me. Of course, I can get to the console, but I was not able to connect to the network. My PM interface is connected to my cisco switch and I have the port untagged for my VLAN05_MGMT, and tagged for VLAN10, VLAN20, VLAN25, and VLAN30, which are defined on my pfSense box. So looking at the examples I saw earlier, I need to install openvswitch, define my VLANs and life will be groovy once again, correct?

If that is the track I am suppose to, do I need to define my VLAN05_MGMT as well, since my interface is on the VLAN05_MGMT segment? Even though on the switch, the port is untagged?

killmasta93 · Jul 9, 2018

kcallis said:
I am glad that I ran into the thread, because I have been running to issue a week or so and not getting anywhere. I have Proxmox running on a laptop with a single interface. I run externally pfSense and have several VLANs defined, and I thought I would place the Proxmox box on my VLAN05_MGMT (192.168.5.0/24) segment. I figured that setting my interface on the PM on my MGMT VLAN would be simple and secure.

I immediately started to create KVM images and lxc containers and gave static IP address to other VLANs. Needless to say, that did not go well for me. Of course, I can get to the console, but I was not able to connect to the network. My PM interface is connected to my cisco switch and I have the port untagged for my VLAN05_MGMT, and tagged for VLAN10, VLAN20, VLAN25, and VLAN30, which are defined on my pfSense box. So looking at the examples I saw earlier, I need to install openvswitch, define my VLANs and life will be groovy once again, correct?

If that is the track I am suppose to, do I need to define my VLAN05_MGMT as well, since my interface is on the VLAN05_MGMT segment? Even though on the switch, the port is untagged?

So let me get this correct you have proxmox on a laptop, and pfSense on another box and a cisco swich. You want you proxmox to get the VLAN 5 from which ur pfSense manages the DHCP. If this is correct on the cisco swiche you need to create that VLAN 5 with TAG the port on proxmox connected

kcallis · Jul 9, 2018

I have a static IP on the PM host and it works with no issues. My problem is with my images and container that I am have an issue with. When I create a KVM image (for instance Ubuntu Server 18.04), I give it a static IP that is on my VLAN10_CLRNET (192.168.10.0/24) segment. I incorrectly assumed that the hypervisor would automatically know what to do with my VLANS and life would be perfect. Needless to say, that when I went to the console of the Ubuntu image, it did not work.

As I started reading more about Proxmox network, I through that I needed to define VLANs:

Code:

/etc/network/interfaces

auto lo
iface lo inet loopback
iface enp0s25 inet manual
iface enp0s25.10 inet manual
iface enp0s25.20 inet manual
iface enp0s25.25 inet manual
iface enp0s25.30 inet manual


#auto vmbr0v5
#iface vmbr0v5 inet static
#       address 192.168.5.250
#       netmask 255.255.255.0
#       gateway 192.168.5.1
#       bridge_ports enp0s25.5
#       bridge_stp off
#       bridge_fd 0

auto vmbr0v10
iface vmbr0v5 inet static
       address 192.168.10.250
       netmask 255.255.255.0
       address 192.168.10.250
       netmask 255.255.255.0
       gateway 192.168.10.1
       bridge_ports enp0s25.10
       bridge_stp off
       bridge_fd 0

auto vmbr0v20
iface vmbr0v20 inet static
       address 192.168.20.250
       netmask 255.255.255.0
       gateway 192.168.20.1
       bridge_ports enp0s25.20
       bridge_stp off
       bridge_fd 0

auto vmbr0v25
iface vmbr0v25 inet static
       address 192.168.25.250
       netmask 255.255.255.0
       gateway 192.168.25.1
       bridge_ports enp0s25.25
       bridge_stp off
       bridge_fd 0


auto vmbr0
iface vmbr0 inet static
        address 192.168.5.250
        netmask 255.255.255.0
        gateway 192.168.5.1
        bridge_ports enp0s25
        bridge_stp off
        bridge_fd 0

As this point I am out of my depth and regardless, I still was not able to get my images and containers to connect to the network. So reading "Mastering Proxmox", I ran across openvswitch. That is when I ran across this thread! I would like to make use of ovenvswitch, but at the same time I would just like to just be able to get my images connected. Incidentally, I am just using the standard default /etc/network/interfaces.

killmasta93 · Jul 10, 2018

ooOooo so your using containers and not vms?

kcallis · Jul 10, 2018

Right now, I am doing VMs (for instance, I am trying to get Atomic Toolkit under Ubuntu Server 18.04), but as some point, I would rather get the lxc containers in play since from a density standpoint, considering that I am running a HP ProBook 6570b with 12GB RAM and 2TB hd. At some point I would even like to wrangle docker into the equation, hence using ovs.

kcallis · Jul 11, 2018

I have created an /etc/network/interfaces:

Code:

/etc/network/interfaces.ovs

# Loopback interface
auto lo
iface lo inet loopback

# Bridge for our enp0s25 physical interfaces and vlan virtual interfaces (our VMs will
# also attach to this bridge)
auto vmbr0
allow-ovs vmbr0
iface vmbr0 inet manual
  ovs_type OVSBridge
  # NOTE: we MUST mention enp0s25, vlan1, and vlan55 even though each
  #       of them lists ovs_bridge vmbr0!  Not sure why it needs this
  #       kind of cross-referencing but it won't work without it!
  ovs_ports enp0s25 vlan5 vlan10 vlan20 vlan25 vlan30
  mtu 9000

# Physical interface for traffic coming into the system.  Retag untagged
# traffic into vlan 5, but pass through other tags.
auto enp0s25
allow-vmbr0 enp0s25
iface enp0s25 inet manual
  ovs_bridge vmbr0
  ovs_type OVSPort
  ovs_options tag=5 vlan_mode=native-untagged trunks=10.20,25,30
  mtu 9000

# Virtual interface to take advantage of originally untagged traffic
allow-vmbr0 vlan5
iface vlan5 inet static
  ovs_type OVSIntPort
  ovs_bridge vmbr0
  ovs_options tag=5
  ovs_extra set interface ${IFACE} external-ids:iface-id=$(hostname -s)-${IFACE}-vif
  address 192.168.5.250
  netmask 255.255.255.0
  gateway 192.168.5.1
  mtu 1500

allow-vmbr0 vlan10
iface vlan5 inet static
   ovs_type OVSIntPort
   ovs_bridge vmbr0
   ovs_options tag=10
   ovs_extra set interface ${IFACE} external-ids:iface-id=$(hostname -s)-${IFACE}-vif
   address 192.168.10.250
   netmask 255.255.255.0
   gateway 192.168.10.1
   mtu 1500

allow-vmbr0 vlan20
iface vlan20 inet static
  ovs_type OVSIntPort
  ovs_bridge vmbr0
  ovs_options tag=20
  ovs_extra set interface ${IFACE} external-ids:iface-id=$(hostname -s)-${IFACE}-vif
  address 192.168.20.250
  netmask 255.255.255.0
  gateway 192.168.20.1
  mtu 1500

allow-vmbr0 vlan25
iface vlan25 inet static
  ovs_type OVSIntPort
  ovs_bridge vmbr0
  ovs_options tag=25
  ovs_extra set interface ${IFACE} external-ids:iface-id=$(hostname -s)-${IFACE}-vif
  address 192.168.25.250
  netmask 255.255.255.0
  gateway 192.168.25.1
  mtu 1500

allow-vmbr0 vlan30
iface vlan30 inet static
  ovs_type OVSIntPort
  ovs_bridge vmbr0
  ovs_options tag=30
  ovs_extra set interface ${IFACE} external-ids:iface-id=$(hostname -s)-${IFACE}-vif
  address 192.168.30.250
  netmask 255.255.255.0
  gateway 192.168.30.1
  mtu 1500

I am using the example show on the wiki for openvswitch that I changed for my layout. Again, the management interface is set to 192.168.5.250 (which resides on my MGMT VLAN VLAN05). The interface on my PM box is connected to my Cisco SG300 switch and the VLAN membership is that VLAN05 is untagged, and VLAN10, VLAN20, VLAN25, and VLAN30 are tagged.

I would like to create a Ubuntu Server 18.04 vm that will connect to the VLAN20. At some point, I would like to add containers (both LXC and Docker images) in the future, but for now, I just want the Ubuntu server to play nicely. So am I on the correct path, because I am now two weeks with a Ubuntu Server that is using resources, but not functional since it can't get to the network or internet.

kcallis · Jul 11, 2018

I forgot to set the VLAN tag id when I created the VM image. I would still like to move towards ovs, but for now I am a happy camper.

[SOLVED] Vlan Tag?

Renowned Member

Attachments

Renowned Member

Member

Renowned Member

Member

Renowned Member

Member

Renowned Member

Member

Renowned Member

Member

Active Member

Renowned Member

Active Member

Renowned Member

Active Member

Renowned Member

Active Member

Active Member

Active Member