[SOLVED] Vlan Tag?

Discussion in 'Proxmox VE: Networking and Firewall' started by killmasta93, Nov 12, 2017.

  1. killmasta93

    killmasta93 Member

    Joined:
    Aug 13, 2017
    Messages:
    415
    Likes Received:
    15
    Hi, I was wondering if someone else has accomplish this. Not sure why im struggling, I have done it before when pfSense was on bare metal but no sure whats the issue.

    Currently pfSense is virtualized inside of Proxmox 5.0. Which is connected to a netgear smart switch which i configured the tag ports 2 and 4 (p2 is proxmox and p4 is unifi ap). And the AP unifi created another guest network with vlan 50 see picture.

    I was looking around saw few users saying to connect a linux bridge or bond but not sure.

    Thank you
     

    Attached Files:

  2. killmasta93

    killmasta93 Member

    Joined:
    Aug 13, 2017
    Messages:
    415
    Likes Received:
    15
    SOLVED: i needed to check vlan aware tab on the LAN tab of the bridge for the VLAN tag to go though
     
  3. x307

    x307 New Member

    Joined:
    May 22, 2016
    Messages:
    11
    Likes Received:
    9
    Anytime vlans, bonding and bridging are involved I always use openvswitch right away:

    The following assumes you're using Proxmox version 5

    Add proxmox repo:
    Code:
    wget -q -O- 'http://download.proxmox.com/debian/pve/dists/stretch/proxmox-ve-release-5.x.gpg' | apt-key add -
    
    Add Proxmox repo:
    Code:
    echo "deb http://download.proxmox.com/debian/pve stretch pve-no-subscription" > /etc/apt/sources.list.d/pve-no-subscription.list
    
    Update apt
    Code:
    apt update
    
    Install openvswitch (it MUST be the Proxmox version, from the repo above)
    Code:
    apt install openvswitch-switch
    
    Then configure the networking for your hypervisor, something like this (if you're using VLAN 5 and a subnet like 10.1.5.0/24 for example):
    Code:
    auto lo
    iface lo inet loopback
    
    
    allow-vmbr0 bond0
    iface bond0 inet manual
        ovs_bridge vmbr0
        ovs_type OVSBond
        ovs_bonds eth0 eth1
        ovs_options bond_mode=balance-tcp lacp=active other_config:lacp-time=fast
    
    auto vmbr0
    allow-ovs vmbr0
    iface vmbr0 inet manual
        ovs_type OVSBridge
        ovs_ports bond0 vlan5 vlan99
    
    allow-vmbr0 vlan5
    iface vlan5 inet static
        ovs_type OVSIntPort
        ovs_bridge vmbr0
        ovs_options tag=5
        ovs_extra set interface ${IFACE} external-ids:iface-id=$(hostname -s)-${IFACE}-vif
        address 10.1.5.XXX
        netmask 255.255.255.0
        gateway 10.1.5.1
        mtu 1500
    
    
    allow-vmbr0 vlan99
    iface vlan99 inet static
        ovs_type OVSIntPort
        ovs_bridge vmbr0
        ovs_options tag=99
        ovs_extra set interface ${IFACE} external-ids:iface-id=$(hostname -s)-${IFACE}-vif
        address 10.1.99.XXX
        netmask 255.255.255.0
        mtu 9000
    
    Then reboot!
     
    #3 x307, Nov 13, 2017
    Last edited: Nov 13, 2017
    killmasta93 likes this.
  4. killmasta93

    killmasta93 Member

    Joined:
    Aug 13, 2017
    Messages:
    415
    Likes Received:
    15
    Thanks for the reply and great guide, for the openvswitch isnt that mostly for Vms to do VLANS within the proxmox? Something like Hyper V using the Virtual Switch? and would i be able to do this on the webgui? or is this in /etc/network/interfaces?
     
  5. x307

    x307 New Member

    Joined:
    May 22, 2016
    Messages:
    11
    Likes Received:
    9
    Sure, no problem :)

    Openvswitch is a very powerful upgrade to standard Linux networking. It provides a ton of features, options and enhancements.

    The reason I started using Openvswitch was due to a weird situation where a VM guest would share a VLAN with the hypervisor itself. In my prior situation, once the VM guest came online it would steal the VLAN from the hypervisor everytime. Once I started using Openvswitch that problem went away.

    Personally, I don't use most of the features of Openvswitch but it does provide cleaner logic, more closely matching the underlying network concepts (and switch-level configuration commands), IMO, so it's easier for me to read and troubleshoot.

    Openswitch isn't just for internal Proxmox networking; it's an important enhancement to Linux networking itself. It sort of works in parallel with standard Linux networking and as you can see above, you need to add some special syntax to enable Openvswitch on an interface or connection (things like "allow-ovs vmbr0", "ovs_type OVSBridge", "ovs_ports bond0 vlan5", etc).

    I suggest everyone spend a little time with it in the lab to understand it's syntax and features, it's well worth the time.
     
    #5 x307, Nov 13, 2017
    Last edited: Nov 27, 2017
    Alexander Pytlev and killmasta93 like this.
  6. killmasta93

    killmasta93 Member

    Joined:
    Aug 13, 2017
    Messages:
    415
    Likes Received:
    15
    wow thanks for the reply, i did not know that as currently still running proxmox test lab for the past few months. when you say
    do you mean when you reboot proxmox and the vm would start up for a moment proxmox IP would die then come back one line?

    and can the openVswitch be configured thought the WebGui on the network page?
     
  7. x307

    x307 New Member

    Joined:
    May 22, 2016
    Messages:
    11
    Likes Received:
    9
    Personally I configure Openvswitch in /etc/network/interfaces but you can also do that in the GUI (see attached sample from my lab)

    ovs2.jpg ovs.jpg
     
    killmasta93 likes this.
  8. killmasta93

    killmasta93 Member

    Joined:
    Aug 13, 2017
    Messages:
    415
    Likes Received:
    15
    Thanks for the reply, out of curiosity, looking at your picture, so you have 2 nics physical on your proxmox setup. You then bonded both nics using Vmbr0 and then you have proxmox the VLAN 5 and all your other Vms get the vlan 99? and what about the eno1-4 those are physical nics?
     
  9. x307

    x307 New Member

    Joined:
    May 22, 2016
    Messages:
    11
    Likes Received:
    9
    Regarding the VM stealing the VLAN, that was a situation in Proxmox 4. I had a VM which needed to be on the same VLAN as the hypervisor itself. Logically this should be fine, but if you use standard Linux networking, it's not..

    I would setup the VM to avoid starting on start up. Everything worked fine until I started the VM. Once I started the VM, the VLAN would get moved away for use by the VM itself. I could connect to the VM fine, but not the hypervisor (Proxmox)
     
    killmasta93 likes this.
  10. killmasta93

    killmasta93 Member

    Joined:
    Aug 13, 2017
    Messages:
    415
    Likes Received:
    15
    wow very interesting first time i heard about this, in the future will use your guide for the VLANS for internal VLAN networking

    Thank you again
     
    x307 likes this.
  11. x307

    x307 New Member

    Joined:
    May 22, 2016
    Messages:
    11
    Likes Received:
    9
    My hypervisor is on VLAN 5 and has an IP on VLAN 99 as well. The NICs "enp65s0f0" and "enp65s0f1" are 10G NICs used for bond0. The bond0 interface carriers all the traffic. The other Onboard NICs "eno1", "eno2", "eno3" and "eno4" are not being used in this example.

    In this example I am allowing Proxmox have IP addresses on both vlan5 and vlan99, so vmbr0 bridges over both of them.

    The really nice thing about doing it this way is that you can give ALL of your VM guests "vmbr0" as their network device and then specify a VLAN in each of their network interfaces (in the GUI).

    I updated the /etc/network/interfaces config above to more closely match the screenshots, to avoid any confusion there.
     
  12. paraguy

    paraguy New Member

    Joined:
    Jul 5, 2018
    Messages:
    3
    Likes Received:
    1
    I have tried for hours to follow the example as seen above but I am getting no vlan traffic on my vm. The only way I can get out from the vm is to take away vlan 100 and put it on the default vlan.

    auto lo
    iface lo inet loopback

    auto eno1
    iface eno1 inet manual

    auto eno2
    iface eno2 inet manual

    auto eno3
    iface eno3 inet manual

    auto eno4
    iface eno4 inet manual

    allow-vmbr0 bond0
    iface bond0 inet manual
    ovs_bonds eno1 eno2 eno3 eno4
    ovs_type OVSBond
    ovs_bridge vmbr0
    ovs_options bond_mode=balance-tcp lacp=active

    auto vmbr0
    iface vmbr0 inet static
    address 10.44.1.5
    netmask 255.255.255.0
    gateway 10.44.1.1
    ovs_type OVSBridge
    ovs_ports bond0 lab100

    allow-vmbr0 lab100
    iface lab100 inet static
    address 10.44.100.2
    netmask 255.255.255.0
    ovs_type OVSIntPort
    ovs_bridge vmbr0
     
  13. killmasta93

    killmasta93 Member

    Joined:
    Aug 13, 2017
    Messages:
    415
    Likes Received:
    15
    dont complicate, on proxmox on the vmbr0 check that box for the VLAN tag, if your running a dumb switch it will go though easily and on pfSense create the Vlans, if your running smart switch you need to tag the port on which your proxmox LAN is connected to the smart switch
     
  14. kcallis

    kcallis New Member

    Joined:
    Apr 5, 2018
    Messages:
    21
    Likes Received:
    0
    I am glad that I ran into the thread, because I have been running to issue a week or so and not getting anywhere. I have Proxmox running on a laptop with a single interface. I run externally pfSense and have several VLANs defined, and I thought I would place the Proxmox box on my VLAN05_MGMT (192.168.5.0/24) segment. I figured that setting my interface on the PM on my MGMT VLAN would be simple and secure.

    I immediately started to create KVM images and lxc containers and gave static IP address to other VLANs. Needless to say, that did not go well for me. Of course, I can get to the console, but I was not able to connect to the network. My PM interface is connected to my cisco switch and I have the port untagged for my VLAN05_MGMT, and tagged for VLAN10, VLAN20, VLAN25, and VLAN30, which are defined on my pfSense box. So looking at the examples I saw earlier, I need to install openvswitch, define my VLANs and life will be groovy once again, correct?

    If that is the track I am suppose to, do I need to define my VLAN05_MGMT as well, since my interface is on the VLAN05_MGMT segment? Even though on the switch, the port is untagged?
     
  15. killmasta93

    killmasta93 Member

    Joined:
    Aug 13, 2017
    Messages:
    415
    Likes Received:
    15
    So let me get this correct you have proxmox on a laptop, and pfSense on another box and a cisco swich. You want you proxmox to get the VLAN 5 from which ur pfSense manages the DHCP. If this is correct on the cisco swiche you need to create that VLAN 5 with TAG the port on proxmox connected
     
  16. kcallis

    kcallis New Member

    Joined:
    Apr 5, 2018
    Messages:
    21
    Likes Received:
    0
    I have a static IP on the PM host and it works with no issues. My problem is with my images and container that I am have an issue with. When I create a KVM image (for instance Ubuntu Server 18.04), I give it a static IP that is on my VLAN10_CLRNET (192.168.10.0/24) segment. I incorrectly assumed that the hypervisor would automatically know what to do with my VLANS and life would be perfect. Needless to say, that when I went to the console of the Ubuntu image, it did not work.

    As I started reading more about Proxmox network, I through that I needed to define VLANs:

    Code:
    /etc/network/interfaces
    
    auto lo
    iface lo inet loopback
    iface enp0s25 inet manual
    iface enp0s25.10 inet manual
    iface enp0s25.20 inet manual
    iface enp0s25.25 inet manual
    iface enp0s25.30 inet manual
    
    
    #auto vmbr0v5
    #iface vmbr0v5 inet static
    #       address 192.168.5.250
    #       netmask 255.255.255.0
    #       gateway 192.168.5.1
    #       bridge_ports enp0s25.5
    #       bridge_stp off
    #       bridge_fd 0
    
    auto vmbr0v10
    iface vmbr0v5 inet static
           address 192.168.10.250
           netmask 255.255.255.0
           address 192.168.10.250
           netmask 255.255.255.0
           gateway 192.168.10.1
           bridge_ports enp0s25.10
           bridge_stp off
           bridge_fd 0
    
    auto vmbr0v20
    iface vmbr0v20 inet static
           address 192.168.20.250
           netmask 255.255.255.0
           gateway 192.168.20.1
           bridge_ports enp0s25.20
           bridge_stp off
           bridge_fd 0
    
    auto vmbr0v25
    iface vmbr0v25 inet static
           address 192.168.25.250
           netmask 255.255.255.0
           gateway 192.168.25.1
           bridge_ports enp0s25.25
           bridge_stp off
           bridge_fd 0
    
    
    auto vmbr0
    iface vmbr0 inet static
            address 192.168.5.250
            netmask 255.255.255.0
            gateway 192.168.5.1
            bridge_ports enp0s25
            bridge_stp off
            bridge_fd 0
    
    As this point I am out of my depth and regardless, I still was not able to get my images and containers to connect to the network. So reading "Mastering Proxmox", I ran across openvswitch. That is when I ran across this thread! I would like to make use of ovenvswitch, but at the same time I would just like to just be able to get my images connected. Incidentally, I am just using the standard default /etc/network/interfaces.
     
  17. killmasta93

    killmasta93 Member

    Joined:
    Aug 13, 2017
    Messages:
    415
    Likes Received:
    15
    ooOooo so your using containers and not vms?
     
  18. kcallis

    kcallis New Member

    Joined:
    Apr 5, 2018
    Messages:
    21
    Likes Received:
    0
    Right now, I am doing VMs (for instance, I am trying to get Atomic Toolkit under Ubuntu Server 18.04), but as some point, I would rather get the lxc containers in play since from a density standpoint, considering that I am running a HP ProBook 6570b with 12GB RAM and 2TB hd. At some point I would even like to wrangle docker into the equation, hence using ovs.
     
  19. kcallis

    kcallis New Member

    Joined:
    Apr 5, 2018
    Messages:
    21
    Likes Received:
    0
    I have created an /etc/network/interfaces:

    Code:
    /etc/network/interfaces.ovs
    
    # Loopback interface
    auto lo
    iface lo inet loopback
    
    # Bridge for our enp0s25 physical interfaces and vlan virtual interfaces (our VMs will
    # also attach to this bridge)
    auto vmbr0
    allow-ovs vmbr0
    iface vmbr0 inet manual
      ovs_type OVSBridge
      # NOTE: we MUST mention enp0s25, vlan1, and vlan55 even though each
      #       of them lists ovs_bridge vmbr0!  Not sure why it needs this
      #       kind of cross-referencing but it won't work without it!
      ovs_ports enp0s25 vlan5 vlan10 vlan20 vlan25 vlan30
      mtu 9000
    
    # Physical interface for traffic coming into the system.  Retag untagged
    # traffic into vlan 5, but pass through other tags.
    auto enp0s25
    allow-vmbr0 enp0s25
    iface enp0s25 inet manual
      ovs_bridge vmbr0
      ovs_type OVSPort
      ovs_options tag=5 vlan_mode=native-untagged trunks=10.20,25,30
      mtu 9000
    
    # Virtual interface to take advantage of originally untagged traffic
    allow-vmbr0 vlan5
    iface vlan5 inet static
      ovs_type OVSIntPort
      ovs_bridge vmbr0
      ovs_options tag=5
      ovs_extra set interface ${IFACE} external-ids:iface-id=$(hostname -s)-${IFACE}-vif
      address 192.168.5.250
      netmask 255.255.255.0
      gateway 192.168.5.1
      mtu 1500
    
    allow-vmbr0 vlan10
    iface vlan5 inet static
       ovs_type OVSIntPort
       ovs_bridge vmbr0
       ovs_options tag=10
       ovs_extra set interface ${IFACE} external-ids:iface-id=$(hostname -s)-${IFACE}-vif
       address 192.168.10.250
       netmask 255.255.255.0
       gateway 192.168.10.1
       mtu 1500
    
    allow-vmbr0 vlan20
    iface vlan20 inet static
      ovs_type OVSIntPort
      ovs_bridge vmbr0
      ovs_options tag=20
      ovs_extra set interface ${IFACE} external-ids:iface-id=$(hostname -s)-${IFACE}-vif
      address 192.168.20.250
      netmask 255.255.255.0
      gateway 192.168.20.1
      mtu 1500
    
    allow-vmbr0 vlan25
    iface vlan25 inet static
      ovs_type OVSIntPort
      ovs_bridge vmbr0
      ovs_options tag=25
      ovs_extra set interface ${IFACE} external-ids:iface-id=$(hostname -s)-${IFACE}-vif
      address 192.168.25.250
      netmask 255.255.255.0
      gateway 192.168.25.1
      mtu 1500
    
    allow-vmbr0 vlan30
    iface vlan30 inet static
      ovs_type OVSIntPort
      ovs_bridge vmbr0
      ovs_options tag=30
      ovs_extra set interface ${IFACE} external-ids:iface-id=$(hostname -s)-${IFACE}-vif
      address 192.168.30.250
      netmask 255.255.255.0
      gateway 192.168.30.1
      mtu 1500
    
    I am using the example show on the wiki for openvswitch that I changed for my layout. Again, the management interface is set to 192.168.5.250 (which resides on my MGMT VLAN VLAN05). The interface on my PM box is connected to my Cisco SG300 switch and the VLAN membership is that VLAN05 is untagged, and VLAN10, VLAN20, VLAN25, and VLAN30 are tagged.

    I would like to create a Ubuntu Server 18.04 vm that will connect to the VLAN20. At some point, I would like to add containers (both LXC and Docker images) in the future, but for now, I just want the Ubuntu server to play nicely. So am I on the correct path, because I am now two weeks with a Ubuntu Server that is using resources, but not functional since it can't get to the network or internet.
     
  20. kcallis

    kcallis New Member

    Joined:
    Apr 5, 2018
    Messages:
    21
    Likes Received:
    0
    I forgot to set the VLAN tag id when I created the VM image. I would still like to move towards ovs, but for now I am a happy camper.
     
  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.
    Dismiss Notice