RESOLVED: Can't get any VMs to communicate on VLANs

S

ScottZ.

Well-Known Member
Nov 5, 2018
41
2
48
RESOLVED: See last thread as the issue ended up being a switch and firewall config issue, not Proxmox VE. My fault.
------ Original thread below ------
I have kept the trial small.

  1. I installed a new VM on the default bridge (vmbr0) and tagged it for vlan 201 (and bridge was vlan aware)
  2. I was NOT able to ping the firewall ip on VLAN 201
  3. I checked the firewall traffic and nothing with vlan tag was making to the firewall, which means the host/proxmox ve was blocking the traffic
  4. I manually created eno1v201, eno2v201 and attached to a new bridge vmbr0201 (with vlan aware enabled and original vmbr0 UNCHECKed for vlan aware)
  5. rebooted, verified configuration, same exact result...no traffic getting past the host.
Any advice would be recommended. I don't personally want to manually configure /etc/network/interfaces (or interfaces.new) but I will if that's my only option (as I did for step 4 since you can't create new interfaces through the web interfaces). I DO NOT want access to the Promox Management on vlan 201. Attached is my current /etc/network/interfaces config
Code: 
cat /etc/network/interfaces
# network interface settings; autogenerated
# Please do NOT modify this file directly, unless you know what
# you're doing.
#
# If you want to manage part of the network configuration manually,
# please utilize the 'source' or 'source-directory' directives to do
# so.
# PVE will preserve these directives, but will NOT its network
# configuration from sourced files, so do not attempt to move any of
# the PVE managed interfaces into external files!

auto lo
iface lo inet loopback

iface eno1 inet manual

iface eno2 inet manual
#interface1 vlan 201
iface eno1v201 inet manual
#interface2 vlan 201
iface eno2v201 inet manual

auto vmbr0
iface vmbr0 inet static
        address  10.21.32.100
        netmask  255.255.255.0
        gateway  10.21.32.254
        bridge-ports eno1 eno2
        bridge-stp off
        bridge-fd 0

auto vmbr0201
iface vmbr0201 inet manual
        bridge-ports eno1v201 eno2v201
        bridge-stp off
        bridge-fd 0
        bridge-vlan-aware yes
        bridge-vids 2-4094
 
Last edited:
I noticed I have MANUAL instead of STATIC in the vmbr0201 line in interfaces file. I will update to STATIC even though I don't know if I need an IP ADDRESS (again I don't want the Promox host reachable through my clients VMs/vlans), reboot and report back.
 
Hi,
for bridge vlan aware, you only need

auto vmbr0
iface vmbr0 inet static
address 10.21.32.100
netmask 255.255.255.0
gateway 10.21.32.254
bridge-ports eno1 eno2
bridge-stp off
bridge-fd 0
bridge-vlan-aware yes
bridge-vids 2-4094


(BTW, why do you have 2 eno ? if they are on same physical switch/network, you should use bonding, or you'll have network loop)


without vlanaware :

auto vmbr0
iface vmbr0 inet static
address 10.21.32.100
netmask 255.255.255.0
gateway 10.21.32.254
bridge-ports eno1 eno2
bridge-stp off
bridge-fd 0

auto vmbr0v201
iface vmbr0v201 inet manual
bridge-ports eno1.201 eno2.201
bridge-stp off
bridge-fd 0


(but you only need vmbr0, proxmox will create the vmbr0v201 when you'll start the a vm with vlan tag 201)



Of course, you need to configure you physical switch to allow vlans. (cisco trunk for example)
 
Thank you for the reply.

Unfortunately your first example is where I started and it did not work, probably due to starting the VM with a VLAN assigned to the guest VM didn't create the vmbr0v201 as you suggested. I did manually create it and it's adapters and restart to try that and it also did not work.

The 2nd example hasn't worked either (with vlan aware or without)

I have verified the trunk ports and router configuration is correct by assigned a PC with a vlan tag associated within the guest itself and verifying connectivity.

I will redo this process again to verify, but so far it's not working.

Also ENO1 is primary ENO2 is failover. I don't want or need the interfaces bonded because I lose static redundancy that way. I believe I read this is the default behavior in linux but now you have me 2nd guessing, so I'll look again.

Thanks,
Scott
 
This is getting on my last nerve. This documentation is WAYYYY too vague https://pve.proxmox.com/wiki/Network_Configuration

I have since created the bond0 and assigned eno1 and eno2 and assigned the bond-type to active-backup and it works great for VLAN1(untagged) traffic just like it always has.

I have tried creating individual interfaces (en1v201 and eno2v201) and set it to manual along with assigning a physical adapter (eno1 and eno2)

No luck.

I tried creating a new bridge and assigning eno1v201 and eno2v201, no luck

I tried created a bond0v201 and assigning eno1v201 and eno2v201 and then assigned bond0v201 to a bridge on it's own (using the web gui), also didn't work.

Can someone please update the web interface to either completely exclude the linux default if I am stuck using OVS or better yet UPDATE THE DOCUMENTATION TO INCLUDE EVERYTHING SINGLE SCENARIO.

I want UNTAGGED traffic to manage the Proxmox box. Everything else (everything tagged) CAN NOT sign in or see the web interface...at all.

The configuration sample in the official link doesn't do this. It assigned a VLAN and leaves everything else. I tried reversing that scenario and it still didn't work. This isn't rocket science. If you can't have a web interface that works, just remove it all together. (if you think the web interface works, try creating a new interface with LINUX (non ovs) ...you can't because it's not a drop down. Which ALSO means that every single time a client gets assigned a new VLAN I need to restart the host?! Bad design.
 
Code: 
auto lo
iface lo inet loopback

iface eno1 inet manual

iface eno2 inet manual

iface eno1v201 inet manual

iface eno2v201 inet manual

auto bond0
iface bond0 inet manual
        bond-slaves eno1 eno2
        bond-miimon 100
        bond-mode active-backup

auto bond0201
iface bond0201 inet manual
        bond-slaves eno1v201 eno2v201
        bond-miimon 100
        bond-mode active-backup
        vlan-raw-device bond0

auto vmbr0
iface vmbr0 inet static
        address  10.21.32.100
        netmask  255.255.255.0
        gateway  10.21.32.254
        bridge-ports bond0
        bridge-stp off
        bridge-fd 0
        bridge-vlan-aware yes
        bridge-vids 2-4094

auto vmbr0201
iface vmbr0201 inet manual
        bridge-ports bond0201
        bridge-stp off
        bridge-fd 0
        bridge-vlan-aware yes
        bridge-vids 2-4094

I've assign the VM to vmbr0 and vmbr0201 neither can talk to the router and there is absolutely no traffic hitting it because it's being blocked by the Proxmox VE host and I do not use the software firewall built into Proxmox VE
 
Try to remove: "bridge-vlan-aware yes".
Normally, if you have complex network config you do it over shell or use OVS.
In my opinion, the option VLAN aware is not an option you would have in an virtual environment, you would tag the interfaces in your Hypervisor and not in the VM itself. This add an additional layer which you can not control in an easy way. I tried the VLAN aware one time for pfsense, it doesn't worked, all my VMs on this specified node are not reachable anymore. I think there was anything wrong, but I revert my changes and don't check it again.

On my setup I always use the Shell, because PVE isnt able to do my Config. I create an Bonding device and assign multiple VLANs to bond devices and this I assign on the bridges where they get the IP addresses, and it works without problems.
 
Scott Zupek said:
Code: 
auto lo
iface lo inet loopback

iface eno1 inet manual

iface eno2 inet manual

iface eno1v201 inet manual

iface eno2v201 inet manual

auto bond0
iface bond0 inet manual
        bond-slaves eno1 eno2
        bond-miimon 100
        bond-mode active-backup

auto bond0201
iface bond0201 inet manual
        bond-slaves eno1v201 eno2v201
        bond-miimon 100
        bond-mode active-backup
        vlan-raw-device bond0

auto vmbr0
iface vmbr0 inet static
        address  10.21.32.100
        netmask  255.255.255.0
        gateway  10.21.32.254
        bridge-ports bond0
        bridge-stp off
        bridge-fd 0
        bridge-vlan-aware yes
        bridge-vids 2-4094

auto vmbr0201
iface vmbr0201 inet manual
        bridge-ports bond0201
        bridge-stp off
        bridge-fd 0
        bridge-vlan-aware yes
        bridge-vids 2-4094

I've assign the VM to vmbr0 and vmbr0201 neither can talk to the router and there is absolutely no traffic hitting it because it's being blocked by the Proxmox VE host and I do not use the software firewall built into Proxmox VE
Click to expand...


I really don't undestand your config,

what are this interfaces ?

iface eno1v201 inet manual
iface eno2v201 inet manual



If you use vlan in the vm configuration, you simply need:

vlan aware
----------------
Code: 
auto bond0
iface bond0 inet manual
        bond-slaves eno1 eno2
        bond-miimon 100
        bond-mode active-backup

auto vmbr0
iface vmbr0 inet static
        address  10.21.32.100
        netmask  255.255.255.0
        gateway  10.21.32.254
        bridge-ports bond0
        bridge-stp off
        bridge-fd 0
        bridge-vlan-aware yes
        bridge-vids 2-4094

(in this case, if you have vm with vlan 201 in the vm config, on vm start,
proxmox will add vlan tag directly on the vmbr0 port. you can see it with
"bridge -c vlan show" )



vlan unaware:
----------------------
Code: 
auto bond0
iface bond0 inet manual
        bond-slaves eno1 eno2
        bond-miimon 100
        bond-mode active-backup

auto vmbr0
iface vmbr0 inet static
        address  10.21.32.100
        netmask  255.255.255.0
        gateway  10.21.32.254
        bridge-ports bond0
        bridge-stp off
        bridge-fd 0

(in this case, if you have vm with vlan 201 in the vm config, on vm start,
proxmox will create a new vmbr0v201 + bond0.201. (Not added in /etc/network/interfaces, but you can see it with brctl show)





If you don't want to manage vlan config in the vm, you can manually create a vmbr with an interface in the vlan

Code: 
auto bond0
iface bond0 inet manual
        bond-slaves eno1 eno2
        bond-miimon 100
        bond-mode active-backup


auto vmbr0
iface vmbr0 inet static
        address  10.21.32.100
        netmask  255.255.255.0
        gateway  10.21.32.254
        bridge-ports bond0
        bridge-stp off
        bridge-fd 0

auto vmbr1
iface vmbr1 inet manual
        bridge-ports bond0.201
        bridge-stp off
        bridge-fd 0
 
Thank you for the response.

RESOLVED: Out of all the things it ended up being, it was a faulty switch configuration, NOT my /etc/network/interfaces configuration. To answer your question, I was trying everything under the sun to get it work, but in all honesty, the OUT OF BOX solution works just fine.

I was able to determine the problem by trying to ping the switch vlan interface to the firewall and wasn't getting a response. Then, there was a hardware firewall issue (SonicWALL) preventing pings on the vlan interface after that (duh!)

Once I corrected the switch problem the GUI default option worked and for the sake of reference, here is my currently running network config. It should look very familiar as it's all completed in the GUI.
/etc/network/interfaces
Code: 
#cat /etc/network/interfaces
auto lo
iface lo inet loopback

iface eno1 inet manual

iface eno2 inet manual

auto bond0
iface bond0 inet manual
        bond-slaves eno1 eno2
        bond-miimon 100
        bond-mode active-backup

auto vmbr0
iface vmbr0 inet static
        address  10.21.32.100
        netmask  255.255.255.0
        gateway  10.21.32.254
        bridge-ports bond0
        bridge-stp off
        bridge-fd 0
        bridge-vlan-aware yes
        bridge-vids 2-4094
and the brctl show command (not that I adjusted anything)
Code: 
#brctl show
bridge name     bridge id               STP enabled     interfaces
vmbr0           8000.d094669029f5       no              bond0
                                                        tap100i0
                                                        tap101i0
                                                        tap102i0
 
Scott Zupek said:
Thank you for the response.

RESOLVED: Out of all the things it ended up being, it was a faulty switch configuration, NOT my /etc/network/interfaces configuration. To answer your question, I was trying everything under the sun to get it work, but in all honesty, the OUT OF BOX solution works just fine.

I was able to determine the problem by trying to ping the switch vlan interface to the firewall and wasn't getting a response. Then, there was a hardware firewall issue (SonicWALL) preventing pings on the vlan interface after that (duh!)

Once I corrected the switch problem the GUI default option worked and for the sake of reference, here is my currently running network config. It should look very familiar as it's all completed in the GUI.
/etc/network/interfaces
Code: 
#cat /etc/network/interfaces
auto lo
iface lo inet loopback

iface eno1 inet manual

iface eno2 inet manual

auto bond0
iface bond0 inet manual
        bond-slaves eno1 eno2
        bond-miimon 100
        bond-mode active-backup

auto vmbr0
iface vmbr0 inet static
        address  10.21.32.100
        netmask  255.255.255.0
        gateway  10.21.32.254
        bridge-ports bond0
        bridge-stp off
        bridge-fd 0
        bridge-vlan-aware yes
        bridge-vids 2-4094
and the brctl show command (not that I adjusted anything)
Code: 
#brctl show
bridge name     bridge id               STP enabled     interfaces
vmbr0           8000.d094669029f5       no              bond0
                                                        tap100i0
                                                        tap101i0
                                                        tap102i0
Click to expand...


@Scott Zupek
Would you please share your network switch config or what's issue with it? My Proxmox VE host(s) were configuring similar to yours. However, the VM still is not working.
Anything I need to change on the switch to make it works.

RESOLVED:
Here's network config on the switch.

#show run int po 10
interface Port-Channel10
load-interval 30
switchport trunk native vlan 2004
switchport trunk allowed vlan 2001-2005
switchport mode trunk
ipv6 access-group rogue-ipv6 in
storm-control broadcast level 0.5
spanning-tree portfast
spanning-tree bpduguard enable

Of course, you MUST to enter "VLAN Tag" ex: (2001) when creating a new VM for the other VLAN.
 
Last edited:
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom