VM in VLAN Can't Access Internet

mwuz

New Member
Sep 9, 2024
6
1
3
I'm having trouble getting out to the Internet from a guest and need some help figuring out what I did wrong.
  • Proxmox v8.2.2
  • VMBR0 is a Linux bridge (NICs 1 and 2 bonded) that is used for all network activities for the host and guests.
  • VMBR1 is up for now to be used for testing VLAN awareness and is used for only one guest (a management IP isn't assigned to it).
  • Our network is new and flat.
  • All ports in LAN gateway and switches are set to "trunk"
  • VLAN40 (192.168.40.1/27) is setup in our Meraki MX gateway
  • VM NIC configured to use VMBR1 and whether or not the VLANID is added to the config the computer can't get out to the Internet

I thought when I configured my test computer with a 192.168.40.2 (gateway 192.168.40.1, mask: 255.255.255.224) IP address that everything would just work but I'm finding that I can't get out to the Internet. I'm not sure what I've done wrong. I think with a flat network and all device ports set to Trunk that the problem lies in how I have things configured in Proxmox. Can you guys think of what I should be looking at to get this working?

Thanks,

MJ
 
I think with a flat network and all device ports set to Trunk that the problem lies in how I have things configured in Proxmox.
What does the rest of your network look like? Do you have a router on the other side of the managed switch?
 
@louie1961, thank you for replying - the VLAN is set . I wish I had a drawing but it's not a complicated network:
Meraki MX Gateway > Meraki Switch > Dell Managed Switch > Proxmox Server. The network is flat until I figure this out.
  • The VLAN is created in the Meraki gateway.
  • All Gateway and Switch ports are set to trunk
  • I can reach the gateway from the Dell switch
  • Every other vm in Proxmox in the native vlan can access the Internet which leads me to believe the problem is with how I've configured the networking in Proxmox.
/etc/network/interfaces
auto lo
iface lo inet loopback

iface eno1np0 inet manual

iface eno2np1 inet manual

iface eno3np2 inet manual

iface eno4np3 inet manual

auto vmbr0
iface vmbr0 inet static
address 192.168.10.177/24
gateway 192.168.10.1
bridge-ports eno1np0 eno2np1
bridge-stp off
bridge-fd 0
#Linux bridge for VM failover

auto vmbr1
iface vmbr1 inet manual
 
You are correct. Your network is not set up correctly. Here is my VLAN aware etc/network/interfaces file

auto lo
iface lo inet loopback

iface eno1 inet manual

auto vmbr0
iface vmbr0 inet static
bridge-ports eno1
bridge-stp off
bridge-fd 0
bridge-vlan-aware yes
bridge-vids 2-4092

auto vmbr0.100
iface vmbr0.100 inet static
address 192.168.100.2/24
gateway 192.168.100.1

source /etc/network/interfaces.d/*

The "auto vmbr0" section makes the bridge VLAN aware and binds the bridge to my NIC, eno1 The section "auto vmbr0.100" creates a virtual bridge on VLAN 100 which is my management VLAN. The Proxmox web interface is on 192.168.100.2 With this setup I can access any VLAN from any VM. If you have multiple bridges (vmbr0, vmbr1, etc.) you will need to do this for each VLAN aware bridge.
 
@louie1961 my concern is with vmbr1 where there is almost nothing configured. My problem is that I am not a network tech :) and want to use the web interface for most of my Proxmox management and configuration.

Using the Proxmox web interface I created a Linux Bridge that utilizes eno3np2 (my 3rd pNIC) and made sure to check the "VLAN aware" box. When I did this, vmbr1's config should've been saved in "interfaces". Since nothing notable is there, however, I wonder if there is another step that I'm not taking in the GUI that allows the config for vmbr1 to be properly noted in the "interfaces" config?

1725995369087.png

Is it possible to use the GUI to do what I need to do or does some of the config have to take place in an SSH session?
 
Honestly I have never done this via the GUI. I have always done it via ssh or actually plugging in a monitor and keyboard directly to my proxmox host and using the console to make the changes on a fresh install.
 
Here's how my gui looks for the etc/network/interfaces file I posted earlier. I am in dark mode is the only difference. You could try copying this.
1726019546004.png

1726019634312.png
 
Thanks for hanging in there with me @louie1961. My week is getting away from me so I haven't gotten back to Proxmox today but I didn't want to leave your messages unanswered. I'll follow-up later in the week or early next week. So far I still haven't gotten things to work the way I want.
 
  • Like
Reactions: louie1961
I made time for this again this morning and made a few changes. First, I got rid of vmbr1. I made vmbr0 VLAN-aware yesterday. I also added a Linux VLAN for 40 which I didn't have prior to seeing your screenshot @louie1961 . This is what my interface config looks like now:

auto lo
iface lo inet loopback

iface eno1np0 inet manual

iface eno2np1 inet manual

iface eno3np2 inet manual

iface eno4np3 inet manual

auto vmbr0
iface vmbr0 inet static
address 192.168.10.177/24
gateway 192.168.10.1
bridge-ports eno1np0 eno2np1
bridge-stp off
bridge-fd 0
bridge-vlan-aware yes
bridge-vids 2-4094
#Linux bridge for VM failover

auto vmbr0.40
iface vmbr0.40 inet manual

source /etc/network/interfaces.d/*

I needed to get my config closer to mirroring yours while I'm troubleshooting. I'm very new to Proxmox and Linux networking so during our next service window I will be recreating the vmbr0 bridge using a bond that I have yet to create. Otherwise my config now looks similar to yours and this machine still can't get out to the Internet. It can't even ping its gateway (192.168.40.1). Do you see a problem in the config above?
 
Last edited:
You need to move the ip addresses down to the vmbr0.40 section. See below. I would do this without a bond to start with since using a bond requires specific settings in your switch or other network setup.

auto lo
iface lo inet loopback

iface eno1np0 inet manual

iface eno2np1 inet manual

iface eno3np2 inet manual

iface eno4np3 inet manual

auto vmbr0
iface vmbr0 inet static
bridge-ports eno1np0 eno2np1
bridge-stp off
bridge-fd 0
bridge-vlan-aware yes
bridge-vids 2-4094
#Linux bridge for VM failover

auto vmbr0.40
iface vmbr0.40 inet static
address 192.168.10.177/24
gateway 192.168.10.1


source /etc/network/interfaces.d/*
 
Be careful with bridging two ports on vmbr0, you can easily cause a network storm due to a loop. Either start with one interface if you just want to test or bond them (active-bakcup doesn't require any switch support for instance).
 
  • Like
Reactions: louie1961
@shanreich, we realized our error and have corrected it courtesy of an unplanned outage yesterday! @louie1961 I did as you suggested yesterday and boy did that not go well :eek:. I'm back for another round of "What-did-@mwuz-do-wrong-this-time?"

The story is enclosed in quotes so the post wasn't a mile long :)
  • I changed the IP address of the management interface on vmbr0
  • I added the management network to vmbr0.40
  • I deleted the management interface from vmbr0
Then I completely lost access to Proxmox. I made my way to the KVM, logged into Proxmox and opened \etc\network\interfaces and everything looked right. I restarted networking using
Code:
service pveproxy restart
, no change. Then I hit the Internet to find more Debian commands and really messed things up by using
Code:
systemctl restart networking.service
- at this point I lost access to my guests and still I could not reach Proxmox via SSH or web interface.

Then I deleted all of the references to vmbr0.40 and moved the management interface back to vmbr0, saved the changes, and executed the pveproxy restart. No change. I checked the hosts file to see if I needed to modify anything there but it only had IPV6 references in it and I don't use IPV6.

In fact nothing changed until I changed the config to allow for DHCP. With DHCP enabled I got an IP address that allowed me to get back into the web GUI. Here I found my bridge, vmbr0, but I could not edit or delete it. So I created vmbr1 and deleted vmbr0 in the config file and then I was fully functional again.
So . . . .
What do you make of my losing all access to Proxmox simply by moving the management IP from vmbr0 to vmbr0.40?
Aside from "very carefully", are there any suggestions you can offer to avoid the mess I was in yesterday?

In preparation for a time when all of this is working, and we start segmenting our network properly I'm interested in knowing how I will add other VLANs. What I've learned here is that introducing VLAN40 forced the need for me to move my management network to VLAN40. Where does the management network reside when I've got multiple VLANs?
 
Are you hooking the machine to a managed switch, AND is the port or ports you are using set up to be trunk ports (i.e., tagged ports)? Also, what do the firewall rules for you VLAN look like? The only time I personally have experienced this is when I set up my Proxmox machine to be VLAN aware and connected it to access/un-tagged port, or visa versa, when I set up a new install of Proxmox, and haven't yet modified my vmbr to be LAN aware, and I connected it to a trunked port. Both situations will break your network connection to the box.

Now I assumed VLAN 40 was you management VLAN? If not that could be the problem. In the example I shared with you, VLAN 100 is my management VLAN, and the IP address that I set vmbr0.100 to is my management interface for Proxmox. I do not need to do anything to configure other VLANs other than the following:
1. Make sure the VLANs exist in my firewall router. If you are using a layer 3 switch, I would imagine you have to create them there as well. But I have no experience with L3 switches. I do everything in my firewall/router machine (pfSense)
2. VLANs only work on managed switches. If you are trying to connect through an unmanaged switch, that will be a problem
3. Your managed switch needs to proved a trunk (tagged) port to the NIC you are using for your management interface/vmbr0
4. Your chain of network devices all the way back to your WAN has to be connected via trunked ports. So my managed switch is connected to my pfSense box via trunk ports.
5. Set your firewall (or routing rules/ACLs in an L3 switch) to allow inter VLAN communication as you see fit. Whatever VLAN your PC is on has to be able to traverse to your management VLAN or you won't be able to access the gui/log in via SSH

Where does the management network reside when I've got multiple VLANs?
It only resides in the network hardware configuration for individual VMs and CTs (one of the earlier pictures I attached). As long as your vmbr0 (not vmbr0.40) is VLAN aware, it will pass the traffic to your network, assuming all the conditions I outlined above. vmbr0.40 is only applicable to your management interface/SSH into the Proxmox host.

At the risk of being overly verbose here, let me explain my set up a little more and hopefully it will illustrate the point for you. My network consists of a cable modem for a WAN which feeds into my pfSense box (firewall/router). I run a link from my pfSense box to my managed switch (on a trunk port). My LAN is segmented into 6 VLANs: Trusted, Untrusted, TVs, IOT, Guest, and Management. Of the 24 ports on my switch, some are trunked (I have to set this up in the switch console) and connect to my Proxmox machines as well as to my WAP, which is VLAN aware and had up to 8 SSIDs. The remaining ports are access ports (dedicated to only one VLAN). Some of those are dedicated to the Trusted VLAN, some to the Untrusted VLAN, some to the IOT VLAN. None are dedicated to Guests, TVs or Management, just because I don't need wired connections for those.

Inside my Proxmox boxes, you have seen how I configure my vmbr0 and vmbro.100. The IP address is the address of the server itself (GUI/SSH). The gateway points back to my router, always at 192.168.100.1. I don't need to provide any other gateways or routes, since it is all on one physical device. Proxmox does not require you to set up a gateway for each VLAN, as long as the gateway goes to your router.

When I set up a new VM or CT, I select which VLAN I want to be on at setup time. Or I can actually change it later if I want (like if using a template to spin up a new VM).

My home PC that I use to run/manage everything from sits on the Trusted VLAN. That VLAN, in firewall rules, can access everything in every other VLAN. My Guest, IOT, TV, Untrusted, and Management VLANs can only see the internet/WAN. They are not allowed to access the pfSense management interface, and they are not allowed to see into any other VLANs.

Each VLAN is on its own subnet and has its own DHCP server. I set up new VMs with DHCP and I control all the IP addressing through DHCP reservations in pfSense based on the mac address of the NIC in question.

When I need to log into a pfSense box or SSH into a VM, I do it from my home PC on the trusted VLAN. Even though that PC is on a completely different subnet/ip address range, the firewall rules let me access whatever I want on any VLAN
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!