Documented Linux VLAN Configuration not working

[czi.jmp]

First I want to preface with some facts that might be relevant:

1. I am not a network admin by day, but I'm a very technical person and I hope that comes across in my testing / issue description. If not, sorry.
2. I am not hosting ProxMox in a data center, this is my home network for my home lab. I have full control over all networking infrastructure and systems.
3. I've tried researching this but maybe I'm barking up the wrong tree. All the examples I've seen just don't work
4. I'm sure I'm doing something wrong or missing something, but I'm not sure what.
5. I have an Ubiquity USG with VLAN's already setup, configured, and working.
   1. I have tagged the ports for both network cards to different VLANs (enp4s0 to VLAN 5 - 192.168.5.0/24 and enp5s0 to VLAN 7 - 192.168.7.0/24)
   2. I've been using these VLANs as they are currently configured for quite some time with singular server and personal systems and everything has been working just fine.
   3. All VLAN's provide their own DHCP and gateways, no central DHCP, DNS, or Gateways being served up by anything to complicate things.
   4. I have checked my Ubiquity Firewall config and traffic from my home device VLAN 2 (192.168.2.0/24) to VLAN 5 is unrestricted. Traffic to VLAN 7 is restricted to specific ports, but those ports are for the docker images running on the CT/VM and I've confirmed the ports are correct.
   5. Everything outside of proxmox works fine with the VLAN configs as is (not saying there's not a problem with it comes to proxmox and Ubiquity VLANS that I might be missing)

I have two network cards. One is on the motherboard (enp4s0), and a 10gig ethernet card (enp5s0).

I've tried this same approach below on both network cards with the same result, loss of connection.

So let's start with something very simple the default configuration for ProxMox host networking. This works fine. I can get to the management interface no problem.

auto lo
iface lo inet loopback

iface enp4s0 inet manual

iface enp5s0 inet manual

auto vmbr0
iface vmbr0 inet static
        address 192.168.5.7/24
        gateway 192.168.5.1
        bridge-ports enp4s0
        bridge-stp off
        bridge-fd 0

iface wlo1 inet manual

Now, I refer to the Proxmox documentation on setting up a network interface that supports VLANs and isolates management traffic to a specific VLAN.



This causes a complete loss of connectivity. I have to manually revert the /etc/network/interfaces file to the original and issue an ifreload -a

But here's my Interfaces file after the modifications which causes the loss of connection:

auto lo
iface lo inet loopback

iface enp4s0 inet manual

iface enp5s0 inet manual

auto vmbr0
iface vmbr0 inet manual
        bridge-ports enp4s0
        bridge-stp off
        bridge-fd 0
        bridge-vlan-aware yes
        bridge-vids 2-4094

iface wlo1 inet manual

auto vlan0
iface vlan0 inet static
        address 192.168.5.7/24
        gateway 192.168.5.1
        vlan-raw-device vmbr0

Things I've tried:

• Using the bridge version of segmenting the management interface - complete loss of connectivity
• Setting up the secondary network card (enp5s0 - 10GB - VLAN 7 - 192.168.7.0/24) on a separate bridge (vmbr1) with VLAN
  • CT/VM configured with VLAN 7 and vmbr1, with static IP - complete loss of connectivity
  • CT/VM configured with VLAN 7 and vmbr1, with DHCP - CT/VM never gets an assigned IP Address
• Setting up the second network card (enp5s0 - 10GB - VLAN7 - 192.168.7.0/24) on a separate bridge without VLAN
  • CT/VM configured with out VLAN, with vmbr1, with DHCP - Works fine.

Now, I think I can probably get by with that last option for what I run today, I just basically don't use ProxMox VLAN tagging. But I will miss out on the ability to run multiple VLAN'd systems off my 10G interface and that feels limiting.

I'd really like to understand what I'm missing so I can figure out how to use ProxMox VLAN tagging.

Any help would be greatly appreciated.

 

[gurubert]

If you want to use the VLAN on the Proxmox host and not only for the VMs it is better to create a VLAN interface attached to the Ethernet interface and then a bridge interface on top of that VLAN interface:

iface enp4s0 inet manual

iface enp4s0.5 inet manual

iface vmbr5 inet static
    bridge-ports enp4s0.5

You can still use a bridge on top of enp4s0 that is VLAN aware for all the VMs.

 

[czi.jmp]

If you want to use the VLAN on the Proxmox host and not only for the VMs it is better to create a VLAN interface attached to the Ethernet interface and then a bridge interface on top of that VLAN interface:

iface enp4s0 inet manual

iface enp4s0.5 inet manual

iface vmbr5 inet static
    bridge-ports enp4s0.5

You can still use a bridge on top of enp4s0 that is VLAN aware for all the VMs.

I will definitely try this.

I don't mind using the VLAN just for VM's if that's an option, but I configured the 10G ethernet for the VM's the same way as I did above and the VM's would never get a DHCP assignment.

So would your recommendation above still be the recommended approach for the VM's?

 

[gurubert]

I have tagged the ports for both network cards to different VLANs (enp4s0 to VLAN 5 - 192.168.5.0/24 and enp5s0 to VLAN 7 - 192.168.7.0/24)

Are these ports tagged or untagged in their VLANs?
From the rest of your text it looks like they are untagged. The Ethernet packets then do not transport a VLAN ID and you do not need to configure any VLAN aware interfaces on the Proxmox host.

auto lo
iface lo inet loopback

iface enp4s0 inet manual

iface enp5s0 inet manual

auto vmbr5
iface vmbr5 inet manual
        bridge-ports enp4s0
        bridge-stp off
        bridge-fd 0
        address 192.168.5.7/24
        gateway 192.168.5.1

auto vmbr7
iface vmbr7 inet manual    
        bridge-ports enp5s0
        bridge-stp off
        bridge-fd 0

Use vmbr5 for VMs in VLAN 5 and vmbr7 for VMs in VLAN 7.