Need some help figuring out a VLAN Trunk over VMs

[BathToast]

Hey everyone, still getting some things figured out.

So i've been working trying to get some networking up and running. I've tried a few solutions i've seen online but havent gained much traction personally, so i figured i'd reach out.

This is my interface setup at the moment

auto lo
iface lo inet loopback

iface eno1 inet manual

iface enp4s0f0 inet manual

iface enp4s0f1 inet manual

iface eno2 inet manual

iface eno3 inet manual

iface eno4 inet manual

auto vmbr0
iface vmbr0 inet static
        address 10.100.0.10/29
        gateway 10.100.0.9
        bridge-ports eno1
        bridge-stp off
        bridge-fd 0

auto vmbr1
iface vmbr1 inet manual
        bridge-ports enp4s0f0
        bridge-stp off
        bridge-fd 0
        bridge-vlan-aware yes
        bridge-vids 2-4094
#Fiber Trunk

I have two NIC's in use here. One thats hard copper linked up (eno1) thats the vmbr0 bridge. This is on its own IP address for the web client.

Then i have vmbr1, my 10g Fiber connection (enp4s0f0), which i intend to push my VLAN traffic through.

On the other end of all this, I have my Cisco Switch pushing VLAN 200 (set to access) to eno1, and for enp4s0f0 i'm pushing VLANs 302, 303, 400, 401, and 402 on a Trunk with no native on it.

I have them separated for primarily arbitrary reasons in all honesty, but i'm not sure if thats part of my problem. However, I figured i'd ask what would be the best path forward. For my test windows VM i did install the Virtual IO drivers fromt he proxmox website, and set it up to the proper Subnet for VLAN 400.

Any detailed instruction or advice is much appreciated.

 

[Dunuin]

You can either create one virtual NIC for each VLAN the VM should be part of and then tag those vNICs with a VLAN (YourVM -> Hardware -> Your vNIC -> Edit -> VLAN Tag). In this case tagging/untagging will be done on the PVE host and the VM will only send/receive untagged packets.
Or you give that VM a single vNIC and don't set a VLAN tag. Then that VM can listen/send to all tagged packets on that bridge and you need to set up VLANs inside the VM.

 

[BathToast]

You can either create one virtual NIC for each VLAN the VM should be part of and then tag those vNICs with a VLAN (YourVM -> Hardware -> Your vNIC -> Edit -> VLAN Tag). In this case tagging/untagging will be done on the PVE host and the VM will only send/receive untagged packets.
Or you give that VM a single vNIC and don't set a VLAN tag. Then that VM can listen/send to all tagged packets on that bridge and you need to set up VLANs inside the VM.

@Dunuin,

Finally getting around to trying this, still running into a bit of trouble.

auto lo
iface lo inet loopback

iface eno1 inet manual

iface enp4s0f0 inet manual

iface enp4s0f1 inet manual

iface eno2 inet manual

iface eno3 inet manual

iface eno4 inet manual

auto vmbr0
iface vmbr0 inet static
        address 10.100.0.10/29
        gateway 10.100.0.9
        bridge-ports eno1
        bridge-stp off
        bridge-fd 0
#Copper Wire Management Interface

auto vmbr1
iface vmbr1 inet manual
        bridge-ports enp4s0f0
        bridge-stp off
        bridge-fd 0
        bridge-vlan-aware yes
        bridge-vids 2-4094
#Fiber Link Bridge

auto vmbr1.400
iface vmbr1.400 inet manual
#Test VLAN

This is what i have so far. I'm running a test VLAN of 400, its tagged on my managed switch and being trunked over enp4s0f0 to my Proxmox Host.

I have a VM set up to use vmbr1, with VLAN tag 400 using a VirtIO Model Network Card, and internally on that VM its set to the first availible address of VLAN400

However, it doesnt get a connection. OPNSense Firewall rules are set to be completely open, no restrictions what so ever, and I cant ping out to anything from the VM or to the VM. I can ping the default gateway for the VLAN from my Proxmox Host.

So that tells me i'm configuring something here wrong, I'm not particuarly wrapping my head around how Proxmox handles tagged traffic, coming from what i'm used to its a little different. The theory makes sense but the practice doesnt. So if you could provide some step by step information it would be much appreciated. CLI or Gui, either is fine.

Once i get this figured out i should be able to do the rest.

 

[cwt]

So you did use vmbr1.400 for the VM? What happens if you switch to vmbr1 instead and set tag 400 within the VM‘s config?

 

[BathToast]

So you did use vmbr1.400 for the VM? What happens if you switch to vmbr1 instead and set tag 400 within the VM‘s config?

I'm unable to select vmbr1.400 in the gui, i have tried setting vmbr1 and the tag to 400, nothing changes: its still entirely incapable of reaching the network. hence my confusion.

 

[BathToast]

Update to this situation:

I was confused as to why my ethernet would have this work just fine (and hence my orginal assumption that whatever interface was configured initially had to be the one used for VLAN trunking)

My Fiber card is a QLogic 10Gb 2P QLE8262, QLogic cards seem to have drivers that are at the very least: not exactly eager to get along with proxmox. I've found any QLogic Card thread on this forum often ends in the need for a work around, use of Proxmox 6.1 or older, or the most often solution: Replacing the card entirely with a different model such as a broadcomm.

Thanks to those who tried helping, their solutions work just fine on copper, and i've isolated the problem as a driver issue with my PCIe SFP+ Card, and can confidently say that there was nothing wrong with my Configs nor an issue with Proxmox.

I'm just going to bond 2 ethernet ports and call it good at that for the time being until I can get a new card in.

 

[GeoStigma]

@Dunuin,

Finally getting around to trying this, still running into a bit of trouble.

auto lo
iface lo inet loopback

iface eno1 inet manual

iface enp4s0f0 inet manual

iface enp4s0f1 inet manual

iface eno2 inet manual

iface eno3 inet manual

iface eno4 inet manual

auto vmbr0
iface vmbr0 inet static
        address 10.100.0.10/29
        gateway 10.100.0.9
        bridge-ports eno1
        bridge-stp off
        bridge-fd 0
#Copper Wire Management Interface

auto vmbr1
iface vmbr1 inet manual
        bridge-ports enp4s0f0
        bridge-stp off
        bridge-fd 0
        bridge-vlan-aware yes
        bridge-vids 2-4094
#Fiber Link Bridge

auto vmbr1.400
iface vmbr1.400 inet manual
#Test VLAN

This is what i have so far. I'm running a test VLAN of 400, its tagged on my managed switch and being trunked over enp4s0f0 to my Proxmox Host.

I have a VM set up to use vmbr1, with VLAN tag 400 using a VirtIO Model Network Card, and internally on that VM its set to the first availible address of VLAN400

However, it doesnt get a connection. OPNSense Firewall rules are set to be completely open, no restrictions what so ever, and I cant ping out to anything from the VM or to the VM. I can ping the default gateway for the VLAN from my Proxmox Host.

So that tells me i'm configuring something here wrong, I'm not particuarly wrapping my head around how Proxmox handles tagged traffic, coming from what i'm used to its a little different. The theory makes sense but the practice doesnt. So if you could provide some step by step information it would be much appreciated. CLI or Gui, either is fine.

Once i get this figured out i should be able to do the rest.

@BathTost, this solution worked for me. Thanks for sharing.