Host with 1 iface, VM with 2 ifaces (one in VLAN)

[marcus.cps]

Hi,

I read about how to create VLANs on a VM using nmcli, but I believe I have a more fundamental question.

My host has only 1 physical network interface, connected to a VLAN enabled swtich, on a untagged port (no vlan tag).

I have vlans 10 (the default) and 20.

My VM has 2 network deviecs: net0 and net1, with net0 untagged and net1 tagged for vlan 20.
The VM enp0s18 (net0) interface gets an IP 10.0.10.100 from the DHCP (Edge Router X router connected to the same switch).
However, enp0s19 (net1) interface gets no IP even though it is configured just as enp0s18. I expected it would be able to get the IP 10.0.20.100 from my vlan 20 dhcp server (also running on the router).

I thought proxmox would abstract the vlan tagging and make it transparent for the VM. Did I get it wrong?
Can someone point me on the right direction?

Thanks in advance, and sorry for my my lack of knowledge on the matter.

---
Edit 1:
I've found a video that seems to address the issue: https://www.youtube.com/watch?v=ljq6wlzn4qo

If I understood correctly, I'll have to create a vlan bridge on proxmox vmbr0.20 and set net1's bridge to vmbr0.20.
I also need to change my router to only accept tagged traffic on the port.
The switch port will have to be set as well to accept both Vlans 10 and 20.

I'll also have to create vmbr0.10 for the default vlan and update all VMs to be vlan aware as well, using 10 or 20.
Is it reasonable?

 

[Spaneta]

Hello,

You don't have to create vmbr0.X for any X VLAN, setting the VLAN in the virtual NIC properties of a VM is perfectly fine. The video show this around 24-25 minutes. Creating vmbrX.Y is mainly used to give different IP's to a PVE node itself, for management, clustering, dedicated network storage, ...

The bare minimum, given your situation :
- ensure the "VLAN aware" option is enabled for the vmbr
- make the switch port also accept traffic for tagged VLAN 20, which can be summarized by :

- untagged VLAN 10​

- tagged VLAN 20​

As a better option I would recommend doing only tagged traffic in this case, thus giving as settings :
- ensure the "VLAN aware" option is enabled for the vmbr (like above)
- add VLAN tag 10 to net0 of the VM
- make the switch port accept only tagged traffic in wanted VLAN's and send all other traffic to a "black hole" VLAN

- tagged VLAN 10​

- tagged VLAN 20​

- PVID = number of a VLAN used for NOTHING and different from 1 (will probably need to be created on the switch to be used as PVID for the port)​

- remove VLAN 1 from the port (explicitely needed on some switch to prevent VLAN 1 traffic to be trasmitted "by default")​

Another (small) refinement would be to create 2 black-hole VLAN's :
- 1 to be used for access / untagged ports that are not used (even if shutdown)
- 1 to be used on trunk / tagged ports

Hope it helps !

EDIT: readability + typos

 

[marcus.cps]

Hello,

You don't have to create vmbr0.X for any X VLAN, setting the VLAN in the virtual NIC properties of a VM is perfectly fine. The video show this around 24-25 minutes. Creating vmbrX.Y is mainly used to give different IP's to a PVE node itself, for management, clustering, dedicated network storage, ...

The bare minimum, given your situation :
- ensure the "VLAN aware" option is enabled for the vmbr
- make the switch port also accept traffic for tagged VLAN 20, which can be summarized by :

- untagged VLAN 10​

- tagged VLAN 20​

As a better option I would recommend doing only tagged traffic in this case, thus giving as settings :
- ensure the "VLAN aware" option is enabled for the vmbr (like above)
- add VLAN tag 10 to net0 of the VM
- make the switch port accept only tagged traffic in wanted VLAN's and send all other traffic to a "black hole" VLAN

- tagged VLAN 10​

- tagged VLAN 20​

- PVID = number of a VLAN used for NOTHING and different from 1 (will probably need to be created on the switch to be used as PVID for the port)​

- remove VLAN 1 from the port (explicitely needed on some switch to prevent VLAN 1 traffic to be trasmitted "by default")​

Another (small) refinement would be to create 2 black-hole VLAN's :
- 1 to be used for access / untagged ports that are not used (even if shutdown)
- 1 to be used on trunk / tagged ports

Hope it helps !

EDIT: readability + typos

Thanks a lot for your help. Very detailed and clear answer. I'll try these soon.

The only part I didn't understand so well was the last small refinement with 2 black hole vlans. But this is certainly due to my lack of knowledge. I'll study a bit more to get it right.

I'll post back here my final setup as soon as I make it right, for the benefit of others.

 

[marcus.cps]

Hello,

You don't have to create vmbr0.X for any X VLAN, setting the VLAN in the virtual NIC properties of a VM is perfectly fine. The video show this around 24-25 minutes. Creating vmbrX.Y is mainly used to give different IP's to a PVE node itself, for management, clustering, dedicated network storage, ...

The bare minimum, given your situation :
- ensure the "VLAN aware" option is enabled for the vmbr
- make the switch port also accept traffic for tagged VLAN 20, which can be summarized by :

- untagged VLAN 10​

- tagged VLAN 20​

As a better option I would recommend doing only tagged traffic in this case, thus giving as settings :
- ensure the "VLAN aware" option is enabled for the vmbr (like above)
- add VLAN tag 10 to net0 of the VM
- make the switch port accept only tagged traffic in wanted VLAN's and send all other traffic to a "black hole" VLAN

- tagged VLAN 10​

- tagged VLAN 20​

- PVID = number of a VLAN used for NOTHING and different from 1 (will probably need to be created on the switch to be used as PVID for the port)​

- remove VLAN 1 from the port (explicitely needed on some switch to prevent VLAN 1 traffic to be trasmitted "by default")​

Another (small) refinement would be to create 2 black-hole VLAN's :
- 1 to be used for access / untagged ports that are not used (even if shutdown)
- 1 to be used on trunk / tagged ports

Hope it helps !

EDIT: readability + typos

I'm back. I spent a few hours trying to solve the issue yesterday.

I followed your suggestion:

1 - Tagged VM net0 as VLAN 10
2 - Tagged VM net1 as VLAN 20
3 - Enabled "VLAN Aware" for vmbr
4 - Modified the switch configuration to accept VLANs 10 and 20 and redirected untagged traffic to a Black-Hole VLAN on router switch0 config

PVID = 4096
VID = 10,20

This solved the problem but created another.
Now the VM can get IPs from DHCP in both VLANS. That's great!

But now I can't access the Proxmox Web UI anymore.
My understanding is Proxmox itself can't get an IP address as its own traffic is not tagged with VLAN 10.
I suppose that's why in the video he creates the vmbr0.10 Linux Vlan on Proxmox. I tried doing it, and everything stopped working!

I tagged the switch port I'm connecting my PC with VLAN 10 for untagged traffic, and accepting VLAN 20, to be sure I'd be able to reach any devices in any VLANs for maintenance. This works well and I can ping all devices.

PVID = 10
VID = 20

However, some of my services running on VMs started misbehaving. I have 2 VMs:
- VM1: NGINX Reverse Proxy - Centralises all HTTPs and Certificates Management. It also set net0 tagged for VLAN 10.
- VM2: Home Assistant (that's the one I applied all the configs above).

I can load Home Assistant using my external domain address, but it won't load its data correctly. Something is off there...

I tried tagging switch0's eth0 with VLAN 10 so WAN incoming traffic would default to VLAN 10, but the router doesn't accept it (configuration is invalid).
So I don't think I can use a black hole VLAN for the Trunk. Or maybe I'm doing something wrong.

Thanks again for your help.

 

[Spaneta]

Yes, if the PVE node also use VLAN 10 for its management it needs to have a vmbr0.10 (or similarly vmbrX.Y) if you use only tagged VLAN's on the switch port connected to your PVE.

Please also note that the server need to have _ONE_ gateway - in a correctly configured VLAN on PVE side and switch side - to be reachable from other VLAN's.

For the black hole VLAN, this is optionnal but except if your switch have some unusual "management values", VLAN 4096 should _NOT_ be valid in _ANY_ network equipment. Please try a VLAN in the 2-9 range for example if you plan on using only VLAN 10 and above for "real" use.

For the switch port connected to your management PC it should only be "untagged X" and no need to add any tagged VLAN here because this is the router or L3 switch of your home / site that forward traffic from VLAN to other VLAN.

What I don't understand - based on your previous post - is the map of the network and what remains to be fixed.
Could you elaborate on this ?

Have a nice day,

 

[marcus.cps]

Yes, if the PVE node also use VLAN 10 for its management it needs to have a vmbr0.10 (or similarly vmbrX.Y) if you use only tagged VLAN's on the switch port connected to your PVE.

Please also note that the server need to have _ONE_ gateway - in a correctly configured VLAN on PVE side and switch side - to be reachable from other VLAN's.

For the black hole VLAN, this is optionnal but except if your switch have some unusual "management values", VLAN 4096 should _NOT_ be valid in _ANY_ network equipment. Please try a VLAN in the 2-9 range for example if you plan on using only VLAN 10 and above for "real" use.

For the switch port connected to your management PC it should only be "untagged X" and no need to add any tagged VLAN here because this is the router or L3 switch of your home / site that forward traffic from VLAN to other VLAN.

What I don't understand - based on your previous post - is the map of the network and what remains to be fixed.
Could you elaborate on this ?

Have a nice day,

It makes sense. I'll try it this weekend and will come back with results.
I'll also clarify the explanation so others can benefit.

Thanks for your reply.