VLAN tagging

[martijnk]

Hi,

I'm trying to set up isolated LANs, so I need VLANs.

Now when I give 2 VMs same VLAN, e.g. 300 it works, both machines see each other and can't ping. But... only when they're on the same node. 2 VMs on different nodes are not able to ping. Probably because the switch doesn't understand this.

Can this be done with an OVS virtual switch? I can't configure VLANs on the switch, since I have a lot of clients that all want to use VLANs, I can't just add one port to 200 different VLANs on the switch. Well I could but it's a lot of work and admin. I just want to let the traffic go to untagged vlan 1 on the switch and let proxmox do the tag work.

Thanks!

 

[spirit]

Probably because the switch doesn't understand this.
I can't configure VLANs on the switch

Indeed, you need to configure your swich port to allow vlan tag going though interface (on cisco, it's called trunk mode for example).
(you could allow "any" vlans, or a big range on your switch like 2-4096 for example)

I just want to let the traffic go to untagged vlan 1 on the switch and let proxmox do the tag work.

no, you can't, or the vlan tag will be removed when going though your switch.

Another way is to use vxlan tunnel for example
you can use the new sdn feature for this, currently in beta
https://forum.proxmox.com/threads/proxmox-6-2-sdn-beta-test.69655

you can setup multiple vxlan-id for each customer network, or 1 big vxlan tunnel and do vlan tag inside it)

 

[martijnk]

Indeed, you need to configure your swich port to allow vlan tag going though interface (on cisco, it's called trunk mode for example).
(you could allow "any" vlans, or a big range on your switch like 2-4096 for example)


no, you can't, or the vlan tag will be removed when going though your switch.

Another way is to use vxlan tunnel for example
you can use the new sdn feature for this, currently in beta
https://forum.proxmox.com/threads/proxmox-6-2-sdn-beta-test.69655

you can setup multiple vxlan-id for each customer network, or 1 big vxlan tunnel and do vlan tag inside it)

Thanks spirit.

I've been trying multiple things but none of them seem to work. I'm using Dell switches, so I found out the native vlan option. I've added the port to untagged vlan 1 (because my storage uses this traffic). Then I added the port of both hypervisors to tagged VLAN 300 on the switch as well.

Still though, machines can't ping/see each other when I tag them both to vlan 300.

I'm a little confused at this point, but I must say I'm a bit new to this as well

 

[martijnk]

Any idea? I now have my switch configured like it should, according to Dell. But VLAN isn't working cross machines. I've tagged 2 VMs on VLAN 300 which is configured on the switch but no connection. The proxmox bridge is VLAN aware.

Am I missing something?

 

[bobmc]

Are both hypervisors on the same physical switch? if not, all of the links which connect the switches also need to be tagged for VLAN 300

 

[martijnk]

Are both hypervisors on the same physical switch? if not, all of the links which connect the switches also need to be tagged for VLAN 300

Not an easy answer. It's a stacked setup, both hypervisors use LACP, one cable to each switch.

 

[bobmc]

should still work if all members of the LAG are tagged for the VLAN

this guide for esxi networking might give you some pointers
https://www.dell.com/downloads/global/power/ps3q08-20080213-Ramos.pdf

 

[martijnk]

Yes I have two lags and they both are in the VLAN.

 

[Whatever]

Any idea? I now have my switch configured like it should, according to Dell. But VLAN isn't working cross machines. I've tagged 2 VMs on VLAN 300 which is configured on the switch but no connection. The proxmox bridge is VLAN aware.

Am I missing something?

We have been facing strange behavior with VLAN tagged connections inside VM with ifupdown2 package been installed on host. After removing this package from PVE and host reboot everything started working as before (and expected)

 

[martijnk]

We have been facing strange behavior with VLAN tagged connections inside VM with ifupdown2 package been installed on host. After removing this package from PVE and host reboot everything started working as before (and expected)

Hmm I wonder what that has to do with it but I'm kinda desperate so I will test it anyway

 

[martijnk]

So I tried everything I could but I'm failing miserably, I don't understand. I even asked on the Dell forums and they gave me clear instructions, yet I still can't do any communication over VLANs between different hosts. Same host always works.

So here is my switch config, LACP port channel is configured in hybrid mode. Untagged vlan 1 and tagged vlan 300:

Name: fortyGigE 2/2 (Port-channel 2)
802.1QTagged: Hybrid
Vlan membership:
Q       Vlans
U       1
T       300

Port-channel config on the switch:

!
interface Port-channel 2
 no ip address
 mtu 9216
 portmode hybrid
 switchport
 lacp long-timeout
 no shutdown

On proxmox, I have vmbr3 which is a bridge to the LACP bond of both port-channel members. The switch has VLAN-aware checked. I've created two VMs and tagged both network interfaces with VLAN 300.

Yet no communication. I checked the switch logs and I don't even see any traffic on VLAN 300, just 1.

 

[RobFantini]

you've got to learn how to set these up on your dell switch: vlan T vlan U and pvid. make sure you have an understanding of those 3 pieces of data.
try learning those by checking some youtube vlan videos like https://www.youtube.com/watch?v=plrGKoMCJdM




vlan U on switch: - generally there should only be at the most 2 per port, preferably one. we use the switch default 1U and the vlan number for proxmox [ for us U 10] . we have 7 proxmox hosts attacted to our switch, at each port vlan U is set for 1 and 10 ]

vlan T on switch : at the ports used for proxmox we have multiple vlan T for all the vlans we use. so here is an example of a port

at pve for vm set the vlan at hardware > network device.

to test use ping from one vm to the other.


usually there is a GUI on the switch to set VLAN U , T and PVID .

for reference here are switch settings for 2 ports on our main switch:

interface 1/0/31
auto-voip protocol-based
mtu 9216
vlan pvid 10
vlan participation auto 1
vlan participation include 3,7-8,10,15,20,70
vlan tagging 3,7-8,15,20,70
exit


interface 1/0/32
auto-voip protocol-based
mtu 9216
vlan pvid 10
vlan participation auto 1
vlan participation include 3,7-8,10,15,20,70
vlan tagging 3,7-8,15,20,70
exit

 

[martijnk]

you've got to learn how to set these up on your dell switch: vlan T vlan U and pvid. make sure you have an understanding of those 3 pieces of data.
try learning those by checking some youtube vlan videos like https://www.youtube.com/watch?v=plrGKoMCJdM




vlan U on switch: - generally there should only be at the most 2 per port, preferably one. we use the switch default 1U and the vlan number for proxmox [ for us U 10] . we have 7 proxmox hosts attacted to our switch, at each port vlan U is set for 1 and 10 ]

vlan T on switch : at the ports used for proxmox we have multiple vlan T for all the vlans we use. so here is an example of a port

at pve for vm set the vlan at hardware > network device.

to test use ping from one vm to the other.


usually there is a GUI on the switch to set VLAN U , T and PVID .

for reference here are switch settings for 2 ports on our main switch:

interface 1/0/31
auto-voip protocol-based
mtu 9216
vlan pvid 10
vlan participation auto 1
vlan participation include 3,7-8,10,15,20,70
vlan tagging 3,7-8,15,20,70
exit


interface 1/0/32
auto-voip protocol-based
mtu 9216
vlan pvid 10
vlan participation auto 1
vlan participation include 3,7-8,10,15,20,70
vlan tagging 3,7-8,15,20,70
exit

Thanks for the response! This is a S6010-ON switch, which doesn't have a Web GUI nor does it have a PVID option.

I do have a powerconnect switch as well, N2024 but that's limited to 1 gigabit, while the S6010-ON is 40 so I would rather use that for LAN. But I will test your settings on the N2024, maybe at least I'll get a working situation and can then try further on the S6010-ON.

Thx!

 

[martijnk]

Ok now I'm really confused. I was always testing on VLAN300, then for fun I switched to VLAN 100 and suddenly it worked! Beats the hell out of me why. Then I went back to 300, it stopped working. Then I did 299, nope nothing. Then I switched to VLAN2 and it worked again.....