Troubleshooting VLAN Configuration: VMs in VLAN 25 Unable to Obtain IP Address

[empbilly]

hello,

All network configurations, including VLAN settings, are correctly implemented both on the server and the network switch. The VMs in VLAN 25 have their network settings correct, including IP address, subnet mask, and gateway. However, the VMs still fail to obtain an IP.

switch configuration:



pve3 network configuration:



What additional diagnostic steps can be taken to resolve this issue?

 

[empbilly]

Any thoughts?

 

[louie1961]

Where is your DHCP server? On my setup, pfsense provides the DHCP server, not my switch or Proxmox.

 

[empbilly]

Where is your DHCP server? On my setup, pfsense provides the DHCP server, not my switch or Proxmox.

hi, We also use pfsense's dhcp.

 

[MrPete]

WIreshark is your friend. And tcpdump within a VM. And learn how to do live remote (ssh-based) tcpdump to wireshark as needed. But for this situation where you're not getting the DHCP, just doing tcpdump in pfSense and in your VM ought to tell you a lot.

You're certain the hash policy is correct for your switch? That's what killed me at first.

 

[louie1961]

hi, We also use pfsense's dhcp.

So, do you have DHCP set up on that VLAN interface in pfsense? Also check the range of available IP addresses in that VLAN to make sure there isn't a conflict.

 

[empbilly]

You're certain the hash policy is correct for your switch? That's what killed me at first.

It's a Dell N2048. I think layer2+3 is correct.

So, do you have DHCP set up on that VLAN interface in pfsense? Also check the range of available IP addresses in that VLAN to make sure there isn't a conflict.

Yes, I have dhcp configured on that vlan. In fact, all the other vlans pass and only 25 doesn't.

How do you configure the trunk port of the switch that the pve is connected to?

 

[louie1961]

The same as my trunk ports to the pfsense box: 1UP (untagged, PID), 2T, 3T, 4T etc.

Make sure it is configured as a trunk port, untagged on the primary VLAN ID (the primary VLAN for your switch (usually VLAN 1 on the less expensive switches like mine) and tagged on all the others.

 

[empbilly]

The same as my trunk ports to the pfsense box: 1UP (untagged, PID), 2T, 3T, 4T etc.

Make sure it is configured as a trunk port, untagged on the primary VLAN ID (the primary VLAN for your switch (usually VLAN 1 on the less expensive switches like mine) and tagged on all the others.

See the first image of my initial post. There is the configuration I do on all the uplink ports.

But in my scenario, the management IP and access to the pve webgui is in the same vlan (25) as some VMs. It seems to me that this is interfering, because if I change the VM's vlan to another, for example 26, it gets the IP normally, but in vlan 25, it doesn't.

 

[louie1961]

Honestly I can't make any sense of that picture. Its not an interface I am familiar with. Your connection from your switch to Proxmox needs to be VLAN agnostic. If you have the switch port(s) tied to any specific VLAN, then your interface in proxmox won't work correctly.

 

[empbilly]

Honestly I can't make any sense of that picture. Its not an interface I am familiar with. Your connection from your switch to Proxmox needs to be VLAN agnostic. If you have the switch port(s) tied to any specific VLAN, then your interface in proxmox won't work correctly.

The image describes a status of four interfaces that are part of a link aggregation group (LAG) named "LAG-PVE3-R720" and the running configuration of that LAG interface. The interfaces are in a full and operational state, and the LAG interface is configured to carry VLANs 1 to 2000 and operate in trunk mode.

A Port-channel is a logical interface that combines multiple physical interfaces to increase bandwidth and provide redundancy. It is also known as Link Aggregation Group (LAG) or Etherchannel. It uses protocols such as Link Aggregation Control Protocol (LACP) or Port Aggregation Protocol (PAgP) to negotiate and form the port-channel.

In this case, the port-channel interface "LAG-PVE3-R720" is composed of four physical interfaces Gi1/0/2, Gi1/0/8, Gi1/0/9, and Gi1/0/20. The switch treats this port-channel as a single interface and provides load balancing, redundancy and increased bandwidth.

 

[empbilly]

@louie1961

Do you configure the uplink port between the switch and the proxmox in trunk or general mode?

 

[MrPete]

1) Is pfSense on your Proxmox server or elsewhere?
2) You're certain the link to the switch is 100% fine? If not, I would temporarily simplify by backing off from LACP/LAG group. Keep it a simple one-connection link until you have this solved. Fewer variables.

3) Here's how I have trunk + VLAN-for-VM set up:

First, my LACP bond. NOTE: I have never succeeded in attaining VLAN smarts on the bond itself.
Mine connects to a Netgear (not-so) "smart switch", which only supports layer2. I tried to look up the hash policies for yours. It's quite complex; not sure which policy would be correct in either direction! Make sure you use a policy that linux can understand!

auto bond0
iface bond0 inet manual
bond-slaves ens4f1 ens4f2
bond-miimon 100
bond-mode 802.3ad
bond-xmit-hash-policy layer2
#LAN LAG

Then in the host, a trunk for pfSense to my switch. Not sure it matters but in my case pfSense is in a host VM.
NOTE that I had to define bridge-pvid for everything to work correctly. That identifies the VLAN that will be untagged..

auto vmbr2
iface vmbr2 inet manual
bridge-ports bond0
bridge-stp off
bridge-fd 0
bridge-vlan-aware yes
bridge-vids 2-4094
bridge-pvid 71
#LACP LAG 4 LAN

Then in a specific VM configuration (/etc/pve/nodes/<node>/qemu-server/<nnn>.conf):

net1: virtio=<mac>,bridge=vmbr2,tag=9

I can grab as many virtual NICs as I want off of vmbr2. I have a pfSense VM that gets the "raw" vmbr2. I have a complex VM with four virtual NICs each talking to a different VLAN. pfSense supplies DHCP to every VLAN, and every VM correctly obtains IP's via DHCP on any VLAN.

I also created a static virtual nic in the host tapping into a specific VLAN. In this case while I could have used DHCP I didn't because I want the core infrastructure to function even when pfSense/DHCP is down:

auto vmbr2.77
iface vmbr2.77 inet static
address 192.168.7.236/24
#host diagnostic port on .7

4) If you need to debug LACP, this person did some intense debugging. When you get there this might be informative...
https://tchien.com/technical-ramblings/2021/4/8/proxmox-and-unifi-lacp-vlans-and-bridges

I'm away for a bit now. I hope this is helpful!

 

[louie1961]

@louie1961

Do you configure the uplink port between the switch and the proxmox in trunk or general mode?

Trunk