PVE not assigning VM vlans, please help

[LooneyTunes]

Hi,
I had to reinstall PVE due to a failing disk and find myself in a predicament. I have only partial documentation of the network setup, and that is at best outdated.

My goal is to have static IP on PVE, and vlan's made available to VM's. Config below failed to load the interface even. Where did I go wrong?

This is my /etc/network/interfaces

auto lo
iface lo inet loopback

iface eno1 inet manual

auto vmbr0.2
iface vmbr0.2 inet static
        address 10.10.10.2/24
        gateway 10.10.10.1

auto vmbr0
iface vmbr0 inet static
        bridge-ports eno1
        bridge-stp off
        bridge-fd 0
        bridge-vlan-aware yes
        bridge-vids 2-4094

 

[remark]

Why do you need vmbr0.2 interface?

 

[LooneyTunes]

Why do you need vmbr0.2 interface?

Very good question. I thought that was PVE's IP, and the section below used by VM's...?

After commenting that out, PVE boots to login and it has connectivity, but VMs can still not get their IP's using the VM-Nic config

 

[remark]

Very good question. I thought that was PVE's IP, and the section below used by VM's...?

Just assign ip address that you will use to access PVE to vmbr0.
Then add network interface to your VM and specify VLAN tag. In VM, you will get one untagged network interface per VLAN.

 

[remark]

This is my working example of pfSense router working in PVE environment.
Tagged traffic comes from switch to PVE on vmbr0 bridge.

This is VM configuration:


This picture shows how interfaces are visible inside VM:


So, VM can access VLANs without playing VLAN tags inside VM.

 

[LooneyTunes]

Thanks, what happens for me now is unfortunately the same as before, PVE gets it's intended IP, but any started VM's just time-out when it's time for network configuration;


I even removed the firewall I had on previously, to exactly mimic your config... Still no go...

This is what is in /etc/network/interfaces now

auto lo
iface lo inet loopback

iface eno1 inet manual

auto vmbr0
iface vmbr0 inet static
        address 192.168.2.2/28
        gateway 192.168.2.1
        bridge-ports eno1
        bridge-stp off
        bridge-fd 0
        bridge-vlan-aware yes
        bridge-vids 2-4094
root@pve:~#

 

[remark]

Is your DNS server up and running?

 

[LooneyTunes]

Is your DNS server up and running?

Yes, and working well. I have also checked the firewall, but nothing has changed since before the incident, so should not be that...

 

[remark]

VM on the screenshot you posted is just hangs during bootup? Or it booting to login?

 

[LooneyTunes]

VM on the screenshot you posted is just hangs during bootup? Or it booting to login?

It boots to login eventually, without an IP though

 

[remark]

It boots to login eventually, without an IP though

Is IP static or acquired from DHCP?

 

[LooneyTunes]

Is IP static or acquired from DHCP?

reserved DHCP, so shoud get the same one every time

 

[remark]

VLAN tag 2, are you sure you have tagged traffic on vmbr0?

 

[LooneyTunes]

VLAN tag 2, are you sure you have tagged traffic on vmbr0?

No, I can't say that I am. That is what all this is about I suppose. I had this working before, and kept too bad documentation of how it was setup unfortunately... :/ That is why I was messing with tags in /etc/network/interfaces before...

 

[remark]

No, I can't say that I am. That is what all this is about I suppose. I had this working before, and kept too bad documentation of how it was setup unfortunately... :/

Then try to see it.
tcpdump -i eth0 -e
Replace eth0 to your physical network interface. Wait for a while, then press Ctrl+C. If you have vlan tagged traffic, you should see something like that:

 

[LooneyTunes]

Then try to see it.
tcpdump -i eth0 -e
Replace eth0 to your physical network interface. Wait for a while, then press Ctrl+C. If you have vlan tagged traffic, you should see something like that:
View attachment 49490

Running on PVE I see no traces of vlan-tags, so this was a good finding! Which would explain why it refuses to use one... Question now then is how to add them to vmbr0, right?

 

[remark]

Should this be run on PVE or the client VM?

Run it on PVE. Maybe you have to install tcpdump: apt-get install tcpdump

 

[LooneyTunes]

Run it on PVE. Maybe you have to install tcpdump: apt-get install tcpdump

I did, and no vlan-tags unfortunately

 

[remark]

I did, and no vlan-tags unfortunately

Then I need more information about your network setup, and, maybe you want to show documentation you have.

 

[LooneyTunes]

Now _this_ is quite strange... I ran it on my client as well, and guess what. No vlan-tags there either...!

I changed my router some time ago and have not reconfigured the VM or PVE NICs while doing so... Somehow it seems I am not running a tagged network any more?! huh... So this must be a router config issue if anything. I will have to look into that before continuing with this... WoW. Had no idea. I'm very sorry, but learned about tcpdump so far! I'll have some reading to do then I suppose

Just strange PVE or the VM's haven't complained about this earlier, it is almost a year ago now...