Proxmox on management VLAN but seems to be responding with VM VLAN in pings etc

[PMX151]

I have the following in /etc/network/interfaces

auto lo
iface lo inet loopback
# Management vLAN 200 - 172.16.200.0/24

iface enp3s0f0 inet manual
#1 PVE management port

auto enp3s0f0.200
    iface enp3s0f0.200 inet dhcp
    address 172.16.200.202/24
    gateway 172.16.200.1

iface enp3s0f1 inet manual
iface enp4s0f0 inet manual
iface enp4s0f1 inet manual

auto vmbr0
    iface vmbr0 inet dhcp
    bridge-ports enp3s0f1
    bridge-ports enp4s0f0
    bridge-ports enp4s0f1
    bridge-stp off
    bridge-fd 0

source /etc/network/interfaces.d/*

I can connect to Proxmox, so the VLAN seems to be functioning but Proxmox doesn't seem to be able to connect to the internet. I can ping the router and the router can ping Proxmox, plus I can connect across VLANs (no firewalling currently) so the VLAN should also have free access across the LAN (and by extension out of the firewall too). Running something like nslookup www.google.co.uk and I see;

;; communications error to 192.168.50.2#53: timed out
;; communications error to 192.168.50.2#53: timed out
;; communications error to 192.168.50.2#53: timed out
;; no servers could be reached

but 192.168.50.0/24 is a network assigned to a VLAN on a VM so the host shouldn't see this? I feel like there's something missing here or I've got the config wrong, or I totally misunderstand the errors. I tried adding the management port to a bridge so I could make it "VLAN Aware" but that made no difference.

Additionally, and probably related, in the DNS section of the settings I manually set it to 172.16.200.1 and it keeps picking up 192.168.99.2 from somewhere, but even when it is set correctly it doesn't seem to fix the issue completely. nslookup works but then internally Proxmox still has no internet, so it won't download templates for instance.

 

[ggoller]

Hi!
your bridge is not vlan aware. Try adding this config to the bridge:

    bridge-vlan-aware yes
    bridge-vids 2-4094

Could you show me the config of your VM? Also make sure that enp3s0f1, enp4s0f0 and enp4s0f1 are not attached to the same switch, otherwise there will be a loop.

 

[PMX151]

I did that as part of the testing, and every combination of every bridge/interface setting.

I can't tell if this is entirely a Proxmox issue but even after setting the dns-nameserver property in /etc/network/interfaces it still grabs the wrong address.

I've also removed the VLAN part of the puzzle and just untagged all the ports on the router. Next step I guess is just remove the VLAN entirely at this point, pretty sure that is confusing the issue. Just can't work out why it sort of works but then doesn't but then the cluster is healthy and online but none of the nodes can talk to each other (as in, via the GUI) but they must be able to otherwise the cluster wouldn't be healthy.

 

[ggoller]

Check my edited answer above. For the DNS issue, make sure that you have dns-nameserver in /etc/network/interfaces and nameserver x.x.x.x in /etc/resolv.conf !

 

[PMX151]

make sure that enp3s0f1, enp4s0f0 and enp4s0f1 are not attached to the same switch, otherwise there will be a loop.

I thought that was the point of the bridge? They are all connected to the same switch/network/VLAN.

make sure that you have dns-nameserver in /etc/network/interfaces and nameserver x.x.x.x in /etc/resolv.conf !

I do, but it changes back to one of the random DNS addresses it picks up from elsewhere on the network. Changing the DNS server directly or via the GUI seems to make no difference, after a short time (minute or so) it usually changes away from the assigned address. One of the addresses it picks up is a VLAN assigned network on a VM so nothing should be getting an address on that anyway so it can only be an internal Proxmox error here as nothing outside of Proxmox can see that DHCP/DNS server.

Could you show me the config of your VM?

Which bit specifically? Is there a command line option here or do I have to screenshot bits of the GUI? These are the 2 networks on the Technitium DNS/DHCP server that the host keeps getting an address from.



Also confused as to how the second host is getting addresses on this VLAN, assume because it is clustered it sharing info from the lead node? Neither should be able to see the VLAN from what I understand.

 

[PMX151]

As this is a lab I have backed up a few steps and got rid of the cluster. I have also changed the network config to static just to rule out any issues there. After setting the correct IP in resolv.conf after a few minutes it is changed to the DNS server handed out by the DHCP server. I've also got rid of VLANs on the network and undone the bridge "VLAN Aware" setting. This feels like an issue within Proxmox now.

/etc/network/interfaces is now just;

auto lo
iface lo inet loopback

auto enp3s0f0
iface enp3s0f0 inet static

#1 PVE management port
    address 172.16.200.203/24
    gateway 172.16.200.1
    dns-nameservers 172.16.200.1

iface enp3s0f1 inet manual
#2 VM bridge port 1
iface enp4s0f0 inet manual
#3 VM bridge port 2
iface enp4s0f1 inet manual
#4 VM bridge port 3

auto vmbr0
iface vmbr0 inet manual
        bridge-ports enp3s0f1 enp4s0f0 enp4s0f1
        bridge-stp off
        bridge-fd 0

source /etc/network/interfaces.d/*

The DNS server should not be getting changed unless there is something I have missed?