Cisco switch - Proxmox VLANS - cant get it work

[papatikka]

I have been troubleshooting for a long time and I feel I get more confused the deeper I try to troubleshoot.

my pve node resides in vlan 20, which is tagged.
My PVE can NOT ping gateway 192.168.20.1

A container in vlan 1 (management), can ping its gateway. This vlan is untagged

A container in vlan 30, can ping other devices in same vlan, however cant ping its own gateway. ALSO internetaccess and being able to ping other devices in same vlan ONLY works if I MANUALLY add ip route default gateway.

default via 192.168.30.1 dev eth0
192.168.30.0/24 dev eth0 proto kernel scope link src 192.168.30.9

I run this network: internet - opnsense fw - cisco switch - proxmox ve

opnsense vlans etc is working, nothing more I can do there.

Switch is configured as LAG to Proxmox VE and the config is following:

Spoiler: long code section

Port     Type         Duplex  Speed Neg      ctrl State       Pressure Mode
-------- ------------ ------  ----- -------- ---- ----------- -------- -------
gi1      1G-Copper    Full    1000  Enabled  Off  Up          Disabled Off
gi2      1G-Copper    Full    1000  Enabled  Off  Up          Disabled On

Ch       Type    Duplex  Speed  Neg      control  State
-------- ------- ------  -----  -------- -------  -----------
Po1      1G      Full    1000   Enabled  Off      Up

interface Port-Channel1
 description pve01-lag
 switchport mode trunk
 switchport trunk allowed vlan 1,10,20,30,40,90
!

Port-Channel Po1
       Port Type Gigabit Ethernet
       Attached Lag id:
       Actor
               System Priority:1
               MAC Address:    XXXXXXXXXX
               Admin Key:      1000
               Oper Key:       1000
       Partner
               System Priority:65535
               MAC Address:    XXXXXXXXXX
               Oper Key:       9
             
interface GigabitEthernet1
 description "pve01 LAG trunk"
 channel-group 1 mode auto
 switchport mode trunk
 switchport trunk allowed vlan 1,10,20,30,40,90
!
             
interface GigabitEthernet2
 description "pve01 LAG trunk"
 channel-group 1 mode auto
 switchport mode trunk
 switchport trunk allowed vlan 1,10,20,30,40,90
!

Allright, so far it looks good? The only difference that I can see is under Port Vlan Membership is this


There is no operational VLANs, is it necessary?

My PVE /etc/network/interfaces config is following:

Spoiler: long code section

# network interface settings; autogenerated
# Please do NOT modify this file directly, unless you know what
# you're doing.
#
# If you want to manage parts of the network configuration manually,
# please utilize the 'source' or 'source-directory' directives to do
# so.
# PVE will preserve these directives, but will NOT read its network
# configuration from sourced files, so do not attempt to move any of
# the PVE managed interfaces into external files!

auto lo
iface lo inet loopback

auto enp3s0
iface enp3s0 inet manual

auto eno1
iface eno1 inet manual

auto bond0
iface bond0 inet manual
        bond-slaves eno1 enp3s0
        bond-miimon 100
        bond-mode 802.3ad
        bond-xmit-hash-policy layer3+4
#Dual 1G LACP

auto vmbr0
iface vmbr0 inet manual
        bridge-ports bond0
        bridge-stp off
        bridge-fd 0
        bridge-vlan-aware yes
        bridge-vids 10 20 30 40 90

auto vmbr0.20
iface vmbr0.20 inet static
        address 192.168.20.5/24
        gateway 192.168.20.1
#DC - tagged

auto vmbr0.1
iface vmbr0.1 inet static
        address 192.168.1.0/24
#MANAGEMENT - untagged

auto vmbr0.10
iface vmbr0.10 inet static
        address 192.168.10.0/24
#DEVICE - tagged

auto vmbr0.30
iface vmbr0.30 inet static
        address 192.168.30.0/24
#       gateway 192.168.30.1
#IOT - tagged

auto vmbr0.90
iface vmbr0.90 inet static
        address 192.168.90.0/24
#GUEST - tagged

auto vmbr0.40
iface vmbr0.40 inet static
        address 192.168.40.0/24
#KIDS - tagged

source /etc/network/interfaces.d/*

What/where I am I doing wrong? I have followed every guide and still cant get it to work.

Is it WORKING as intended that pve machines on tagged vlans running nslookup will go by the gateway set up on the host and not by the gateway of the vlan?
I start to wonder if I actually built everything functioning and working as intended?