multiple nics not working

[meme111]

Hi, i wasn't sure how to explain this.. but essentially, I have a proxmox machine with 5 NICs
eno1 -> access port (vlan50)
enp1s0f0 ->
enp1s0f1 ->
enp1s0f2 -> trunk
enp1s0f3 -> access port (vlan 20)

No matter what I try, I just cannot get any interface to work except for eno1. I created a bridge for each NIC, added all of them to a VM, but they don't get an ip address.

The device inside the VM reports

ens19: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet6 fe80::708c:80ff:fe32:6dd9  prefixlen 64  scopeid 0x20<link>
        ether 72:8c:80:32:6d:d9  txqueuelen 1000  (Ethernet)
        RX packets 855  bytes 52335 (52.3 KB)
        RX errors 0  dropped 367  overruns 0  frame 0
        TX packets 21  bytes 1662 (1.6 KB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

My network interfaces config is currently as follows-

auto lo
iface lo inet loopback

iface eno1 inet manual
#mboard

iface enp1s0f0 inet manual
#DMZ

iface enp1s0f1 inet manual
#SERVERS

iface enp1s0f3 inet manual
#HOME

iface enp1s0f2 inet manual

auto enp1s0f2.70
iface enp1s0f2.70 inet manual
        vlan-id 70

auto vmbr0
iface vmbr0 inet static
        address 192.168.50.9/24
        gateway 192.168.50.1
        bridge-ports eno1
        bridge-stp off
        bridge-fd 0

auto vmbr2
iface vmbr2 inet manual
        bridge-ports enp1s0f1
        bridge-stp off
        bridge-fd 0
#PORT 2 VLAN50

auto vmbr1
iface vmbr1 inet manual
        bridge-ports enp1s0f0
        bridge-stp off
        bridge-fd 0
#PORT1 DMZ

auto vmbr4
iface vmbr4 inet manual
        bridge-ports enp1s0f3
        bridge-stp off
        bridge-fd 0
#HOME LAN

auto vmbr3
iface vmbr3 inet manual
        bridge-ports enp1s0f2
        bridge-stp off
        bridge-fd 0
        bridge-vlan-aware yes
        bridge-vids 2-4094
#TRUNK

I've tried bridges and ovs. but just can't seem to figure this out. Have I done something wrong here?

 

[meme111]

Directly passing enp1s0f3 (pci passthru, connected to a vlan20 access port) to a guest VM successfully returns an ip address on the .20 network.

So these interfaces aren't working at the proxmox level but they do work perfectly fine inside a VM - the switch and cabling is all fine.

Problem is, I do need multi-vlan support for other VMs so can't just solve this with passthru.

Can somebody help, please?

 

[meme111]

One step forward, 2 steps back.
I am using 1x intel NIC on the motherboard, and 1x quad Intel NIC via PCI.

Now that i've passed through 2 of the quads NICS to a VM (2 passed through, 2 remaining)... I no longer see any of the NICS on pve command line, but I do see them via the proxmox GUI.

Trying to create a new bridge for enp1s0f0 results in:

error: netlink: enp1s0f0: cannot enslave link enp1s0f0 to vmbr1: operation failed with 'No such device' (19)
error: vmbr1: bridge port enp1s0f0 does not exist

Any help greatly appreciated... I'm close to admitting defeat and returning back to ESX

 

[meme111]

Removing the pass-through devices and rebooting now allows me to see the interfaces at the pve level.
I created a bridge on the trunked interface... added it to the VM, and this time it works. That's the exact same thing I've been doing all day and week.. so on one hand i'm happy that it's working, on the other hand this doesn't feel very reliable