Proxmox trunk with palo alto sub interfaces

[bughatti]

Hoping someone can help me with this, I have been struggling for a couple days now. I have proxmox 8.1.4 setup with 2 x 1 gb ethernet ports connected to a cisco switch and 2 x 10gb dac ports connected to a brocade. 1 10gb port is set as a trunk on the brocade which is connected to enp8s0f0 in proxmox. I have vmbr0 setup on eno1 which is the 1gb port for management. I have created multiple vlanxxx ports that use enp8s0f0 as the raw device and are tagged with xxx and work perfectly fine, these ports are untagged in palo alto which makes sense and ping works perfectly fine. The main issue I am running into is the palo alto we plan to move from vmware uses ethernet1/5 with multiple sub interfaces. What I tested was I created vmbr1 with enp8s0f0 as the raw device and tried both selections of vlan aware, checked and unchecked. In the new palo alto I am testing, I have tried layer 2 vlans sub interfaces and layer 3 sub interfaces. Assigned an ip, made sure interface management has a ping set on those interfaces, but I am getting no pings at all. I guess my overall question is how to I assign a trunk port properly in proxmox so that the palo alto can utilize sub interfaces

brocade

interface TenGigabitEthernet 4/0/19
 cee default
 mtu 9208
 description proxmox-mgmt
 no fabric isl enable
 fabric trunk enable
 switchport
 switchport mode trunk
 switchport trunk allowed vlan all
 switchport trunk tag native-vlan
 spanning-tree shutdown
 no shutdown

auto lo
iface lo inet loopback

iface eno1 inet manual

iface eno2 inet manual

auto enp8s0f0
iface enp8s0f0 inet manual
        mtu 9000

iface enp8s0f1 inet manual

auto vmbr0
iface vmbr0 inet static
        address 10.0.0.25/24
        gateway 10.0.0.2
        bridge-ports eno1
        bridge-stp off
        bridge-fd 0

auto vmbr109
iface vmbr109 inet manual
        bridge-ports vlan109
        bridge-stp off
        bridge-fd 0

auto vmbr1
iface vmbr1 inet manual
        bridge-ports enp8s0f0
        bridge-stp off
        bridge-fd 0
        bridge-vlan-aware yes
        bridge-vids 2-4094

auto vmbr204
iface vmbr204 inet manual
        bridge-ports vlan204
        bridge-stp off
        bridge-fd 0

auto vlan109
iface vlan109 inet manual
        vlan-raw-device enp8s0f0

auto vlan400
iface vlan400 inet manual
        vlan-raw-device enp8s0f0

auto vlan204
iface vlan204 inet manual
        vlan-raw-device enp8s0f0

auto vlan2700
iface vlan2700 inet manual
        vlan-raw-device enp8s0f0

auto vlan117
iface vlan117 inet manual
        vlan-raw-device enp8s0f0

auto vlan205
iface vlan205 inet manual
        vlan-raw-device enp8s0f0

auto vlan1434
iface vlan1434 inet manual
        vlan-raw-device enp8s0f0

auto vlan1436
iface vlan1436 inet manual
        vlan-raw-device enp8s0f0

Palo Alto

admin@PA-VM> show interface all

total configured hardware interfaces: 5

name                    id    speed/duplex/state            mac address
--------------------------------------------------------------------------------
ethernet1/1             16    ukn/ukn/up                    bc:24:11:68:76:07  
ethernet1/2             17    ukn/ukn/up                    bc:24:11:dd:42:a3
ethernet1/3             18    ukn/ukn/up                    bc:24:11:9b:69:09
ethernet1/5             20    ukn/ukn/up                    bc:24:11:1e:f0:56
vlan                    1     [n/a]/[n/a]/up                ba:db:ee:fb:ad:01

aggregation groups: 0


total configured logical interfaces: 7

name                id    vsys zone             forwarding               tag    address
------------------- ----- ---- ---------------- ------------------------ ------ ------------------
ethernet1/1         16    1                     N/A                      0      N/A
ethernet1/2         17    1    PDM-Office       vr:default               0      10.0.0.27/24          ***Works fine***
ethernet1/3         18    1    PDM-Servers      vr:default               0      192.168.1.30/24          *** Works fine***
ethernet1/5         20    1                     vr:default               0      N/A
ethernet1/5.117     258   1    PDM-Voip         vr:default               117    192.168.117.30/24      ***No ping at all, doesn't even show trying to hit PA in monitor tab***
vlan                1     1                     N/A                      0      N/A
vlan.111            257   1    PDM-DMZ          vr:default               0      192.168.111.30/24

 

[bughatti]

I find it hard to believe no one else has tried to trunk to a router and setup sub interfaces, even for a pfsense. It appears that the mac address for the proxmox vmbr1 interface is making it across the trunk but the palo altos interface mac for that trunk is not.

tcpdump on proxmox

root@mgmt:~# tcpdump -envi vmbr1 -e '(vlan 205)'
tcpdump: listening on vmbr1, link-type EN10MB (Ethernet), snapshot length 262144 bytes
15:03:53.943810 ba:db:ee:fb:ad:01 > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 60: vlan 205, p 0, ethertype ARP (0x0806), Ethernet (len 6), IPv4 (len 4), Request who-has 192.168.205.1 (ff:ff:ff:ff:ff:ff) tell 192.168.205.30, length 42
15:03:54.972816 ba:db:ee:fb:ad:01 > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 60: vlan 205, p 0, ethertype ARP (0x0806), Ethernet (len 6), IPv4 (len 4), Request who-has 192.168.205.1 (ff:ff:ff:ff:ff:ff) tell 192.168.205.30, length 42

Ping on Palo Alto

admin@PA-VM> ping source 192.168.205.30 host 192.168.205.1
PING 192.168.205.1 (192.168.205.1) from 192.168.205.30 : 56(84) bytes of data

Mac lookup on brocade

VMWare04# show mac-address-table interface tengigabitethernet 4/0/19
VlanId   Mac-address       Type     State        Ports
109      bc24.11cb.b530    Dynamic  Active       Te 4/0/19
109      bc24.11dd.42a3    Dynamic  Active       Te 4/0/19
205      badb.eefb.ad01    Dynamic  Active       Te 4/0/19
1436     90e2.ba80.ef60    Dynamic  Active       Te 4/0/19
Total MAC addresses    :  4

 

[bughatti]

It would be nice if a PVE support member would pop in here and at least say this can only be handled with a support ticket, seems dumb to pay for a support ticket and then be told its not possible!!

 

[bughatti]

So I took the the 10gb dac card and put it in pci passthru directly to the palo alto and low and behold, the sub interfaces in palo alto work. Not trying to come across as slamming proxmox, but food for thought for other users dealing with the broadcom/vmware issues and finding a solution, proxmox is not as enterprise ready as thought. Virtual routers has been around for a while, putting them on trunks in a type 1 hypervisor is not new and works perfectly fine in many other situations, it should work here but does not.

 

[ce3rd]

hi, I have this with a Fortigate VM working behind a HP switch -> vmbr1 vlan aware on interface of pve host (no need for IP) -> give it to fortigate vm -> create tagged vlan subinterfaces on fortigate vm - ok... ;-) I don´t know Palo Alto VMs, maybe it works the same way? I aslo have a Mikrotik CHR running like this and a pfsense... or you can tag the networkinterface via Proxmox, when you create the VM with it's interafaces on top of vmbr1...