[SOLVED] Trunking VLANs to a VM (pfSense)

[unassassinable]

Looks like I am trying to create a similar setup many before have successfully implemented and I am only adding to the pool of confused admins...

I am trying to trunk several VLANs to a pfSense VM over a physical interface (eno3). Here I the steps:

1. Configure Trunking on switch (Cisco Catalyst 3560) without a native VLAN (I don't need anything untagged)
   

   interface GigabitEthernet0/3
    switchport trunk encapsulation dot1q
    switchport trunk allowed vlan 100,300,400
    switchport mode trunk
   !

2. Plug Proxmox interface eno3 into port 0/3
3. Create a bridge
   1. Name: vmbr2
   2. autostart: checked
   3. VLAN aware: checked
   4. Bridge port: eno3
4. Create VLAN 100
   1. Name: vmbr2.100
   2. IPv4/CIDR: 10.1.0.98/24
   3. autostart: checked
   4. Vlan raw device: vmbr2
   5. VLAN tag: 100
5. On my pfSense VM, I create a NIC (vtnet1)
   1. Bridge: vmbr2
   2. Model: VirtIO
   3. VLAN Tag: 100
   4. Firewall: checked and unchecked (both not working)
6. In pfSense I do the following:
   1. press 1 to assign interfaces
   2. should VLANs be setup first: y
   3. enter parent interface name for the new VLAN: vtnet1 (vtnet0 is WAN, and that is working)
   4. enter VLAN tag: 100
   5. Enter LAN interface name: vtnet1.100
      
   6. It then completes:
      

Notice I set an IP address of 10.1.0.8 on the PVE host VLAN 100. From the PVE shell I can ping any other physical hosts on VLAN 100 through the switch. This tells me the switch is configured correctly. On the pfSense VM, I cannot ping anything including the PVE ip address (10.1.0.8). I do not know if the trouble is on the PVE host side, or the pfSense side.


Here is the relevant sections of my /etc/network/interfaces:

auto eno3
iface eno3 inet manual

auto vmbr2
iface vmbr2 inet manual
        bridge-ports eno3
        bridge-stp off
        bridge-fd 0
        bridge-vlan-aware yes
        bridge-vids 2-4094
#Server traffic bridge

auto vmbr2.100
iface vmbr2.100 inet static
        address 10.1.0.98/24
#VLAN 100

Is there anything else I can provide to help us troubleshoot this?

 

[unassassinable]

Further information, though not sure it's entirely helpful. When I run tcpdump on the PVE host filtering only to VLAN 100, and attempt to ping from the pfSense VM, I can see arp requests coming from pfSense:




I pinged a bunch of hosts (including the pfSense box from the switch, and viewed the resulting arp table:

COBRA-SWI-EP02#ping 10.1.0.8
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.1.0.8, timeout is 2 seconds:
.....
Success rate is 0 percent (0/5)
COBRA-SWI-EP02#show arp
Protocol  Address          Age (min)  Hardware Addr   Type   Interface
Internet  10.1.0.2                0   0024.50f5.26c3  ARPA   Vlan100
Internet  10.1.0.3                -   001e.7983.4e41  ARPA   Vlan100
Internet  10.1.0.8                0   Incomplete      ARPA
Internet  10.1.0.98               0   a4ba.db41.32ec  ARPA   Vlan100
Internet  10.1.0.99               2   a4ba.db41.3b4b  ARPA   Vlan100

 

[unassassinable]

FYI, I have successfully done the following, and I guess this could work, though it is not my first choice. I would still like to trunk up to pfSense and Tag at pfSense, which would save me from having to create possibly dozens of virtual NICs on pfSense...

VM NIC - attached to vmbr100
vmbr100 - virtual bridge using bridge port vmbr2.100
vmbr2.100 - VLAN 100 uses raw device vmbr2
vmbr2 - virtual bridge connected to eno3
eno3 - Physical interface

 

[unassassinable]

Ok, I found my problem. I was mistakenly tagging a specific VLAN on the VM's NIC. This should have been done at the VM level. I have fixed this and it works.