[SOLVED] Configuring VLANs in Proxmox with OPNsense Router

E

eth10747

New Member
Nov 11, 2023
2
1
3
Hello! I am not sure if I have followed the instructions on various sites properly but here is my setup, and the items I have done. I do want to note that my proxmox server is in a cluster with 2 other nodes (3 nodes total, and they are on 10.0.0.0/25 subnet at the moment with no VLAN attached yet)

Quick Summary of network
Hardware
Router: OPNSense on Zimaboard
Proxmox Server: PVE02 with 1 NIC.
Switch: TP TL-SG1024DE

IP Configuration
OPNSense gateway: 10.0.0.1/8
Proxmox Node (PVE02): 10.0.0.11/25
Gateway for VLAN 30: 10.0.0.193/26

My Subnet and VLANs configuration (The VLANs are not configured on anything yet as I am trying to get VLAN 30 to work with a VM before I move forward with the rest.)
SubnetSubnet MaskStartingGatewayEndingVLAN ID
10.0.0.0/25255.255.255.12810.0.0.010.0.0.210.0.0.12710
10.0.0.128/26255.255.255.19210.0.0.12810.0.0.12910.0.0.19120
10.0.0.192/26255.255.255.19210.0.0.19210.0.0.19310.0.0.25430
10.0.1.0/24255.255.255.010.0.1.010.0.1.110.0.1.25440

Proxmox Settings/Configuration
Network Interfaces configuration
Code: 
auto lo
iface lo inet loopback

iface enp34s0 inet manual

auto vmbr0
iface vmbr0 inet static
        address 10.0.0.12/25
        gateway 10.0.0.1
        bridge-ports enp34s0
        bridge-stp off
        bridge-fd 0
        bridge-vlan-aware yes
        bridge-vids 2-4094

auto vmbr0.30
iface vmbr0.30 inet static
        address 10.0.0.192/26
        gateway 10.0.0.193

IP A command
Code: 
root@pve02:~# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host noprefixroute
       valid_lft forever preferred_lft forever
2: enp34s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master vmbr0 state UP group default qlen 1000
    link/ether 00:d8:61:3a:02:6f brd ff:ff:ff:ff:ff:ff
3: vmbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 00:d8:61:3a:02:6f brd ff:ff:ff:ff:ff:ff
    inet 10.0.0.12/25 scope global vmbr0
       valid_lft forever preferred_lft forever
    inet6 fe80::2d8:61ff:fe3a:26f/64 scope link
       valid_lft forever preferred_lft forever
4: vmbr0.30@vmbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 00:d8:61:3a:02:6f brd ff:ff:ff:ff:ff:ff
    inet 10.0.0.192/26 scope global vmbr0.30
       valid_lft forever preferred_lft forever
    inet6 fe80::2d8:61ff:fe3a:26f/64 scope link
       valid_lft forever preferred_lft forever
5: tap204i0: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master vmbr0 state UNKNOWN group default qlen 1000
    link/ether 5e:66:79:fe:9a:de brd ff:ff:ff:ff:ff:ff
6: tap206i0: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master fwbr206i0 state UNKNOWN group default qlen 1000
    link/ether da:35:85:b0:c2:30 brd ff:ff:ff:ff:ff:ff
14: fwbr206i0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether ea:3e:d8:bc:9b:ac brd ff:ff:ff:ff:ff:ff
15: fwpr206p0@fwln206i0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master vmbr0 state UP group default qlen 1000
    link/ether d2:cf:e7:51:80:9b brd ff:ff:ff:ff:ff:ff
16: fwln206i0@fwpr206p0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master fwbr206i0 state UP group default qlen 1000
    link/ether 02:8b:56:b5:98:a7 brd ff:ff:ff:ff:ff:ff
17: tap112i0: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master fwbr112i0 state UNKNOWN group default qlen 1000
    link/ether 1a:00:ac:7e:0f:e3 brd ff:ff:ff:ff:ff:ff
18: fwbr112i0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 6e:3c:a7:d7:75:a5 brd ff:ff:ff:ff:ff:ff
19: fwpr112p0@fwln112i0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master vmbr0 state UP group default qlen 1000
    link/ether ba:53:71:ae:22:25 brd ff:ff:ff:ff:ff:ff
20: fwln112i0@fwpr112p0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master fwbr112i0 state UP group default qlen 1000
    link/ether ce:3f:61:b9:66:3f brd ff:ff:ff:ff:ff:ff
21: tap207i0: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master fwbr207i0 state UNKNOWN group default qlen 1000
    link/ether e6:fd:f7:b7:b7:70 brd ff:ff:ff:ff:ff:ff
25: fwbr207i0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether e2:dd:42:97:68:95 brd ff:ff:ff:ff:ff:ff
26: fwpr207p0@fwln207i0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master vmbr0 state UP group default qlen 1000
    link/ether ae:d8:ce:91:a4:be brd ff:ff:ff:ff:ff:ff
27: fwln207i0@fwpr207p0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master fwbr207i0 state UP group default qlen 1000
    link/ether d2:e0:b0:4f:2f:f6 brd ff:ff:ff:ff:ff:ff

Proxmox Version
Code: 
root@pve02:~# pveversion --verbose
proxmox-ve: 8.0.2 (running kernel: 6.2.16-15-pve)
pve-manager: 8.0.4 (running version: 8.0.4/d258a813cfa6b390)
pve-kernel-6.2: 8.0.5
proxmox-kernel-helper: 8.0.3
proxmox-kernel-6.2.16-15-pve: 6.2.16-15
proxmox-kernel-6.2: 6.2.16-15
proxmox-kernel-6.2.16-6-pve: 6.2.16-7
pve-kernel-6.2.16-3-pve: 6.2.16-3
ceph-fuse: 17.2.6-pve1+3
corosync: 3.1.7-pve3
criu: 3.17.1-2
glusterfs-client: 10.3-5
ifupdown2: 3.2.0-1+pmx5
ksm-control-daemon: 1.4-1
libjs-extjs: 7.0.0-4
libknet1: 1.26-pve1
libproxmox-acme-perl: 1.4.6
libproxmox-backup-qemu0: 1.4.0
libproxmox-rs-perl: 0.3.1
libpve-access-control: 8.0.5
libpve-apiclient-perl: 3.3.1
libpve-common-perl: 8.0.9
libpve-guest-common-perl: 5.0.5
libpve-http-server-perl: 5.0.4
libpve-rs-perl: 0.8.5
libpve-storage-perl: 8.0.2
libspice-server1: 0.15.1-1
lvm2: 2.03.16-2
lxc-pve: 5.0.2-4
lxcfs: 5.0.3-pve3
novnc-pve: 1.4.0-2
proxmox-backup-client: 3.0.3-1
proxmox-backup-file-restore: 3.0.3-1
proxmox-kernel-helper: 8.0.3
proxmox-mail-forward: 0.2.0
proxmox-mini-journalreader: 1.4.0
proxmox-widget-toolkit: 4.0.9
pve-cluster: 8.0.4
pve-container: 5.0.4
pve-docs: 8.0.5
pve-edk2-firmware: 3.20230228-4
pve-firewall: 5.0.3
pve-firmware: 3.8-2
pve-ha-manager: 4.0.2
pve-i18n: 3.0.7
pve-qemu-kvm: 8.0.2-6
pve-xtermjs: 4.16.0-3
qemu-server: 8.0.7
smartmontools: 7.3-pve1
spiceterm: 3.3.0
swtpm: 0.8.0+pve1
vncterm: 1.8.0
zfsutils-linux: 2.1.13-pve1

VM Setting
vmNetwork.png

Network Configuration in the VM
vmIPsettings.png

Switch Configuration
The proxmox server is on Port 15, and was tagged for VLAN 30.
vlan config-switch.png

Under PVID Settings, as soon as I change it from 1 to 30, the node would no longer exist and it does not look like I can have multiple PVIDs either.
pvid.png

OPNSense Router Configuration
The Interface, assignment and firewall rules were set.
Interfaces
interfaces.png
Assignment
assignments.png
Firewall rules on the interface
firewall.png

If I need to add another NIC to the server, I can look into doing that. It was my understanding that it was possible to use one NIC and use VLAN Aware/Tagging,
I do recall one guide showing that you needed remove the IP from the Linux Bridge on the Node so that it only goes to VLANs interfaces, but I'm not sure if that is the correct way of doing it. I would have thought it would just route through the main interface (much like router on a stick configuration.)

Please let me know if there is further information needed.

Any help is very much appreciated!
Thank you!
Ethan
 
You probably don't need to enable VLAN membership on the OPNsense. VLAN Tagging is already done on the Linux Bridge in PVE. The packets that leave the node via enp34s0 already have the VLAN tag attached. At this point, it should already enable isolated communication between VMs.

The VLAN feature in the router usually means that a physical cable connected to a port will be tagged in the router and is not what you want because you want to only use a single cable.

For PVE specific config there are some examples in the Wiki [1]

[1] https://pve.proxmox.com/wiki/Network_Configuration#sysadmin_network_vlan
 
  • Like
Reactions: eth10747
Hi there!

Thank you for your response!
I have managed to figure it out.

I've learned that I did not need the VLAN interfaces in Proxmox since VLAN-aware works as what you have mentioned.
The network switch still had to be tagged with the respective VLANs. So my Proxmox nodes had to be tagged 20 and 30; untagged was 40.

I was also trying to set up DHCP on VLAN 40 which took me longer than I would like to admit to get it work. I also had to set the VLAN ID of the port to 40 whereas the rest of the VLAN IDs are still at 1. VLAN 40 was set to untagged due to the AP not supporting VLAN itself (it what I have come to understand.)

I am now starting to get a better idea with how VLANs work in these types of network configuration.

I will mark your response as resolved.
Thank you!
Ethan
 
  • Like
Reactions: s.lendl
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom
Back