I need some help trying to understand/adjusting something with my configuration.
I have a two-node cluster (and a pi for quorum), one running pve 6.4-13 and the other running pve 7.0-13. The two machines are nearly identical, with the older version having a smaller disk and less RAM. On the older node I am running pfSense with a dedicated NIC passed directly through to the VM. On the newer node I am also running pfSense with a "dedicated" NIC, but I am using bridges to pass the 4 ports over vs. using passthru. Meanwhile, my switches support VLANs and some managed features, but they do NOT allow me to have them use anything other than VLAN 1 for their mgmt interface.
On the switch, I have the ports servicing the dedicated NICs to require all VLANs to be tagged, even VLAN 1, and the pvid set to a dead VLAN. The idea being that any untagged frames would just go to la la land as the two ports on the switch have different dead VLANs as the pvid, so the untagged frames have nowhere to go. On the older/primary node, I have pfSense set to tag every VLAN, including VLAN 1 and everything works perfectly (remember this is 6.4-13 and passthru though). On the newer/secondary node, I have pfSense set to tag every VLAN except 1, which is set to be untagged. On this node, I have the bridge set up as:
auto vmbr11
iface vmbr11 inet manual
bridge-ports enp1s0f1
bridge-stp off
bridge-fd 0
bridge-vlan-aware yes
bridge-vids 1-4094
bridge-pvid 4094
Note that I manually changed the bridge-vids to include VLAN 1 to make this work. This is working, but I scratch my head at it. If I change pfSense to tag VLAN 1 instead of leaving it untagged, I lose connectivity to the VM on that VLAN.
I am guessing here, but I believe the VM is passing VLAN 1 traffic untagged to the bridge, the bridge then tags it and sends it on, and the switch happily accepts the tagged VLAN 1 frame. However, why can't I pass VLAN 1 tagged from the VM to the bridge? Is this some sort of bug? Everything I read leads me to believe that should work (as it does on the older/passthru node), but it doesn't and I have rechecked over and over. Would my older instance run into this bug if I were to upgrade it to pve7?
I understand that we should not use VLAN 1 if at all possible, but that isn't an option here where these switches don't allow changing where their mgmt interface lives.
I have a two-node cluster (and a pi for quorum), one running pve 6.4-13 and the other running pve 7.0-13. The two machines are nearly identical, with the older version having a smaller disk and less RAM. On the older node I am running pfSense with a dedicated NIC passed directly through to the VM. On the newer node I am also running pfSense with a "dedicated" NIC, but I am using bridges to pass the 4 ports over vs. using passthru. Meanwhile, my switches support VLANs and some managed features, but they do NOT allow me to have them use anything other than VLAN 1 for their mgmt interface.
On the switch, I have the ports servicing the dedicated NICs to require all VLANs to be tagged, even VLAN 1, and the pvid set to a dead VLAN. The idea being that any untagged frames would just go to la la land as the two ports on the switch have different dead VLANs as the pvid, so the untagged frames have nowhere to go. On the older/primary node, I have pfSense set to tag every VLAN, including VLAN 1 and everything works perfectly (remember this is 6.4-13 and passthru though). On the newer/secondary node, I have pfSense set to tag every VLAN except 1, which is set to be untagged. On this node, I have the bridge set up as:
auto vmbr11
iface vmbr11 inet manual
bridge-ports enp1s0f1
bridge-stp off
bridge-fd 0
bridge-vlan-aware yes
bridge-vids 1-4094
bridge-pvid 4094
Note that I manually changed the bridge-vids to include VLAN 1 to make this work. This is working, but I scratch my head at it. If I change pfSense to tag VLAN 1 instead of leaving it untagged, I lose connectivity to the VM on that VLAN.
I am guessing here, but I believe the VM is passing VLAN 1 traffic untagged to the bridge, the bridge then tags it and sends it on, and the switch happily accepts the tagged VLAN 1 frame. However, why can't I pass VLAN 1 tagged from the VM to the bridge? Is this some sort of bug? Everything I read leads me to believe that should work (as it does on the older/passthru node), but it doesn't and I have rechecked over and over. Would my older instance run into this bug if I were to upgrade it to pve7?
I understand that we should not use VLAN 1 if at all possible, but that isn't an option here where these switches don't allow changing where their mgmt interface lives.