VLAN - Proxmox or linux?

[John Morris]

I'm evaluating Proxmox and I've got a small lab setup and I'm having a problem getting network access to my VMs. My intent is for the VMs to appear on the network as if plugged in directly. Each VM may have one or more VLANs configured on its local network stack. The connection is a dot1q trunk. I'm using a bond because should this test go well, that's what I will be using in production.

I am not currently using the Proxmox firewall. As far as I can tell the network connection is wide open.

Here's a crude drawing of my setup:

Network setup on VM-Host in simplest terms.

------- ----------------- --------
| VMBR0 |------------|eno1 BOND0 eno2|---------| SWITCH |
------- ----------------- --------

Test VLAN 10 from VM-Host to Switch

----------- ---------- -------------
| VMBR0.10 | | | | SWITCH |
| IP Addr |------------| BOND0.10 | | Int VLAN 10 |
| 10.0.0.10 | | | | 10.0.0.2 |
----------- ---------- -------------
| |
| |
------- ----------------- --------
| VMBR0 |------------|eno1 BOND0 eno2|---------| SWITCH |
------- ----------------- --------

VM Test VLAN 10

---------- -------------
| vm100 | | SWITCH |
| ens18.10 | | Int VLAN 10 |
| 10.0.0.1 | | 10.0.0.2 |
---------- _____________
| |
| |
------- ------- ----------------- --------
| VM100 | | | | | | |
| ens18 |---------| VMBR0 |------------|eno1 BOND0 eno2|---------| SWITCH |
| No IP | | | | | | |
------- ------- ----------------- --------

From the shell on the host, I ping from 10.0.0.10 to 10.0.0.2 successfully.

Next up... VM

The operating system in the VM is a minimal ubuntu server installation. The only interface with an IP address is ens18.10

I am unable to ping the switch, 10.0.0.1 to 10.0.0.2

Using tcpdump on the host, I watch bond0 while the VM pings the switch, and I see the VM ask "who has 10.0.0.2 tell 10.0.0.1" and I see the switch reply "10.0.0.2 is at MA:CAD:RE:SS" That seems to confirm that bond0 to the switch (and back again) is good. VM->VMBR0->BOND0->SWITCH->BOND0 OK

Using tcpdump to view vmbr0, I see the VM ask "who has 10.0.0.2 tell 10.0.0.1" but see no reply. The reply packets from the switch aren't making it from bond0 to vmbr0.

I honestly don't have a clue as to what would stop the packets from going from bond0 to vmbr0. Can anyone clue me in?

--------

The network setup on the VM-Host is pretty simple. I manage the VM-Host through a management interface on eno3. eno4 is unused.

auto lo
iface lo inet loopback

auto eno3
iface eno3 inet static
address 192.168.0.1/24
gateway 192.168.0.254

iface eno4 inet manual

iface eno1 inet manual

iface eno2 inet manual

auto bond0
iface bond0 inet manual
bond-slaves eno1 eno2
bond-miimon 100
bond-mode 802.3ad
bond-xmit-hash-policy layer2+3

iface bond0.10 inet manual

auto vmbr0
iface vmbr0 inet manual
bridge-ports bond0
bridge-stp off
bridge-fd 0
bridge-vlan-aware yes
bridge-vids 2-4094

auto vmbr0v10
iface vmbr0v10 inet static
address 10.0.0.10/24
bridge-ports bond0.10
bridge-stp off
bridge-fd 0

 

[jsterr]

So you want to have Tagged VLANs but in VM you dont want to enter VLAN-TAG? Since 8,1 theres a SDN that helps a lot with these, especially on multuple nodes in a cluster (if you have or plan to have). Even with one node it might be worth look at it, as its really good written down here: https://pve.proxmox.com/pve-docs/chapter-pvesdn.html#pvesdn_zone_plugin_vlan

Can you show how the switchport config looks like? If you are using VLAN-IDs its also needs to be tagged vlan on switch, not untagged.

 

[John Morris]

I want each VM to have access to the entire trunk, and then each VM will tag the VLANs of the networks it is using.

My understanding is, that the VM should have access to vmbr0, and again, that tagging will be done in the VM.

I can see the packets going out from the VM through the host and to the switch. Then I see the packets coming back in from the switch, but the packet disappears between bond0 and vmbr0, so the VM never sees the reply.

 

[jsterr]

So if VM should see the entire trunk and use it on demand (by tagging) you just need one vmbr0 with vlan aware yes and a vlan tag inside the vm. Theres not more to do. You dont need vmbr0v10, youj can have one vmbr0 with 100+ vlans all working, by settings vlanaware yes and putting the correct vlan id inside the hardware-configuration of the vm/ct.

But the ID needs to be inside the network-adapter (hw-definition) of your vm, not inside the actual vm (in os). no tagging needed inside the os.