[SOLVED] Multiple VLAN's in Proxmox

[ferium]

Hello. I spended all day for search information about multiple VLAN's on Proxmox, but nothing found. So, what I want.

I have router with configured VLAN's. Let name this:
VLAN2 - For management. (Web interface Proxmox, ipmi, web interface network devices etc) 10.172.2.x

VLAN3 - For infrastructure VMs 10.172.3.x

VLAN4 - For dev VMs 10.172.4.x

I configured completely VLAN2 on Proxmox

/etc/network/interfaces

auto vmbr0

iface vmbr0 inet manual

        bridge-ports eth0

        bridge-stp off

        bridge-fd 0

        bridge-vlan-aware yes

        bridge-vids 2-4094


auto vmbr0.2

iface vmbr0.2 inet static

        address 10.172.2.253/24

        gateway 10.172.2.254

I tried this config, but it's not working:

auto vmbr0
iface vmbr0 inet manual
        bridge-ports enp0s31f6
        bridge-stp off
        bridge-fd 0
        bridge-vlan-aware yes
        bridge-vids 2-4094

auto vmbr0.2
iface vmbr0.2 inet static
        address 10.172.2.253/24
        gateway 10.172.2.254

auto vmbr0.3
iface vmbr0.3 inet manual

Also I tried set VLAN tag in VM network config, and this not working too.

But I need configured VLAN3 and VLAN4 for VMs and I haven't idea, how to realize this. Perfect, if this will be work without openSwitch. I heared, OVS very glitch. I will be very grateful for your help

 

[aaron]

VLAN 2 is used just for mgmt and guests should not have access to it?
I think the problem comes from making vmbr0 vlan aware and letting all possible VLAN IDs stay in the birdge-vids option.

What you could do is something like the following (not tested, so there might be typos and errors, but you should get the idea):

auto mgmt
iface mgmt inet manual
        address 10.172.2.253/24
        gateway 10.172.2.254
        vlan-raw-device enp0s31f6
       vlan-id 2
#mgmt net

auto vmbr3
iface vmbr3 inet manual
        bridge-ports enp0s31f6.3
        bridge-stp off
        bridge-fd 0
        bridge-vlan-aware yes
        bridge-vids 2-4094
#infra VMs

auto vmbr4
iface vmbr4 inet manual
        bridge-ports enp0s31f6.4
        bridge-stp off
        bridge-fd 0
        bridge-vlan-aware yes
        bridge-vids 2-4094
#dev VMs

Basically, for the vmbr interfaces, we use the dot notation in the bridge-ports section to define the VLAN they are in directly on the physical port. If you would have set up a bond (LAG), we would use it with the dot notation.
I chose the name of vmbrX interfaces to reflect the VLAN ID just for consitency purposes.

For the mgmt interface we use a "Linux VLAN" device, if you would have created it through the GUI with the physical port as vlan-raw-device.
This assumes that the guests don't need access to this VLAN, as we are not using a bridge (internal switch).

If you have multiple hosts in a cluster, take a look at the SDN and VLAN zones. This makes it easier to define them in one place clusterwide.

 

[ferium]

VLAN 2 is used just for mgmt and guests should not have access to it?
I think the problem comes from making vmbr0 vlan aware and letting all possible VLAN IDs stay in the birdge-vids option.

What you could do is something like the following (not tested, so there might be typos and errors, but you should get the idea):

auto mgmt
iface mgmt inet manual
        address 10.172.2.253/24
        gateway 10.172.2.254
        vlan-raw-device enp0s31f6
       vlan-id 2
#mgmt net

auto vmbr3
iface vmbr3 inet manual
        bridge-ports enp0s31f6.3
        bridge-stp off
        bridge-fd 0
        bridge-vlan-aware yes
        bridge-vids 2-4094
#infra VMs

auto vmbr4
iface vmbr4 inet manual
        bridge-ports enp0s31f6.4
        bridge-stp off
        bridge-fd 0
        bridge-vlan-aware yes
        bridge-vids 2-4094
#dev VMs

Basically, for the vmbr interfaces, we use the dot notation in the bridge-ports section to define the VLAN they are in directly on the physical port. If you would have set up a bond (LAG), we would use it with the dot notation.
I chose the name of vmbrX interfaces to reflect the VLAN ID just for consitency purposes.

For the mgmt interface we use a "Linux VLAN" device, if you would have created it through the GUI with the physical port as vlan-raw-device.
This assumes that the guests don't need access to this VLAN, as we are not using a bridge (internal switch).

If you have multiple hosts in a cluster, take a look at the SDN and VLAN zones. This makes it easier to define them in one place clusterwide.

Ok, I configured network like your config. And it's not working, if I didn't set ip address on vmbr2/3. I can create policy in firewall, which reject access to IP address on vmbr3, but it's not right.

This config not working

auto mgmt
iface mgmt inet manual
        address 10.172.2.253/24
        gateway 10.172.2.254
        vlan-raw-device enp0s31f6
        vlan-id 20

auto vmbr3
iface vmbr3 inet manual
        bridge-ports enp0s31f6.3
        bridge-stp off
        bridge-fd 0
        bridge-vlan-aware yes
        bridge-vids 2-4094

This config working

auto mgmt
iface mgmt inet manual
        address 10.172.2.253/24
        gateway 10.172.2.254
        vlan-raw-device enp0s31f6
        vlan-id 20

auto vmbr3
iface vmbr3 inet manual
        address 10.172.3.253/24
        gateway 10.172.3.254
        bridge-ports enp0s31f6.3
        bridge-stp off
        bridge-fd 0
        bridge-vlan-aware yes
        bridge-vids 2-4094

 

[_gabriel]

it's not working

what does mean ?
vmbr do not require ip (except special conf like routed or na
subnet)
VMs attached to vmbr require static ip or dhcp server on the VLAN.

 

[ferium]

what does mean ?

I mean, no ping to GW and other IP's from VM and Proxmox Server from VLAN interfaces. No logs from firewall

 

[ferium]

Ok, I understanded, what is problem. I think problem in this:
I change network config in proxmox, restart networking service and network in VM crashed. And it's working again, when I change and undo some settings in VM network.
Here working config, who want multiple VLANs in one NIC in Proxmox.

auto mgmt
iface mgmt inet manual
        address 10.172.2.253/24
        gateway 10.172.2.254
        vlan-raw-device enp0s31f6 # You need write here your physical interface, which connected to your local network
        vlan-id 2 # You need write here your vlan id

auto vmbr3
iface vmbr3 inet manual
        bridge-ports enp0s31f6.3 # after dot, you need write vlan id
        bridge-stp off
        bridge-fd 0
        bridge-vlan-aware yes
        bridge-vids 2-4094 # I think bridge-vids not requied here, but i leave this here, beacuse it's working.
        vlan-id 3

Also, you need setup tagging on interface, on your switch (if present)