Proxmox on Management network and vms on own network not working

[edwinherren]

Hi everybody

Not sure if this is an OPNsense issue or Proxmox or both but I hope someone can help me. I did search, read and try numerous posts/suggestions/solutions here and elsewehere but none helped. I'm not even sure if Im going about this the right way.....

My setup: OPNsense on dedicated/standalone server and two proxmox machines, all connected to a managed switch. I have a management network, LAN, and a DMZ network. Each network separated by vlans. OPNsense, and the Proxmox server's web UI are on the same management network and can only be accessed from the LAN. Which is what I want. Only the LAN can access devices on all other networks but other devices cannot acces the LAN and each other. All physical devices work fine, even when using vlans. The problem are the VMs.

I have tried all combinations with and without vlans but have failed on all counts. VMs only work when assigned to the network the proxmox host belongs to: i.e. the management network. The idea is not to have them on the management interface/network but their own network/s.
Does anybody have an idea what could be my problem? Any help would be highly appreciated.

NOTE: Please, I know how to create bridges, vlans, networks, etc. But this is my first time having a management network and I think thats where I am getting lost.

Thank you
Edwin

 

[supermicro_server]

Hello,
what does "VMs only work when assigned to the network the proxmox host belongs to" mean?
No access to internet?
No access to physical network?
No ping to the gateway?

Thanks

 

[edwinherren]

Hi supermicro_server and thanx for your response.
When I create a vm and assign it an IP on the same subnet as the Proxmox host then it works - network and internet acess.
Otherwise the vms created on different subnet have no internet access, cannot be pinged from anywhere. No access to physical network.
Thanx again

 

[supermicro_server]

Can you check if the "Autostart" funcion is checked on the network interface?
Please past here this command:

ethtool <interface_name>

thank you

 

[louie1961]

Please post the contents of your /etc/network/interfaces file here for us to review. Have you made the bridge VLAN aware? Are you connecting your Promox nodes to the managed switch on a trunk port? Your Opensense box must connect to the switch on a trunk port (i.e., a tagged port), and your VLAN aware NIC on your Proxmox nodes must also be connected to a trunked (i.e., untagged) port as well. AND if you are using a VLAN aware WAP, that must connect to ta trunled port as well.

 

[edwinherren]

Can you check if the "Autostart" funcion is checked on the network interface?
Please past here this command:

ethtool <interface_name>

thank you

Hi supermicro_server
Yes the network interface is on autostart

 

[edwinherren]

Please post the contents of your /etc/network/interfaces file here for us to review. Have you made the bridge VLAN aware? Are you connecting your Promox nodes to the managed switch on a trunk port? Your Opensense box must connect to the switch on a trunk port (i.e., a tagged port), and your VLAN aware NIC on your Proxmox nodes must also be connected to a trunked (i.e., untagged) port as well. AND if you are using a VLAN aware WAP, that must connect to ta trunled port as well.

Hi louie1961
Thanx
Here's the current /etc/network/interfaces settings:
auto lo
iface lo inet loopback

auto enp0s31f6
iface enp0s31f6 inet manual

auto vmbr0
iface vmbr0 inet manual
bridge-ports enp0s31f6
bridge-stp off
bridge-fd 0
bridge-vlan-aware yes
bridge-vids 2-4094

auto vmbr0.10
iface vmbr0.10 inet static
address 10.62.10.100/24
gateway 10.62.10.1

auto vmbr1
iface vmbr1 inet manual
bridge-ports none
bridge-stp off
bridge-fd 0
bridge-vlan-aware yes
bridge-vids 2-4094

auto vmbr1.20
iface vmbr1.20 inet static
address 10.62.20.100/24

source /etc/network/interfaces.d/*


And yes, all connections from Opnsense box to switch connected via trunk/tagged ports. The interface is vlan aware.
All connected devices with above /etc/network/interfaces config file work. The Proxmox boxes web UI are accessible. It's only the vms and containers on these boxes that I cannot make work.

The Proxmox boxes are on vlan10 and they are accessible. The vms and cts I want on vlan20, which is on a different subnet to the Proxmox box they reside on.

Thanx a lot