[SOLVED] VMs on separate networks but share single internet connection

S

sugarleaves

New Member
Feb 1, 2020
8
1
1
35
I doubt I'm the only person trying to do this. What's the easiest solution to have VMs on the same proxmox host be on separate networks (ie. they can't talk to each other), but share a connection to the internet?

I currently have my proxmox server set up with multiple VMs all under the same network (ie. internet/modem -> router -> switch -> proxmox server). This is working great, but I now would like to host a public website as a VM on my proxmox server. For security reasons, I do not want the public website VM to be able to access the other existing VMs on the proxmox server. However, I want all my VMs to be able to access the internet.

In my router (Ubiquiti Edgerouter), I am able to create a second network on a different port on the router, so both ports can use the same internet connection from the modem. However, the network address looks like it must be different, and proxmox doesn't allow me to use two gateway addresses (ie. I currently use 192.168.0.1 as my network on the first router port and my proxmox gateway, and I wouldn't be able to use 192.168.2.1 or something else on proxmox as a second gateway).

It looks like VLANs might be a solution, but I can't figure out how to apply them to my situation. It looks like I can get the public website VM on a different VLAN than the existing VMs in proxmox, but then the public website VM VLAN would not have a connection to the router for the internet connection (since it would be isolated on the switch to its own VLAN). Or that's what I think is going to happen?

Again, my setup:
An internet modem is connected to my router via a single ethernet cable. Another ethernet cable connected to the router connects to a 24 port switch (capable of VLAN, etc). The switch is connected to my proxmox server via a single ethernet cable, although I can connect up to three more ethernet cables to the proxmox server.
 
How many NICs do you have in the Proxmox VE server? If you have spare ones you can simplify the setup by defining one of the available ports on your router for the new network and attach it directly to the second NIC.

On the PVE side you then create a new Linux Bridge with that NIC as bridge port. You don't even need to configure an IP address. If you then assign that new bridge to the NIC of the VM it should work.

If you don't have a second NIC available you will have to resort to VLANs. All intermediate steps need to support VLANs though. The router most likely will, so will the PVE server. The big question is if your switch supports VLANs.

You would create a new VLAN on the switch and the router on the same ports. It needs to be a `tagged` VLAN. On the PVE server you can specify the VLAN ID in the settings of the VMs NIC.


I believe that direct cables from additional router ports to additional NICs on the PVE server would be the easiert solution. It has the advantage thaty you have a physical separation by cable which is much easier to debug should it not work right away.
 
  • Like
Reactions: sugarleaves
Thanks for the response!

I have 4 NICS on my Proxmox server:
- One is attached to a Linux Bridge and is reserved for the management console
- Two are in a Linux Bond, which is then used by a Linux Bridge for use by VMs in my LAN (ie. 192.168.0.0/24)
- One is leftover to create a Linux Bridge on a network separate from the other networks (ie. 192.168.2.0/24)

I took your advice since it looks like I was overthinking it. In my original post, I was trying to specify a gateway, thinking that I HAD to specify it to get an IP that was different than my LAN network. Like you said, I can leave all the fields blank when creating the Linux Bridge in the Proxmox network interface, and everything will still work as expected when creating a VM using that Linux Bridge. :)

I connected the leftover NIC to a separate port on my router, which let me configure a different network and firewall rules within the router to separate the networks. Proxmox was able to pick up the correct IP and connect to the internet when I created a VM with the new network (ie. I got an IP of 192.168.2.42!).

Here's a screenshot of my setup now in Proxmox:
1593711606545.png
 
  • Like
Reactions: aaron
aaron said:
How many NICs do you have in the Proxmox VE server? If you have spare ones you can simplify the setup by defining one of the available ports on your router for the new network and attach it directly to the second NIC.

On the PVE side you then create a new Linux Bridge with that NIC as bridge port. You don't even need to configure an IP address. If you then assign that new bridge to the NIC of the VM it should work.

If you don't have a second NIC available you will have to resort to VLANs. All intermediate steps need to support VLANs though. The router most likely will, so will the PVE server. The big question is if your switch supports VLANs.

You would create a new VLAN on the switch and the router on the same ports. It needs to be a `tagged` VLAN. On the PVE server you can specify the VLAN ID in the settings of the VMs NIC.


I believe that direct cables from additional router ports to additional NICs on the PVE server would be the easiert solution. It has the advantage thaty you have a physical separation by cable which is much easier to debug should it not work right away.
Click to expand...
@aaron could you elaborate on the scenario if using a single NIC. I cant seem to get this working even though it was easily configured on VMware.
 
carlsonlv said:
@aaron could you elaborate on the scenario if using a single NIC. I cant seem to get this working even though it was easily configured on VMware.
Click to expand...
What exactly have you tried?

In the NIC settings for the VMs NIC you can define the VLAN tag for the VM and then you will have to set up the switch and router accordingly.
 
aaron said:
What exactly have you tried?

In the NIC settings for the VMs NIC you can define the VLAN tag for the VM and then you will have to set up the switch and router accordingly.
Click to expand...
auto lo
iface lo inet loopback

iface enp2s0 inet manual

auto vmbr0
iface vmbr0 inet static
address 10.0.0.5/24
gateway 10.0.0.1
bridge-ports enp2s0
bridge-stp off
bridge-fd 0
bridge-vlan-aware yes
bridge-vids 2-4094

auto vmbr0.3
iface vmbr0.3 inet static
address 192.168.20.50/24

root@proxmox:/etc/network#


###################
Above is my current config.
I have a single trunk link connected with 4 active VLANs. However all connected VM connects to the default VLAN.

I tried manually setting the vlan in the nic of the VM but still no connection.
 
carlsonlv said:
auto lo
iface lo inet loopback

iface enp2s0 inet manual

auto vmbr0
iface vmbr0 inet static
address 10.0.0.5/24
gateway 10.0.0.1
bridge-ports enp2s0
bridge-stp off
bridge-fd 0
bridge-vlan-aware yes
bridge-vids 2-4094

auto vmbr0.3
iface vmbr0.3 inet static
address 192.168.20.50/24

root@proxmox:/etc/network#


###################
Above is my current config.
I have a single trunk link connected with 4 active VLANs. However all connected VM connects to the default VLAN.

I tried manually setting the vlan in the nic of the VM but still no connection.
Click to expand...
I was testing out VLAN 3 but still no luck
 
Code: 
auto vmbr0.3
iface vmbr0.3 inet static
       address 192.168.20.50/24
This will not work.
First, because the PVE naming scheme for bridges will only allow for vmbrN where N is an integer. How do assign that bridge (vmbr0.3) to a VM? The GUI should not support it AFAICT.

You can try to assign the VLAN tag directly in the settings for each VMs NIC. If you don't want to do that, you can try to do the following:

Code: 
auto vmbr3
iface vmbr3 inet static
    address 192.168.20.50/24
    bridge-port enp2s0.3
    bridge-stp off
    bridge-fd 0

This is settings the VLAN tag on the bridge port.

The VLAN aware option is needed if the VMs themselves should be able to assign VLANs to their NICs.
 
aaron said:
Code: 
auto vmbr0.3
iface vmbr0.3 inet static
       address 192.168.20.50/24
This will not work.
First, because the PVE naming scheme for bridges will only allow for vmbrN where N is an integer. How do assign that bridge (vmbr0.3) to a VM? The GUI should not support it AFAICT.

You can try to assign the VLAN tag directly in the settings for each VMs NIC. If you don't want to do that, you can try to do the following:

Code: 
auto vmbr3
iface vmbr3 inet static
    address 192.168.20.50/24
    bridge-port enp2s0.3
    bridge-stp off
    bridge-fd 0

This is settings the VLAN tag on the bridge port.

The VLAN aware option is needed if the VMs themselves should be able to assign VLANs to their NICs.
Click to expand...

Tried this but still unable to communicate on the network. Unable to pull IP address via DHCP.
Any other suggestion?
NB: The same trunk link work perfectly on my netgear layer 3 switch without any issue.
 
carlsonlv said:
Tried this but still unable to communicate on the network. Unable to pull IP address via DHCP.
Any other suggestion?
NB: The same trunk link work perfectly on my netgear layer 3 switch without any issue.
Click to expand...
@aaron Thank you for the assistance. I used your recommended settings and it finally worked. not sure what was the issue yesterday but thanks again.
 
  • Like
Reactions: aaron
You must log in or register to reply here.
Top Bottom