[SOLVED] Creating vmbr0 brings down connectivity

[bromatofiel]

Hello,

TL;DR: I want a Proxmox instance for a homelab, but after installing it, `vmbr0` is not configured ; configuring it manually brings down access to the Proxmox instance.

First time user Proxmox user here. Full disclaimer: I'm not a networking expert, I do have basics knowledge on how usual home network work, but I suppose the issue I'm experiencing today is related to my lack of knowledge here. I planned to follow a full course on Linux networking, but for that, I wanted to set up.... a proxmox instance as a lab.

Hardware:
Dell Optiplex 9020 SFF with

• i7 4790
• 24Go RAM
• 256Gb SSD
• AMD RX 6400
• BIOS option VT-d enabled

Installation:
Due to Xorg errors when trying to install via the bare-metal ISO installer, I opted for the Proxmox VE on top of debian 11 bullseye installation, then upgraded to the pve-kernel-6.2 kernel. All went well there.

Network context:
Can't make it more basic : I have a flat network behind my ISP router/switch with 192.168.0.0/24. The proxmox instance is connected via Ethernet (and I would like it to have fixed IP 192.168.0.14, more on that later).

What happened:
Once the installation was done, I quickly realized that my VMs didn't have Internet connectivity, and after looking in the Proxmox settings, I saw that there was no bridge interface. I set up to add it through the GUI. I attached it to the physical `eno1` interface, put a CIDR 10.0.0.0/24, then applied configuration, and I wasn't able to access my instance both via ssh or GUI afterwards.

The configuration before the apply was the following :

# /etc/network/interfaces
# The loopback network interface
auto lo iface lo inet loopback

# The primary network interface
auto eth0
iface eth0 inet dhcp

The applied configuration is the following :

# /etc/network/interfaces
# The loopback network interface
auto lo iface lo inet loopback

# The primary network interface
auto eth0
iface eth0 inet dhcp

auto vmbr0
iface vmbr0 inet static
  address 10.0.0.0/24
  gateway 10.0.0.1
  bridge-ports eno1
  bridge-stp off
  bridge-fd 0

Then all connection to the host is lost.

Can you help please ?

 

[leesteken]

# /etc/network/interfaces # The loopback network interface auto lo iface lo inet loopback # The primary network interface auto eth0 iface eth0 inet dhcp auto vmbr0 iface vmbr0 inet static address 10.0.0.0/24 gateway 10.0.0.1 bridge-ports eno1 bridge-stp off bridge-fd 0

10.0.0.0 is not a valid IPv4 address (for the Proxmox host on vmbr0). There is also no configuration for eno1. Maybe something like this will work for you:
# The loopback network interface auto lo iface lo inet loopback # The primary network interface auto eth0 iface eth0 inet dhcp iface eno1 inet manual auto vmbr0 iface vmbr0 inet static address 192.168.0.14/24 gateway 192.168.0.1 bridge-ports eno1 bridge-stp off bridge-fd 0

 

[cheiss]

Hi,

in short, vmbr0 is the main bridge of the system and it's IP address is also the external IP address at which the machine is reachable.
Thus, when you changed the configuration to 10.0.0.0/24, it isn't reachable anymore at the previous address.

Apart from that, 10.0.0.0/24 is a network address, not a (valid) host address. (There are loads of resources on that topic on the internet)

You probably want something more like this:

auto vmbr0
iface vmbr0 inet static
  address 192.168.0.14/24
  gateway 192.168.0.1
  bridge-ports eno1
  bridge-stp off
  bridge-fd 0

auto vmbr1
iface vmbr1 inet static
  address 10.0.0.1/24
  bridge-ports none
  bridge-stp off
  bridge-fd 0
  post-up echo 1 >/proc/sys/net/ipv4/ip_forward
  post-up iptables -t nat -A POSTROUTING -s '10.0.0.0/24' -o vmbr0 -j MASQUERADE
  post-down iptables -t nat -D POSTROUTING -s '10.0.0.0/8' -o vmbr0 -j MASQUERADE

You might need to adjust the gateway address of vmbr0, which should be the IP address of your router.

That way, you have two bridges - one external and one for internal networking.
Attach your VMs to vmbr1, and they should be able to communicate with each other (and reach the internet). The VMs attached to vmbr1 should then also be configured to have an IP address in the 10.0.0.0/24 network and their gateway must be set to 10.0.0.1.

 

[bromatofiel]

10.0.0.0 is not a valid IPv4 address (for the Proxmox host on vmbr0). There is also no configuration for eno1. Maybe something like this will work for you:
# The loopback network interface
auto lo iface lo inet loopback

# The primary network interface
auto eth0
iface eth0 inet dhcp

iface eno1 inet manual

auto vmbr0
iface vmbr0 inet static
address 192.168.0.14/24
gateway 192.168.0.1
bridge-ports eno1
bridge-stp off
bridge-fd 0

Touché !

It solved my issue, thank you. And I can confirm that my VMs have Internet connectivity now.

Although I realize that I do have to get some skills in networking...
I can't wrap my head around the how of this solution.
If eno1 is the physical interface for the Proxmox host, I'd expect it to bear the configuration to access the host itself. Any configuration related to the VMs (say... vmbr0) would bear settings for internal VM network, yet the fixed IP for the Proxmox Host (192.168.0.14) is under vmbr0.

I can see that any new VM is assigned an IP on my local network and that the router can see it.
Can I assume that the Proxmox server acts somewhat as a switch here ?

 

[cheiss]

Can I assume that the Proxmox server acts somewhat as a switch here ?

Yes, essentially. Networking is fairly extensively described in the documentation. It gives a good overview of all the possibilities for networking under Proxmox VE (and by extension, Linux in general).

It's also possible to convert it to a "routed" setup, as described here. In this case, the Proxmox VE server essentially acts as a router. Then, the actual physical network interfaces also gets the WAN IP address, probably what you expected at first.

 

[bromatofiel]

Yes, essentially. Networking is fairly extensively described in the documentation. It gives a good overview of all the possibilities for networking under Proxmox VE (and by extension, Linux in general).

It's also possible to convert it to a "routed" setup, as described here. In this case, the Proxmox VE server essentially acts as a router. Then, the actual physical network interfaces also gets the WAN IP address, probably what you expected at first.

Thank you @cheiss !