No network

HomeLabNerd

New Member
Sep 9, 2023
27
0
1
Noob question... I'm trying and searching for hours but I can't seem find the solution.

I have 2 bridges. One for management and one for the vm's. What I understand of a bridge that it is function like a switch. So my vmbr1 hands out ip address to my vm's. I'm correct? For some reason it doesn't and I'm not sure why. I'll post also some screenshots. Please some help.


Scherm­afbeelding 2024-04-23 om 14.46.23.png


Code:
GNU nano 7.2                                      /etc/network/interfaces                                               
# network interface settings; autogenerated
# Please do NOT modify this file directly, unless you know what
# you're doing.
#
# If you want to manage parts of the network configuration manually,
# please utilize the 'source' or 'source-directory' directives to do
# so.
# PVE will preserve these directives, but will NOT read its network
# configuration from sourced files, so do not attempt to move any of
# the PVE managed interfaces into external files!


auto lo
iface lo inet loopback


iface enp87s0 inet manual


iface enp89s0 inet manual


iface enp2s0f0 inet manual


iface enp2s0f1 inet manual


auto vmbr0
iface vmbr0 inet static
        address 192.168.1.21/24
        gateway 192.168.1.1
        bridge-ports enp87s0
        bridge-stp off
        bridge-fd 0


iface wlp90s0 inet manual


auto vmbr1
iface vmbr1 inet static
        address 192.168.3.31/24
        bridge-ports enp2s0f0
 
Bridges don't hand out IP addresses. (Switches don't do that either, though).

Either you have to manually assign static IP addresses to each of your VMs/Containers, or you have to set up and configure a DHCP server.
 
Bridges don't hand out IP addresses. (Switches don't do that either, though).

Either you have to manually assign static IP addresses to each of your VMs/Containers, or you have to set up and configure a DHCP server.
Good one, the dhcp server does on the router. My bad.

If I connect a vm to this bridge and I set the vm on dhcp it should get a ip address I guess?
 
Well if your DHCP server is connected to the vmbr1 bridge (so to enp2s0f0) and configured right, then yeah.

You seem to have two interfaces, two bridges and two different subnets, so be sure that it gives out addresses to the right network.
 
Yeah that was my assumption too but it doen't. If I select vmbr0 it works okay but vmbr1 not.. Any way to diagnostic this?
 
It is really strange because the leds from the nic and switch are on. So there is a connection. Also an old Proxmox server is connected and uses this same vmbr1.
 
I have 2 bridges. One for management and one for the vm's.
If you want management separate from your VM/CTs, then don't use a bridge for management. Just configure an interface with an IP address for Proxmox. And configure the bridge without and IP address (but with a gateway) for your VM/CTs.

Maybe something like this, but note that this is completely untested!
Code:
auto lo
iface lo inet loopback

auto enp87s0
iface enp87s0 inet static
        address 192.168.1.21/24
        gateway 192.168.1.1

iface enp89s0 inet manual

iface enp2s0f0 inet manual

iface enp2s0f1 inet manual

auto vmbr0
iface vmbr0 inet static
        gateway 192.168.1.1
        bridge-ports enp2s0f0
        bridge-stp off
        bridge-fd 0

iface wlp90s0 inet manual
 
  • Like
Reactions: HomeLabNerd
If you want management separate from your VM/CTs, then don't use a bridge for management. Just configure an interface with an IP address for Proxmox. And configure the bridge without and IP address (but with a gateway) for your VM/CTs.

Maybe something like this, but note that this is completely untested!
Code:
auto lo
iface lo inet loopback

auto enp87s0
iface enp87s0 inet static
        address 192.168.1.21/24
        gateway 192.168.1.1

iface enp89s0 inet manual

iface enp2s0f0 inet manual

iface enp2s0f1 inet manual

auto vmbr0
iface vmbr0 inet static
        gateway 192.168.1.1
        bridge-ports enp2s0f0
        bridge-stp off
        bridge-fd 0

iface wlp90s0 inet manual
Thank you for the suggestion! Very appreciated. I'm a Proxmox beginner to be honest. I have experiment a little bit with Proxmox in the past but I now have a good use for it. I have executed your suggestion and indeed makes more senses. Learning every day. :)
 
I have my added bridge working (vmbr1). It was a stupid typo on my side. I need to wear my glasses more often.. Thanks all.
 
After the update the problem is back. My 10Gb sftp+ isn't working anymore. For some reason the interface isn't starting.

Current /etc/network/interfaces:

Code:
# network interface settings; autogenerated
# Please do NOT modify this file directly, unless you know what
# you're doing.
#
# If you want to manage parts of the network configuration manually,
# please utilize the 'source' or 'source-directory' directives to do
# so.
# PVE will preserve these directives, but will NOT read its network
# configuration from sourced files, so do not attempt to move any of
# the PVE managed interfaces into external files!

auto lo
iface lo inet loopback

auto enp87s0
iface enp87s0 inet static
        address 192.168.1.21/24
        gateway 192.168.1.1
#management

iface enp2s0f0 inet manual

auto vmbr0
iface vmbr0 inet static
        address 192.168.3.31/24
        bridge-ports enp2s0f0
        bridge-stp off
        bridge-fd 0
#vm's

iface enp89s0 inet manual

iface enp2s0f1 inet manual

iface enp2s0f0np0 inet manual

iface enp2s0f1np1 inet manual

iface wlp90s0 inet manual


Also when doing adjustments on the network part after the 8.2 update I'll get a strange error. Some screenshots:

Scherm­afbeelding 2024-04-24 om 19.39.10.pngScherm­afbeelding 2024-04-24 om 19.39.00.png
Not a clue what is going on. Will have a look again tomorrow evening. Help is appreciated!
 
After the update the problem is back. My 10Gb sftp+ isn't working anymore. For some reason the interface isn't starting.

Current /etc/network/interfaces:

Code:
# network interface settings; autogenerated
# Please do NOT modify this file directly, unless you know what
# you're doing.
#
# If you want to manage parts of the network configuration manually,
# please utilize the 'source' or 'source-directory' directives to do
# so.
# PVE will preserve these directives, but will NOT read its network
# configuration from sourced files, so do not attempt to move any of
# the PVE managed interfaces into external files!

auto lo
iface lo inet loopback

auto enp87s0
iface enp87s0 inet static
        address 192.168.1.21/24
        gateway 192.168.1.1
#management

iface enp2s0f0 inet manual

auto vmbr0
iface vmbr0 inet static
        address 192.168.3.31/24
        bridge-ports enp2s0f0
        bridge-stp off
        bridge-fd 0
#vm's

iface enp89s0 inet manual

iface enp2s0f1 inet manual

iface enp2s0f0np0 inet manual

iface enp2s0f1np1 inet manual

iface wlp90s0 inet manual


Also when doing adjustments on the network part after the 8.2 update I'll get a strange error. Some screenshots:

View attachment 66899View attachment 66898
Not a clue what is going on. Will have a look again tomorrow evening. Help is appreciated!
Maybe network device name was changed by the update. What is the output of ip a?
 
  • Like
Reactions: HomeLabNerd
Code:
auto vmbr0
iface vmbr0 inet static
        address 192.168.3.31/24
        bridge-ports enp2s0f0
        bridge-stp off
        bridge-fd 0
#vm's
Firstly, you mentioned a vmbr1 but your config only shows a vmbr0.
Secondly, assuming vmbr0 is to handle VMs IPs, it should not contain an address.

AFAIK, the easiest way to split management from VMs in your scenario would be the following:
Code:
auto lo
iface lo inet loopback


iface enp87s0 inet manual


iface enp2s0f0 inet manual


auto vmbr0
iface vmbr0 inet static
        address 192.168.1.21/24
        gateway 192.168.1.1
        bridge-ports enp87s0
        bridge-stp off
        bridge-fd 0
#Management


auto vmbr1
iface vmbr1 inet manual
        bridge-ports enp2s0f0
        bridge-stp off
        bridge-fd 0
#VMs

Then:
1. All management will be on enp87s0 as vmbr0, & the PVE itself will be available at 192.168.1.21
2. All VMs you can use on vmbr1, giving them either DHCP or STATIC addresses based on your setup/hw on enp2s0f0
 
  • Like
Reactions: HomeLabNerd
It seems
Maybe network device name was changed by the update. What is the output of ip a?
it seems that the interface name was changed to enp2s0f0np0. Output of ip a:

Code:
root@pve-1:~# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host noprefixroute
       valid_lft forever preferred_lft forever
2: enp87s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    link/ether 58:47:ca:76:9d:25 brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.21/24 scope global enp87s0
       valid_lft forever preferred_lft forever
    inet6 fd46:653:a8f5:6241:5a47:caff:fe76:9d25/64 scope global dynamic mngtmpaddr
       valid_lft 1791sec preferred_lft 1791sec
    inet6 fe80::5a47:caff:fe76:9d25/64 scope link
       valid_lft forever preferred_lft forever
3: enp89s0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 58:47:ca:76:9d:26 brd ff:ff:ff:ff:ff:ff
4: enp2s0f0np0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq master vmbr0 state DOWN group default qlen 1000
    link/ether 58:47:ca:76:9d:23 brd ff:ff:ff:ff:ff:ff
5: enp2s0f1np1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 58:47:ca:76:9d:24 brd ff:ff:ff:ff:ff:ff
6: vmbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 58:47:ca:76:9d:23 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::5a47:caff:fe76:9d23/64 scope link
       valid_lft forever preferred_lft forever
7: wlp90s0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether d0:12:55:2c:10:1e brd ff:ff:ff:ff:ff:ff
8: veth501i0@if2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master vmbr0 state UP group default qlen 1000
    link/ether fe:b1:4f:9a:71:ca brd ff:ff:ff:ff:ff:ff link-netnsid 0

So I changed that. /etc/network/interfaces:

Code:
# network interface settings; autogenerated
# Please do NOT modify this file directly, unless you know what
# you're doing.
#
# If you want to manage parts of the network configuration manually,
# please utilize the 'source' or 'source-directory' directives to do
# so.
# PVE will preserve these directives, but will NOT read its network
# configuration from sourced files, so do not attempt to move any of
# the PVE managed interfaces into external files!

auto lo
iface lo inet loopback

auto enp87s0
iface enp87s0 inet static
        address 192.168.1.21/24
        gateway 192.168.1.1
#management

iface enp89s0 inet manual

iface enp2s0f0np0 inet manual

iface enp2s0f1np1 inet manual

iface wlp90s0 inet manual

auto vmbr0
iface vmbr0 inet manual
        bridge-ports enp2s0f0np0
        bridge-stp off
        bridge-fd 0

Also as mentioned by @gfngfn256 I removed the ip address from my vmbr0. Screenshot:

Schermafbeelding 2024-04-25 om 18.54.46.png

It seems that the interfaces are up but if I start a vm it has no network. I checked in the vm / network if vmbr0 was selected and it is.

Schermafbeelding 2024-04-25 om 18.57.13.png

Not sure what to check more.. Suggestions?
 
Last edited:
Ip a from the LXC container:

Code:
Last login: Thu Apr 25 19:04:50 CEST 2024 on tty1
root@uptimekuma:~# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host noprefixroute
       valid_lft forever preferred_lft forever
2: eth0@if9: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether bc:24:11:0f:74:bf brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet6 fe80::be24:11ff:fe0f:74bf/64 scope link
       valid_lft forever preferred_lft forever
root@uptimekuma:~#
 
Then:
1. All management will be on enp87s0 as vmbr0, & the PVE itself will be available at 192.168.1.21
2. All VMs you can use on vmbr1, giving them either DHCP or STATIC addresses based on your setup/hw on enp2s0f0
enp87s0 is indeed the management interface on 192.168.1.21.

I now have vmbr0 for the vm's. Interface enp2s0f0np0. This was before the update 8.2 enp2s0f0.

@all Hope it makes sense.
 
Code:
root@pve-1:~# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host noprefixroute
       valid_lft forever preferred_lft forever
2: enp87s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    link/ether 58:47:ca:76:9d:25 brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.21/24 scope global enp87s0
       valid_lft forever preferred_lft forever
    inet6 fd46:653:a8f5:6241:5a47:caff:fe76:9d25/64 scope global dynamic mngtmpaddr
       valid_lft 1791sec preferred_lft 1791sec
    inet6 fe80::5a47:caff:fe76:9d25/64 scope link
       valid_lft forever preferred_lft forever
3: enp89s0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 58:47:ca:76:9d:26 brd ff:ff:ff:ff:ff:ff
4: enp2s0f0np0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq master vmbr0 state DOWN group default qlen 1000
    link/ether 58:47:ca:76:9d:23 brd ff:ff:ff:ff:ff:ff
5: enp2s0f1np1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 58:47:ca:76:9d:24 brd ff:ff:ff:ff:ff:ff
6: vmbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 58:47:ca:76:9d:23 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::5a47:caff:fe76:9d23/64 scope link
       valid_lft forever preferred_lft forever
7: wlp90s0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether d0:12:55:2c:10:1e brd ff:ff:ff:ff:ff:ff
8: veth501i0@if2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master vmbr0 state UP group default qlen 1000
    link/ether fe:b1:4f:9a:71:ca brd ff:ff:ff:ff:ff:ff link-netnsid 0

So I changed that. /etc/network/interfaces:

Code:
# network interface settings; autogenerated
# Please do NOT modify this file directly, unless you know what
# you're doing.
#
# If you want to manage parts of the network configuration manually,
# please utilize the 'source' or 'source-directory' directives to do
# so.
# PVE will preserve these directives, but will NOT read its network
# configuration from sourced files, so do not attempt to move any of
# the PVE managed interfaces into external files!

auto lo
iface lo inet loopback

auto enp87s0
iface enp87s0 inet static
        address 192.168.1.21/24
        gateway 192.168.1.1
#management

iface enp89s0 inet manual

iface enp2s0f0np0 inet manual

iface enp2s0f1np1 inet manual

iface wlp90s0 inet manual

auto vmbr0
iface vmbr0 inet manual
        bridge-ports enp2s0f0np0
        bridge-stp off
        bridge-fd 0
Your vmbr0 is configured to include enp2s0f0np0 but enp2s0f0np0 reports NO-CARRIER, which usually means there is no working cable connected.
 
  • Like
Reactions: HomeLabNerd
Your vmbr0 is configured to include enp2s0f0np0 but enp2s0f0np0 reports NO-CARRIER, which usually means there is no working cable connected.
I have connect a cable to that port and it is up. The leds are blinking. So wrong interface I guess. Why is it not visible?

My hardware:

root@pve-1:~# lspci
00:00.0 Host bridge: Intel Corporation Device a706
00:02.0 VGA compatible controller: Intel Corporation Raptor Lake-P [Iris Xe Graphics] (rev 04)
00:06.0 PCI bridge: Intel Corporation Raptor Lake PCIe 4.0 Graphics Port
00:06.2 PCI bridge: Intel Corporation Device a73d
00:07.0 PCI bridge: Intel Corporation Raptor Lake-P Thunderbolt 4 PCI Express Root Port
00:07.2 PCI bridge: Intel Corporation Raptor Lake-P Thunderbolt 4 PCI Express Root Port
00:0d.0 USB controller: Intel Corporation Raptor Lake-P Thunderbolt 4 USB Controller
00:0d.2 USB controller: Intel Corporation Raptor Lake-P Thunderbolt 4 NHI
00:0d.3 USB controller: Intel Corporation Raptor Lake-P Thunderbolt 4 NHI
00:14.0 USB controller: Intel Corporation Alder Lake PCH USB 3.2 xHCI Host Controller (rev 01)
00:14.2 RAM memory: Intel Corporation Alder Lake PCH Shared SRAM (rev 01)
00:16.0 Communication controller: Intel Corporation Alder Lake PCH HECI Controller (rev 01)
00:16.3 Serial controller: Intel Corporation Alder Lake AMT SOL Redirection (rev 01)
00:1c.0 PCI bridge: Intel Corporation Alder Lake-P PCH PCIe Root Port (rev 01)
00:1d.0 PCI bridge: Intel Corporation Alder Lake PCI Express Root Port (rev 01)
00:1d.2 PCI bridge: Intel Corporation Device 51b2 (rev 01)
00:1d.3 PCI bridge: Intel Corporation Device 51b3 (rev 01)
00:1f.0 ISA bridge: Intel Corporation Raptor Lake LPC/eSPI Controller (rev 01)
00:1f.3 Audio device: Intel Corporation Raptor Lake-P/U/H cAVS (rev 01)
00:1f.4 SMBus: Intel Corporation Alder Lake PCH-P SMBus Host Controller (rev 01)
00:1f.5 Serial bus controller: Intel Corporation Alder Lake-P PCH SPI Controller (rev 01)
02:00.0 Ethernet controller: Intel Corporation Ethernet Controller X710 for 10GbE SFP+ (rev 02)
02:00.1 Ethernet controller: Intel Corporation Ethernet Controller X710 for 10GbE SFP+ (rev 02)
57:00.0 Ethernet controller: Intel Corporation Ethernet Controller I226-V (rev 04)
58:00.0 Non-Volatile memory controller: Samsung Electronics Co Ltd NVMe SSD Controller PM9A1/PM9A3/980PRO
59:00.0 Ethernet controller: Intel Corporation Ethernet Controller I226-LM (rev 04)
5a:00.0 Network controller: MEDIATEK Corp. MT7922 802.11ax PCI Express Wireless Network Adapter

It is connected to the x710 10Gbe SFP+.
 
Last edited:
I have connect a cable to that port and it is up. The leds are blinking. So wrong interface I guess.
Okay, what are the output of ip a and cat /etc/network/interfaces (after a reboot) now?
Why is it not visible?
Why is what not visible how? Sorry, I'm doing multiple things interleaved and don't have enough context.

I have a (nested) Proxmox where the virtual bridge for VM/CTs and the Proxmox GUI are on separate network controllers:
Code:
auto lo
iface lo inet loopback

auto ens18
iface ens18 inet static
    address 192.168.0.2/24
    gateway 192.168.0.1

iface ens19 inet manual

auto vmbr0
iface vmbr0 inet manual
    bridge-ports ens19
    bridge-stp off
    bridge-fd 0
It works fine and the Proxmox is only reachable via one network port and the VM/CTs are reachable only via the other network port. Is this not what you want to achieve?
 
So man this was a confusing one!! If your on the back of the server the lowest number of the ethernet port starts on the right. The lowest sfp+ port starts on the left. Also I'm still a newbie with Proxmox and my Linux knowledge is very rusty.. but like learning stuff in particular Linux.
 
You have failed to follow my advice above - follow it & you'll have the required network access you desire. (Obviously with the appropriate cabling in place).
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!