No network

HomeLabNerd · Apr 23, 2024

Noob question... I'm trying and searching for hours but I can't seem find the solution.

I have 2 bridges. One for management and one for the vm's. What I understand of a bridge that it is function like a switch. So my vmbr1 hands out ip address to my vm's. I'm correct? For some reason it doesn't and I'm not sure why. I'll post also some screenshots. Please some help.

Schermafbeelding 2024-04-23 om 14.46.23.png

Code:

GNU nano 7.2                                      /etc/network/interfaces                                               
# network interface settings; autogenerated
# Please do NOT modify this file directly, unless you know what
# you're doing.
#
# If you want to manage parts of the network configuration manually,
# please utilize the 'source' or 'source-directory' directives to do
# so.
# PVE will preserve these directives, but will NOT read its network
# configuration from sourced files, so do not attempt to move any of
# the PVE managed interfaces into external files!


auto lo
iface lo inet loopback


iface enp87s0 inet manual


iface enp89s0 inet manual


iface enp2s0f0 inet manual


iface enp2s0f1 inet manual


auto vmbr0
iface vmbr0 inet static
        address 192.168.1.21/24
        gateway 192.168.1.1
        bridge-ports enp87s0
        bridge-stp off
        bridge-fd 0


iface wlp90s0 inet manual


auto vmbr1
iface vmbr1 inet static
        address 192.168.3.31/24
        bridge-ports enp2s0f0

B.Otto · Apr 23, 2024

Bridges don't hand out IP addresses. (Switches don't do that either, though).

Either you have to manually assign static IP addresses to each of your VMs/Containers, or you have to set up and configure a DHCP server.

HomeLabNerd · Apr 23, 2024

B.Otto said:
Bridges don't hand out IP addresses. (Switches don't do that either, though).

Either you have to manually assign static IP addresses to each of your VMs/Containers, or you have to set up and configure a DHCP server.

Good one, the dhcp server does on the router. My bad.

If I connect a vm to this bridge and I set the vm on dhcp it should get a ip address I guess?

B.Otto · Apr 23, 2024

Well if your DHCP server is connected to the vmbr1 bridge (so to enp2s0f0) and configured right, then yeah.

You seem to have two interfaces, two bridges and two different subnets, so be sure that it gives out addresses to the right network.

HomeLabNerd · Apr 23, 2024

Yeah that was my assumption too but it doen't. If I select vmbr0 it works okay but vmbr1 not.. Any way to diagnostic this?

HomeLabNerd · Apr 23, 2024

It is really strange because the leds from the nic and switch are on. So there is a connection. Also an old Proxmox server is connected and uses this same vmbr1.

leesteken · Apr 23, 2024

MvL72 said:
I have 2 bridges. One for management and one for the vm's.

If you want management separate from your VM/CTs, then don't use a bridge for management. Just configure an interface with an IP address for Proxmox. And configure the bridge without and IP address (but with a gateway) for your VM/CTs.

Maybe something like this, but note that this is completely untested!

Code:

auto lo
iface lo inet loopback

auto enp87s0
iface enp87s0 inet static
        address 192.168.1.21/24
        gateway 192.168.1.1

iface enp89s0 inet manual

iface enp2s0f0 inet manual

iface enp2s0f1 inet manual

auto vmbr0
iface vmbr0 inet static
        gateway 192.168.1.1
        bridge-ports enp2s0f0
        bridge-stp off
        bridge-fd 0

iface wlp90s0 inet manual

HomeLabNerd · Apr 23, 2024

leesteken said:
If you want management separate from your VM/CTs, then don't use a bridge for management. Just configure an interface with an IP address for Proxmox. And configure the bridge without and IP address (but with a gateway) for your VM/CTs.

Maybe something like this, but note that this is completely untested!

Code:

auto lo iface lo inet loopback auto enp87s0 iface enp87s0 inet static address 192.168.1.21/24 gateway 192.168.1.1 iface enp89s0 inet manual iface enp2s0f0 inet manual iface enp2s0f1 inet manual auto vmbr0 iface vmbr0 inet static gateway 192.168.1.1 bridge-ports enp2s0f0 bridge-stp off bridge-fd 0 iface wlp90s0 inet manual

Thank you for the suggestion! Very appreciated. I'm a Proxmox beginner to be honest. I have experiment a little bit with Proxmox in the past but I now have a good use for it. I have executed your suggestion and indeed makes more senses. Learning every day.

HomeLabNerd · Apr 23, 2024

I have my added bridge working (vmbr1). It was a stupid typo on my side. I need to wear my glasses more often.. Thanks all.

HomeLabNerd · Apr 24, 2024

After the update the problem is back. My 10Gb sftp+ isn't working anymore. For some reason the interface isn't starting.

Current /etc/network/interfaces:

Code:

# network interface settings; autogenerated
# Please do NOT modify this file directly, unless you know what
# you're doing.
#
# If you want to manage parts of the network configuration manually,
# please utilize the 'source' or 'source-directory' directives to do
# so.
# PVE will preserve these directives, but will NOT read its network
# configuration from sourced files, so do not attempt to move any of
# the PVE managed interfaces into external files!

auto lo
iface lo inet loopback

auto enp87s0
iface enp87s0 inet static
        address 192.168.1.21/24
        gateway 192.168.1.1
#management

iface enp2s0f0 inet manual

auto vmbr0
iface vmbr0 inet static
        address 192.168.3.31/24
        bridge-ports enp2s0f0
        bridge-stp off
        bridge-fd 0
#vm's

iface enp89s0 inet manual

iface enp2s0f1 inet manual

iface enp2s0f0np0 inet manual

iface enp2s0f1np1 inet manual

iface wlp90s0 inet manual

Also when doing adjustments on the network part after the 8.2 update I'll get a strange error. Some screenshots:

Schermafbeelding 2024-04-24 om 19.39.10.png

Schermafbeelding 2024-04-24 om 19.39.00.png

Not a clue what is going on. Will have a look again tomorrow evening. Help is appreciated!

leesteken · Apr 24, 2024

MvL72 said:
After the update the problem is back. My 10Gb sftp+ isn't working anymore. For some reason the interface isn't starting.

Current /etc/network/interfaces:

Code:

# network interface settings; autogenerated # Please do NOT modify this file directly, unless you know what # you're doing. # # If you want to manage parts of the network configuration manually, # please utilize the 'source' or 'source-directory' directives to do # so. # PVE will preserve these directives, but will NOT read its network # configuration from sourced files, so do not attempt to move any of # the PVE managed interfaces into external files! auto lo iface lo inet loopback auto enp87s0 iface enp87s0 inet static address 192.168.1.21/24 gateway 192.168.1.1 #management iface enp2s0f0 inet manual auto vmbr0 iface vmbr0 inet static address 192.168.3.31/24 bridge-ports enp2s0f0 bridge-stp off bridge-fd 0 #vm's iface enp89s0 inet manual iface enp2s0f1 inet manual iface enp2s0f0np0 inet manual iface enp2s0f1np1 inet manual iface wlp90s0 inet manual

Also when doing adjustments on the network part after the 8.2 update I'll get a strange error. Some screenshots:

View attachment 66899 View attachment 66898
Not a clue what is going on. Will have a look again tomorrow evening. Help is appreciated!

Maybe network device name was changed by the update. What is the output of ip a?

gfngfn256 · Apr 24, 2024

Code:

auto vmbr0
iface vmbr0 inet static
        address 192.168.3.31/24
        bridge-ports enp2s0f0
        bridge-stp off
        bridge-fd 0
#vm's

Firstly, you mentioned a vmbr1 but your config only shows a vmbr0.
Secondly, assuming vmbr0 is to handle VMs IPs, it should not contain an address.

AFAIK, the easiest way to split management from VMs in your scenario would be the following:

Code:

auto lo
iface lo inet loopback


iface enp87s0 inet manual


iface enp2s0f0 inet manual


auto vmbr0
iface vmbr0 inet static
        address 192.168.1.21/24
        gateway 192.168.1.1
        bridge-ports enp87s0
        bridge-stp off
        bridge-fd 0
#Management


auto vmbr1
iface vmbr1 inet manual
        bridge-ports enp2s0f0
        bridge-stp off
        bridge-fd 0
#VMs

Then:
1. All management will be on enp87s0 as vmbr0, & the PVE itself will be available at 192.168.1.21
2. All VMs you can use on vmbr1, giving them either DHCP or STATIC addresses based on your setup/hw on enp2s0f0

HomeLabNerd · Apr 25, 2024

It seems

leesteken said:
Maybe network device name was changed by the update. What is the output of ip a?

it seems that the interface name was changed to enp2s0f0np0. Output of ip a:

Code:

root@pve-1:~# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host noprefixroute
       valid_lft forever preferred_lft forever
2: enp87s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    link/ether 58:47:ca:76:9d:25 brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.21/24 scope global enp87s0
       valid_lft forever preferred_lft forever
    inet6 fd46:653:a8f5:6241:5a47:caff:fe76:9d25/64 scope global dynamic mngtmpaddr
       valid_lft 1791sec preferred_lft 1791sec
    inet6 fe80::5a47:caff:fe76:9d25/64 scope link
       valid_lft forever preferred_lft forever
3: enp89s0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 58:47:ca:76:9d:26 brd ff:ff:ff:ff:ff:ff
4: enp2s0f0np0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq master vmbr0 state DOWN group default qlen 1000
    link/ether 58:47:ca:76:9d:23 brd ff:ff:ff:ff:ff:ff
5: enp2s0f1np1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 58:47:ca:76:9d:24 brd ff:ff:ff:ff:ff:ff
6: vmbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 58:47:ca:76:9d:23 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::5a47:caff:fe76:9d23/64 scope link
       valid_lft forever preferred_lft forever
7: wlp90s0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether d0:12:55:2c:10:1e brd ff:ff:ff:ff:ff:ff
8: veth501i0@if2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master vmbr0 state UP group default qlen 1000
    link/ether fe:b1:4f:9a:71:ca brd ff:ff:ff:ff:ff:ff link-netnsid 0

So I changed that. /etc/network/interfaces:

Code:

# network interface settings; autogenerated
# Please do NOT modify this file directly, unless you know what
# you're doing.
#
# If you want to manage parts of the network configuration manually,
# please utilize the 'source' or 'source-directory' directives to do
# so.
# PVE will preserve these directives, but will NOT read its network
# configuration from sourced files, so do not attempt to move any of
# the PVE managed interfaces into external files!

auto lo
iface lo inet loopback

auto enp87s0
iface enp87s0 inet static
        address 192.168.1.21/24
        gateway 192.168.1.1
#management

iface enp89s0 inet manual

iface enp2s0f0np0 inet manual

iface enp2s0f1np1 inet manual

iface wlp90s0 inet manual

auto vmbr0
iface vmbr0 inet manual
        bridge-ports enp2s0f0np0
        bridge-stp off
        bridge-fd 0

Also as mentioned by @gfngfn256 I removed the ip address from my vmbr0. Screenshot:

Schermafbeelding 2024-04-25 om 18.54.46.png

It seems that the interfaces are up but if I start a vm it has no network. I checked in the vm / network if vmbr0 was selected and it is.

Schermafbeelding 2024-04-25 om 18.57.13.png

Not sure what to check more.. Suggestions?

HomeLabNerd · Apr 25, 2024

Ip a from the LXC container:

Code:

Last login: Thu Apr 25 19:04:50 CEST 2024 on tty1
root@uptimekuma:~# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host noprefixroute
       valid_lft forever preferred_lft forever
2: eth0@if9: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether bc:24:11:0f:74:bf brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet6 fe80::be24:11ff:fe0f:74bf/64 scope link
       valid_lft forever preferred_lft forever
root@uptimekuma:~#

HomeLabNerd · Apr 25, 2024

gfngfn256 said:
Then:
1. All management will be on enp87s0 as vmbr0, & the PVE itself will be available at 192.168.1.21
2. All VMs you can use on vmbr1, giving them either DHCP or STATIC addresses based on your setup/hw on enp2s0f0

Click to expand...

gfngfn256 said:
enp87s0 is indeed the management interface on 192.168.1.21.

I now have vmbr0 for the vm's. Interface enp2s0f0np0. This was before the update 8.2 enp2s0f0.

@all Hope it makes sense.

leesteken · Apr 25, 2024

MvL72 said:

Code:

root@pve-1:~# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host noprefixroute
       valid_lft forever preferred_lft forever
2: enp87s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    link/ether 58:47:ca:76:9d:25 brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.21/24 scope global enp87s0
       valid_lft forever preferred_lft forever
    inet6 fd46:653:a8f5:6241:5a47:caff:fe76:9d25/64 scope global dynamic mngtmpaddr
       valid_lft 1791sec preferred_lft 1791sec
    inet6 fe80::5a47:caff:fe76:9d25/64 scope link
       valid_lft forever preferred_lft forever
3: enp89s0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 58:47:ca:76:9d:26 brd ff:ff:ff:ff:ff:ff
4: enp2s0f0np0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq master vmbr0 state DOWN group default qlen 1000
    link/ether 58:47:ca:76:9d:23 brd ff:ff:ff:ff:ff:ff
5: enp2s0f1np1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 58:47:ca:76:9d:24 brd ff:ff:ff:ff:ff:ff
6: vmbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 58:47:ca:76:9d:23 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::5a47:caff:fe76:9d23/64 scope link
       valid_lft forever preferred_lft forever
7: wlp90s0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether d0:12:55:2c:10:1e brd ff:ff:ff:ff:ff:ff
8: veth501i0@if2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master vmbr0 state UP group default qlen 1000
    link/ether fe:b1:4f:9a:71:ca brd ff:ff:ff:ff:ff:ff link-netnsid 0

So I changed that. /etc/network/interfaces:

Code:

# network interface settings; autogenerated
# Please do NOT modify this file directly, unless you know what
# you're doing.
#
# If you want to manage parts of the network configuration manually,
# please utilize the 'source' or 'source-directory' directives to do
# so.
# PVE will preserve these directives, but will NOT read its network
# configuration from sourced files, so do not attempt to move any of
# the PVE managed interfaces into external files!

auto lo
iface lo inet loopback

auto enp87s0
iface enp87s0 inet static
        address 192.168.1.21/24
        gateway 192.168.1.1
#management

iface enp89s0 inet manual

iface enp2s0f0np0 inet manual

iface enp2s0f1np1 inet manual

iface wlp90s0 inet manual

auto vmbr0
iface vmbr0 inet manual
        bridge-ports enp2s0f0np0
        bridge-stp off
        bridge-fd 0

Your vmbr0 is configured to include enp2s0f0np0 but enp2s0f0np0 reports NO-CARRIER, which usually means there is no working cable connected.

HomeLabNerd · Apr 25, 2024

leesteken said:
Your vmbr0 is configured to include enp2s0f0np0 but enp2s0f0np0 reports NO-CARRIER, which usually means there is no working cable connected.

I have connect a cable to that port and it is up. The leds are blinking. So wrong interface I guess. Why is it not visible?

My hardware:

root@pve-1:~# lspci
00:00.0 Host bridge: Intel Corporation Device a706
00:02.0 VGA compatible controller: Intel Corporation Raptor Lake-P [Iris Xe Graphics] (rev 04)
00:06.0 PCI bridge: Intel Corporation Raptor Lake PCIe 4.0 Graphics Port
00:06.2 PCI bridge: Intel Corporation Device a73d
00:07.0 PCI bridge: Intel Corporation Raptor Lake-P Thunderbolt 4 PCI Express Root Port
00:07.2 PCI bridge: Intel Corporation Raptor Lake-P Thunderbolt 4 PCI Express Root Port
00:0d.0 USB controller: Intel Corporation Raptor Lake-P Thunderbolt 4 USB Controller
00:0d.2 USB controller: Intel Corporation Raptor Lake-P Thunderbolt 4 NHI
00:0d.3 USB controller: Intel Corporation Raptor Lake-P Thunderbolt 4 NHI
00:14.0 USB controller: Intel Corporation Alder Lake PCH USB 3.2 xHCI Host Controller (rev 01)
00:14.2 RAM memory: Intel Corporation Alder Lake PCH Shared SRAM (rev 01)
00:16.0 Communication controller: Intel Corporation Alder Lake PCH HECI Controller (rev 01)
00:16.3 Serial controller: Intel Corporation Alder Lake AMT SOL Redirection (rev 01)
00:1c.0 PCI bridge: Intel Corporation Alder Lake-P PCH PCIe Root Port (rev 01)
00:1d.0 PCI bridge: Intel Corporation Alder Lake PCI Express Root Port (rev 01)
00:1d.2 PCI bridge: Intel Corporation Device 51b2 (rev 01)
00:1d.3 PCI bridge: Intel Corporation Device 51b3 (rev 01)
00:1f.0 ISA bridge: Intel Corporation Raptor Lake LPC/eSPI Controller (rev 01)
00:1f.3 Audio device: Intel Corporation Raptor Lake-P/U/H cAVS (rev 01)
00:1f.4 SMBus: Intel Corporation Alder Lake PCH-P SMBus Host Controller (rev 01)
00:1f.5 Serial bus controller: Intel Corporation Alder Lake-P PCH SPI Controller (rev 01)
02:00.0 Ethernet controller: Intel Corporation Ethernet Controller X710 for 10GbE SFP+ (rev 02)
02:00.1 Ethernet controller: Intel Corporation Ethernet Controller X710 for 10GbE SFP+ (rev 02)
57:00.0 Ethernet controller: Intel Corporation Ethernet Controller I226-V (rev 04)
58:00.0 Non-Volatile memory controller: Samsung Electronics Co Ltd NVMe SSD Controller PM9A1/PM9A3/980PRO
59:00.0 Ethernet controller: Intel Corporation Ethernet Controller I226-LM (rev 04)
5a:00.0 Network controller: MEDIATEK Corp. MT7922 802.11ax PCI Express Wireless Network Adapter

It is connected to the x710 10Gbe SFP+.

leesteken · Apr 25, 2024

MvL72 said:
I have connect a cable to that port and it is up. The leds are blinking. So wrong interface I guess.

Okay, what are the output of ip a and cat /etc/network/interfaces (after a reboot) now?

MvL72 said:
Why is it not visible?

Why is what not visible how? Sorry, I'm doing multiple things interleaved and don't have enough context.

I have a (nested) Proxmox where the virtual bridge for VM/CTs and the Proxmox GUI are on separate network controllers:

Code:

auto lo
iface lo inet loopback

auto ens18
iface ens18 inet static
    address 192.168.0.2/24
    gateway 192.168.0.1

iface ens19 inet manual

auto vmbr0
iface vmbr0 inet manual
    bridge-ports ens19
    bridge-stp off
    bridge-fd 0

It works fine and the Proxmox is only reachable via one network port and the VM/CTs are reachable only via the other network port. Is this not what you want to achieve?

HomeLabNerd · Apr 25, 2024

So man this was a confusing one!! If your on the back of the server the lowest number of the ethernet port starts on the right. The lowest sfp+ port starts on the left. Also I'm still a newbie with Proxmox and my Linux knowledge is very rusty.. but like learning stuff in particular Linux.

gfngfn256 · Apr 25, 2024

You have failed to follow my advice above - follow it & you'll have the required network access you desire. (Obviously with the appropriate cabling in place).

No network

New Member

Active Member

New Member

Active Member

New Member

New Member

Distinguished Member

New Member

New Member

New Member

Distinguished Member

Renowned Member

New Member

New Member

New Member

Distinguished Member

New Member

Distinguished Member

New Member

Renowned Member