[SOLVED] Yet another "ifreload -a" failed with exit code 1

A

Adderol

New Member
Sep 2, 2024
13
0
1
Greetings all!

I have a Dell R730 with an intel x550 4 port 10G network card in it. It was previously a member of the cluster but for drive space reasons I needed to re-install PVE. Any time I try to create a dedicated interface for my storage vlan or management vlan it craps out, doesn't matter if it's in the PVE Gui or in the CLI. This is a fresh install using the 8.2-2 iso and 8.2.1 (currently sitting at 8.2.1). ifreload debug file and interfaces config file are attached.

Please forgive me if this exact problem has been solved already. I have searched through the forums and while many people have similar problems, none of the solutions worked for me.

Thank you in advance for any time spent looking over my issue.
 

Attachments

Probably important excerpt from the log:
Code: 
 error: netlink: eno1.11: cannot create vlan eno1.11 11: operation failed with 'Operation not supported' (95)
On top of that, looks like you're binding the vmbr0 to the eno1, not the eno1.11, so without the vlan which I SUSPECT you want to use (given the IP-range you've set up)

Given a quick google, I found this: https://stackoverflow.com/questions/65159153/proxmox-create-vlan-not-supported and the link back to the wiki https://pve.proxmox.com/wiki/Network_Configuration#sysadmin_network_vlan

Have you tried using a bond "in-between" the eno-interfaces?
So something like this for example?

Code: 
auto lo
iface lo inet loopback

iface eno1 inet manual

iface eno2 inet manual

iface eno3 inet manual

iface eno4 inet manual

auto bond0
iface bond0 inet manual
        bond-slaves eno1
        bond-miimon 100
        bond-mode active-backup
        bond-primary eno1

iface bond0.11 inet manual

auto vmbr0v11
iface vmbr0 inet static
    address 192.168.11.26/24
    gateway 192.168.11.1
    bridge-ports bond0.11
    bridge-stp off
    bridge-fd 0

auto vmbr0
iface vmbr0 inet manual
        bridge-ports bond0
        bridge-stp off
        bridge-fd 0

source /etc/network/interfaces.d/*

Also, have you double-checked / manually checked that the VLAN's are allowed on the switch (and have the correct default/untagged vlan)
 
sw-omit said:
Probably important excerpt from the log:
Code: 
 error: netlink: eno1.11: cannot create vlan eno1.11 11: operation failed with 'Operation not supported' (95)
On top of that, looks like you're binding the vmbr0 to the eno1, not the eno1.11, so without the vlan which I SUSPECT you want to use (given the IP-range you've set up)

Given a quick google, I found this: https://stackoverflow.com/questions/65159153/proxmox-create-vlan-not-supported and the link back to the wiki https://pve.proxmox.com/wiki/Network_Configuration#sysadmin_network_vlan

Have you tried using a bond "in-between" the eno-interfaces?
So something like this for example?

Code: 
auto lo
iface lo inet loopback

iface eno1 inet manual

iface eno2 inet manual

iface eno3 inet manual

iface eno4 inet manual

auto bond0
iface bond0 inet manual
        bond-slaves eno1
        bond-miimon 100
        bond-mode active-backup
        bond-primary eno1

iface bond0.11 inet manual

auto vmbr0v11
iface vmbr0 inet static
    address 192.168.11.26/24
    gateway 192.168.11.1
    bridge-ports bond0.11
    bridge-stp off
    bridge-fd 0

auto vmbr0
iface vmbr0 inet manual
        bridge-ports bond0
        bridge-stp off
        bridge-fd 0

source /etc/network/interfaces.d/*

Also, have you double-checked / manually checked that the VLAN's are allowed on the switch (and have the correct default/untagged vlan)
Click to expand...
Thank you for the swift reply!

I have also seen that article and read the wiki (although I could have missed something, I am human).

To answer you questions in order:

1.) Yes i would like to have eno1.11 as my VMBR0's bridge port, but it is not being created when the config is applied due to the error that you mentioned in your reply. eno1 is directly a member right now so I can have access to the machine via the network and not idrac (just makes life easier)

2.) I have not tried to create the vlan off of a bond interface but I did try a rendition of that with a bridge and it did not work. Same error as before only the interface name changed in the error message. it doesn't seem to matter whether its a bridge, bond, or port, vlan creation just seems busted for some reason. I will give this a shot anyway to see what happens and report back.

3.) Yes, I have confirmed the interface configs on the switch (Cisco 3850 12UX) and I currently have two other hosts running 8.2.4 PVE that work just fine.

Thank you very much for your time and consideration.
 
Other two servers are Supermicro servers, and I don't believe they share the same network card but I will double check to make sure. They also show their interfaces as enoX.

Real quick, I tried the bond but got the message below. I am still researching it.

Code: 
warning: error while writing to file /sys/class/net/bonding_masters: [Errno 13] Permission denied: '/sys/class/net/bonding_masters'
error: netlink: eno2: cannot enslave link eno2 to bond0: ifname bond0 not present in cache
 
Adderol said:
Other two servers are Supermicro servers, and I don't believe they share the same network card but I will double check to make sure. They also show their interfaces as enoX.

Real quick, I tried the bond but got the message below. I am still researching it.

Code: 
warning: error while writing to file /sys/class/net/bonding_masters: [Errno 13] Permission denied: '/sys/class/net/bonding_masters'
error: netlink: eno2: cannot enslave link eno2 to bond0: ifname bond0 not present in cache
Click to expand...
FYI, to answer your earlier question, the Supermicro servers have an Intel X552/557-AT card in them.
 
ok, so same range but not exact same type, plus with a different mobo, it could always be some strange driver / incompatibility issue.

As for that error, haven't seen that one before either, could you post the full network/interfaces you made?
And have you tried configuring the (not-vlan'd) bond from the GUI (there are some more failsaves in place there)
 
sw-omit said:
ok, so same range but not exact same type, plus with a different mobo, it could always be some strange driver / incompatibility issue.

As for that error, haven't seen that one before either, could you post the full network/interfaces you made?
And have you tried configuring the (not-vlan'd) bond from the GUI (there are some more failsaves in place there)
Click to expand...

Have not tried creating the bond from the GUI, I will do that now and see what it does. Below is my current interfaces file.


Code: 
# network interface settings; autogenerated
# Please do NOT modify this file directly, unless you know what
# you're doing.
#
# If you want to manage parts of the network configuration manually,
# please utilize the 'source' or 'source-directory' directives to do
# so.
# PVE will preserve these directives, but will NOT read its network
# configuration from sourced files, so do not attempt to move any of
# the PVE managed interfaces into external files!

auto lo
iface lo inet loopback

iface eno1 inet manual

iface eno2 inet manual

auto bond0
iface bond0 inet manual
        bond-slaves eno2
        bond-miimon 100
        bond-mode active-backup
        bond-primary eno2

iface bond0.11 inet manual

iface eno3 inet manual

iface eno4 inet manual

auto vmbr0
iface vmbr0 inet static
    address 192.168.11.26/24
    gateway 192.168.11.1
    bridge-ports eno1
    bridge-stp off
    bridge-fd 0

source /etc/network/interfaces.d/*
 
sw-omit said:
ok, so same range but not exact same type, plus with a different mobo, it could always be some strange driver / incompatibility issue.

As for that error, haven't seen that one before either, could you post the full network/interfaces you made?
And have you tried configuring the (not-vlan'd) bond from the GUI (there are some more failsaves in place there)
Click to expand...

Ok here are the results from creating the bond via the gui (I did not bother with the bond vlan since it failed to create the bond):

Code: 
netlink : error: netlink: eno2: cannot enslave link eno2 to bond0: ifname bond0 not present in cache
TASK ERROR: command 'ifreload -a' failed: exit code 1
 
Ok so basically the same error, just without the (debug) info beforehand.
Looking around for that error, I did find a couple of references (for example [1]), but without a real cause/solution (other then re-installing.)

My best guess at this point is that either something broke during the install, or some weird driver edge-case

If you don't have anything else installed on this system, maybe a proxmox-on-debian install [2] is an option, see if that behaves better.

[1] https://forum.proxmox.com/threads/proxmox-ve-network-not-configurable.92879/
[2] https://pve.proxmox.com/wiki/Install_Proxmox_VE_on_Debian_12_Bookworm
 
sw-omit said:
Ok so basically the same error, just without the (debug) info beforehand.
Looking around for that error, I did find a couple of references (for example [1]), but without a real cause/solution (other then re-installing.)

My best guess at this point is that either something broke during the install, or some weird driver edge-case

If you don't have anything else installed on this system, maybe a proxmox-on-debian install [2] is an option, see if that behaves better.

[1] https://forum.proxmox.com/threads/proxmox-ve-network-not-configurable.92879/
[2] https://pve.proxmox.com/wiki/Install_Proxmox_VE_on_Debian_12_Bookworm
Click to expand...

As you can imagine I have attempted to re-install several times without success on multiple ISO's all done through the idrac virtual media plugin. Next step for me now i guess is to change the method of delivery of the bits to a flash drive or something. Very very strange that the most basic network configs aren't working.

Again, thank you for your time, consideration, and efforts here. It is much appreciated!
 
Not completely the same, but I've before tried to use the virtual iso through IPMI (for our own Supermicro servers) and have had almost nothing but bad experiences with that too, so I just plugged in a couple of USB's with the iso on it in to be done with it.
 
sw-omit said:
Not completely the same, but I've before tried to use the virtual iso through IPMI (for our own Supermicro servers) and have had almost nothing but bad experiences with that too, so I just plugged in a couple of USB's with the iso on it in to be done with it.
Click to expand...

Sadly, I have re-installed from a thumb drive but still face the same problem.

Edit: I just verified the file hash matches what the download site says.

Code: 
someone@somemachine:~$ sha256sum /home/someone/Downloads/proxmox-ve_8.2-2.iso
c96ad84eacbbcef299ab8f407f9602f832abb5ceb08a9aa288c1e1164df2da97  /home/someone/Downloads/proxmox-ve_8.2-2.iso
 
Last edited:
Try the below for a start. Are you not intending to use any of the other 3 ports of this card, eno2-4?
Example: Use VLAN 5 for the Proxmox VE management IP with VLAN aware Linux bridge
https://pve.proxmox.com/pve-docs/pve-admin-guide.html#_vlan_on_the_host
3 options to accomplish what I think you are asking are listed there in the docs.

Code: 
# please utilize the 'source' or 'source-directory' directives to do
# so.
# PVE will preserve these directives, but will NOT read its network
# configuration from sourced files, so do not attempt to move any of
# the PVE managed interfaces into external files!

auto lo
iface lo inet loopback

iface eno1 inet manual

iface eno2 inet manual

iface eno3 inet manual

iface eno4 inet manual

auto vmbr0.11
iface vmbr0.11 inet static
    address 192.168.11.26/24
    gateway 192.168.11.1
    
auto vmbr0
iface vmbr0 inet manual
    bridge-ports eno1
    bridge-stp off
    bridge-fd 0
    bridge-vlan-aware yes
    bridge-vids 2-4094

source /etc/network/interfaces.d/*
 
vesalius said:
Try the below for a start. Are you not intending to use any of the other 3 ports of this card, eno2-4?
Example: Use VLAN 5 for the Proxmox VE management IP with VLAN aware Linux bridge
https://pve.proxmox.com/pve-docs/pve-admin-guide.html#_vlan_on_the_host
3 options to accomplish what I think you are asking are listed there in the docs.

Code: 
# please utilize the 'source' or 'source-directory' directives to do
# so.
# PVE will preserve these directives, but will NOT read its network
# configuration from sourced files, so do not attempt to move any of
# the PVE managed interfaces into external files!

auto lo
iface lo inet loopback

iface eno1 inet manual

iface eno2 inet manual

iface eno3 inet manual

iface eno4 inet manual

auto vmbr0.11
iface vmbr0.11 inet static
    address 192.168.11.26/24
    gateway 192.168.11.1
   
auto vmbr0
iface vmbr0 inet manual
    bridge-ports eno1
    bridge-stp off
    bridge-fd 0
    bridge-vlan-aware yes
    bridge-vids 2-4094

source /etc/network/interfaces.d/*
Click to expand...

First off I want to thank you for your time and consideration of my issue. To help explain my ultimate configuration goals, I will post my interfaces file from another host that is currently up and running with no issues on 8.2.4.

eno1 - vlan11 mgmt
eno2 - vlan11 mgmt
eno3 - VM traffic
eno4 - iSCSI to shared storage platform

I am currently re-installing with the 8.2-2 iSO. I have tried every iSO all the way back to 7.x.x and still face the issue when attempting to create a vlan. I will attempt your method and get back with you.
 

Attachments

  • Screenshot from 2024-09-03 11-56-13.png
    Screenshot from 2024-09-03 11-56-13.png
    143.3 KB · Views: 5
Adderol said:
First off I want to thank you for your time and consideration of my issue. To help explain my ultimate configuration goals, I will post my interfaces file from another host that is currently up and running with no issues on 8.2.4.

eno1 - vlan11 mgmt
eno2 - vlan11 mgmt
eno3 - VM traffic
eno4 - iSCSI to shared storage platform

I am currently re-installing with the 8.2-2 iSO. I have tried every iSO all the way back to 7.x.x and still face the issue when attempting to create a vlan. I will attempt your method and get back with you.
Click to expand...
vesalius said:
Try the below for a start. Are you not intending to use any of the other 3 ports of this card, eno2-4?
Example: Use VLAN 5 for the Proxmox VE management IP with VLAN aware Linux bridge
https://pve.proxmox.com/pve-docs/pve-admin-guide.html#_vlan_on_the_host
3 options to accomplish what I think you are asking are listed there in the docs.

Code: 
# please utilize the 'source' or 'source-directory' directives to do
# so.
# PVE will preserve these directives, but will NOT read its network
# configuration from sourced files, so do not attempt to move any of
# the PVE managed interfaces into external files!

auto lo
iface lo inet loopback

iface eno1 inet manual

iface eno2 inet manual

iface eno3 inet manual

iface eno4 inet manual

auto vmbr0.11
iface vmbr0.11 inet static
    address 192.168.11.26/24
    gateway 192.168.11.1
   
auto vmbr0
iface vmbr0 inet manual
    bridge-ports eno1
    bridge-stp off
    bridge-fd 0
    bridge-vlan-aware yes
    bridge-vids 2-4094

source /etc/network/interfaces.d/*
Click to expand...

Sad to report that the same error message occurs. Screenshot attached.
 

Attachments

  • Screenshot from 2024-09-03 12-32-07.png
    Screenshot from 2024-09-03 12-32-07.png
    288.1 KB · Views: 3
Are eno1-2 physically connected to a switch? If so, will you bond them together at the switch and in Proxmox or leave them bridged as in the example attachment?
 
vesalius said:
Are eno1-2 physically connected to a switch? If so, will you bond them together at the switch and in Proxmox or leave them bridged as in the example attachment?
Click to expand...
1.) Yes
2.) Leave them bridged

if you have a recommendation I would very much like to know it. Always room to learn :)

EDIT: Also, I just completed a base Debian install and installed ifupdown2 as well. I was able to create a vlan interface with no issues it seems on this distro.
 
With that additional info I would amend my suggestion:

Code: 
# please utilize the 'source' or 'source-directory' directives to do
# so.
# PVE will preserve these directives, but will NOT read its network
# configuration from sourced files, so do not attempt to move any of
# the PVE managed interfaces into external files!

auto lo
iface lo inet loopback

iface eno1 inet manual

iface eno2 inet manual

iface eno3 inet manual

iface eno4 inet manual

auto vmbr0.11
iface vmbr0.11 inet static
    address 192.168.11.26/24
    gateway 192.168.11.1
    
auto vmbr0
iface vmbr0 inet manual
    bridge-ports eno1 eno2
    bridge-stp off
    bridge-fd 0
    bridge-vlan-aware yes
    bridge-vids 2-4094

auto vmbr1
iface vmbr1 inet manual
    bridge-ports eno3 
    bridge-stp off
    bridge-fd 0
    bridge-vlan-aware yes
    bridge-vids 2-4094
#VM Networks

source /etc/network/interfaces.d/*
 
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom