Help with LACP NIC Bonding via command line

[Snipe]

Hello,
I have a server that I am setting up in a Data center, I installed proxmox VE, but for them to allow me to access to the network, and hence the web ui, I need to have network enabled... But for them to enable the network they need me to setup the server's two fiber ports to behave as one using LACP bonding.

I've never done this, let alone having to do this via terminal. I've looked around and found a couple of threads discussing the need to modify the interface file and such, but it seems in all of them I need a local ip address from the datacenter right? (Right now I am connecting to the server via iDRAC virtual console, so perhaps I can use that IP?)

Anyway I wanted to ask if there is a good reference for me to follow in what I need to do. I read a few tutorials
https://cloudibee.com/2020/08/15/network-bonding-modes/
https://sourceexample.com/en/fc2750d5aa9564d834cc/
https://docs.rackspace.com/blog/lacp-bonding-and-linux-configuration/

and saw a couple of posts here with some sample codes of what they do, but I am worried of copying and pasting code I don't understand so if anyone can point me in the right direction I would greatly, greatly, appreciate it.

I know linux ok, but my network foo is weak. before this the biggest server I had setup was a raspberry pi

 

[vesalius]

Is this proxmox 7, uses ifupdown2, or proxmox 6, uses ifupdown?

https://pve.proxmox.com/wiki/Network_Configuration#_linux_bond

2 examples using bond-mode 802.3ad (this is LACP) at the link above. I usually use bond-xmit-hash-policy layer3+4 instead of bond-xmit-hash-policy layer2+3 from those 2 examples, but should work regardless. Edit the /etc/network/interfaces file with nano. Use the 2 fiber port interfaces names (should be listed toward the top of this same file) as the bond-slaves save those files changes then from the CLI use a ifreload -a to bring up the new bond*.

You can create a bond without an IP listed for it. If the datacenter then gives you an IP you can then add that to the bond* directly or to a proxmox linux bridge vmbr* that uses the bond* above. Examples of both of those options are there in the linked section.

 

[Snipe]

Thank you for that reference! I went ahead and did what you said and will wait to hear tomorrow from the datacenter to see if it works now. If not I will try the eno1np0 and will try 3+4 too. By the way this is Proxmox 7.




Thanks again for your help! Will report back on the outcome.

 

[vesalius]

Thank you for that reference! I went ahead and did what you said and will wait to hear tomorrow from the datacenter to see if it works now. If not I will try the eno1np0 and will try 3+4 too. By the way this is Proxmox 7.


View attachment 29998

Thanks again for your help! Will report back on the outcome.

I suspect the 2 fiber ports will be eno1np0 and eno2np1, but that is just my best guess. You could use ethtool eno3 and ethtool eno1np0 to see what speeds (supported link modes) each of those interfaces support. Then use the 2 that match the expected speed of your fiber ports. The correct 2 should also list Supported ports: [ FIBRE ]

https://www.cyberciti.biz/faq/howto-determine-ethernet-connection-speed/

 

[Snipe]

So I wanted to provide an update that I saw your message Monday morning and updated the fiber ports to eno1np0 and eno2np1 and you were indeed correct those were the fiber ports. The datacenter said the bonding had worked. Then I had some issues because of dhcp for ip assignment vs static. So I tweaked around the interfaces and this is what I ended up with.



It all seems to work now I have network access. Which is great. So thank you very much for your help.
I do get spam on the console sometimes that I had to disable about the NIC bond being down



but I mean it's currently working so I just suppressed the messages and will play around with the proxmox web ui now that I finally have access to it.
Thanks again for your help, I couldn't have done it without you!

 

[vesalius]

So I wanted to provide an update that I saw your message Monday morning and updated the fiber ports to eno1np0 and eno2np1 and you were indeed correct those were the fiber ports. The datacenter said the bonding had worked. Then I had some issues because of dhcp for ip assignment vs static. So I tweaked around the interfaces and this is what I ended up with.

View attachment 30112

It all seems to work now I have network access. Which is great. So thank you very much for your help.
I do get spam on the console sometimes that I had to disable about the NIC bond being down

View attachment 30113

but I mean it's currently working so I just suppressed the messages and will play around with the proxmox web ui now that I finally have access to it.
Thanks again for your help, I couldn't have done it without you!

Great, glad it is working.

Looks like only eno1np0 is down and stays down. Could be the switch it is plugged into is misconfigured. eno2np1 must be up and working though. I guess it could be port flapping, but I am no expert or even experienced there. If this is a PCIe nic does it have the latest firmware?

 

[flixhost]

Is your fiber (tip) actually clean?
We've had some hustle and bustle in the past with intel x520 cards and fs.com transceivers.
If there is just some dirt, you're basically screwed.

To fix that I bought myself the handy dandy cleaner thing for SFP modules.