Using multiple ethernet ports for VMs as opposed to one?

[liamlows]

Hey guys,

So a couple weeks ago I decided to start hosting some of my servers that were on a rented VPS at home via a nice homelab solution. I managed to pickup a used poweredge r430 with some nice internals for a couple hundred bucks and its worked like a charm so far (minus the initial driver updates ) and I'm using proxmox for the first time as it seems like an awesome hypervisor!

Currently I am running the interface installed on RAID 5 across 4x600GB drives (configured via onboard H730) and have a few vms running (media server, web servers, game servers). I was poking around the network settings (since the server has 4 gigabit ethernet ports off the back) and noticed that they are listed in proxmox but 3 are not being used, they currently have no cables plugged into them.

I was wondering if there is or could be any benefit to using the other 3 ports as well? Also, how I would go about setting them up? Right now I could see it being useful for the following reasons (please correct me if I'm wrong):

• using dedicated ethernet ports for specific VMs to increase performance to clients (in my case maybe the media server as it can have up to 3 clients on at a given time)
• having the ports work together to improve general network performance for the entire PVE. Basically plugging in the ports and having one "bridge" (probably the wrong term) pool all them together for use across all VMs.
• seperating at least one port for panel access and using the other(s) to run the VMs.

Any advice is appreciated as I am still learning and reading documentation on all of it (proxmox, r430, etc.). For right now I have no issues really using just one, I was just curious if I could be getting more out of my setup!

Thanks guys!

P.S. see attached for screenshot of my current network config

 

[spirit]

you need to create a bond with multiple enoX interfaces, then put the bond in vmbr0

 

[vshaulsk]

You need a switch that supports bonding

 

[liamlows]

you need to create a bond with multiple enoX interfaces, then put the bond in vmbr0

Awesome! So would I essentially just create a linux bond with all the eno ports as slaves (also set up IP information) and then edit the current bridge (vmbr0) to use bond0 instead of eno1? (see attached photos for config)
Also, should I be changing the ip address used by the bridge then since the bond will be using that IP? or is it fine to leave it?

Thanks again for all the help!

 

[spirit]

you can't put an ip on a interface, if this interface is in a bridge.
you need to set interface on bridge in this case.

 

[liamlows]

you can't put an ip on a interface, if this interface is in a bridge.
you need to set interface on bridge in this case.

Im not sure I quite understand. What would I need to add or change from the above? Thanks!

 

[spirit]

Im not sure I quite understand. What would I need to add or change from the above? Thanks!

from your last screenshot, simply remove ip address + gateway from the bond interface, and keep them in the bridge.

 

[liamlows]

from your last screenshot, simply remove ip address + gateway from the bond interface, and keep them in the bridge.

Awesome, I'll give that a try. Thanks!

 

[liamlows]

from your last screenshot, simply remove ip address + gateway from the bond interface, and keep them in the bridge.

So I tried it and everything was fine with proxmox but the VMs no longer had access to the internet, I assume there was additional configuration needed there maybe? Do you by any chance might know whats up?
Thanks!

 

[sammybwonder]

Hi, @liamlows. I find myself looking for answers to the same questions that you asked in your original post. The replies that came in are all very helpful, but no one seemed to mention the benefit of using more than 1 port.

Its been a while since you posted the question but I was hoping maybe you managed to find an answer?

My very basic Proxmox server uses an integrated nic (built into the motherboard). Its starting to act up so I'm on the hunt for a new dedicated card and trying to figure out whether to get a card with just 1 port or a card with multiple ports (2 or 4).

I know the fact that I'm even asking suggests that I probably don't have a need for multiple ports, but I see people with this setup all the time and it would be nice to know why they do that.

Thanks in advance.

 

[Pierre-Yves]

Your question is : what is the pros and cons of network bond ?

 

[sammybwonder]

Your question is : what is the pros and cons of network bond ?

Ahhhh, ok. Thank you

 

[Pierre-Yves]

You want quick explanations ?

 

[sammybwonder]

You want quick explanations ?

Your first answer has pointed me in the right direction to find the answers myself, but yes, if you don't mind, a quick explanation would be appreciated. It will help future searchers I think. For instance, when I did a Google search for "Using multiple ethernet ports proxmox" - this was the first result

 

[Universal4]

Multiple NICs do not always mean bonding the cards.

Many people that use Proxmox use it to run VM for clients of such services as web hosting, application hosting and various other service.

In many of those cases, they will assign multiple nics to different subnets, such as eno1/vmbr0 to public, eno3/vmbr1 to private (like a 10.x.x.x) for things like access to NAS or maybe sql/mysql, and they might use the third NIC for the corosync traffic, etc. The possibilities are only limited in scope to the network design based on needs.

There are a number of reasons why this sometimes makes sense, such as not exposing the NAS, sql or other application server to the public in any way

(Too bad the development folks have never seen how helpful it would be to provide the summary graphs in independent graphs since the host software does a fantastic job of keeping all this traffic separate, except in the graph which would help pin down which subnet could possibly have an issue if one arises.)

I also have considered bonding additional cards on the certain subnets, but so far have not found the need, but I often check back to this forum on a regular basis to read the questions, thoughts, and case histories of ways others make the best use of Proxmox to see if anything ever changes my mind on that.

 

[pvps1]

one usecase: bonding is the way to get redundancy
in production scenarios you would have your nics connected to different switches (within a stack) so parts can fail or reboot (e.g. for firmware upgrades) without interruption of services

 

[Universal4]

Gee, then I guess if a broadcast storm happened, the graphs for each individual card would show which card it is, except Proxmox does the math and adds them together on a single graph.

 

[Pierre-Yves]

bonding permit redundancy
bonding permit add bandiwith of several network cards
but bonding is formerly a choice of architecture taking in account the network topology, SAN, Wan connectivity, VLans, security, number and fonctions of VMs and CTs ...
Designing Proxmox complex architecture is a huge work !