Internet Bandwidth Optimization

[malabida]

Hi,

I have the following dilemma: I have a proxmox node with several VMs with the network through a Bridge (vmbr0) with virtual MACs provided by the internet provider. These VMs have to communicate with each other consuming quite a lot of bandwidth, but the traffic goes out to the internet and back in through the network interface (enp33s0f0). This is consuming public bandwidth and saturating my contracted speed.

I have tried with `ip route` to route the traffic so that it does not leave vmbr0 but to no avail.

Does anyone have any idea how to do it?

Thanks.

 

[gurubert]

Add another internal bridge (vmbr1 with no physical ports). Attach a new vNIC to each VM and configure a private network. All the VMs then can talk to each other through that connection.

 

[malabida]

I want to do this transparently without adding a new adapter. I know that if I put /24 if they communicate with each other directly. Is there any way to do it without another adapter?

 

[_gabriel]

you can add a second ip, a local ip, within guest.

 

[gurubert]

What is the current IP and netmask inside the VMs and why do they send traffic through "the Internet" if they want to communicate directly?

 

[malabida]

I have several public IP addresses assigned to VMs and LXC containers. The problem is when a VM which is on subnet: 5xx.xxx.xxx.xxx.152/29 wants to communicate with an LXC container on subnet: 1xx.xxx.xxx.xxx.72/29. Being different subnets the traffic goes out through vmbr0 directly to the physical adapter, (enp1s0f0) to the internet and back in through the internet. Using public bandwidth. What I would like is to route at proxmox level the traffic back to the vmbr0 interface.

 

[malabida]

I have managed to do it with Open vSwitch. Replacing vmbr0 (bridge) with an Open vSwitch virtual adapter with the same configuration and applying flow rules to redirect the traffic locally by changing the destination mac.

ovs-ofctl add-flow vmbr0 priority=100,ip,nw_dst=<vm1-ip>,actions=mod_dl_dst:<vm1-mac>,mod_nw_dst:<vm1-ip>,output:NORMAL
ovs-ofctl add-flow vmbr0 priority=100,ip,nw_dst=<vm2-ip>,actions=mod_dl_dst:<vm2-mac>,mod_nw_dst:<vm2-ip>,output:NORMAL

As I am not an expert in Open vSwitch either, if someone could confirm me that this way is viable for a production environment it would be of great help.

 

[malabida]

Bump