Advice for Traffic Accounting (bytes-in, bytes-out) on Bridge Ports or VXLAN for many VMs

[eugenevdm]

Our network consists mostly of 10 Gbit/s switched backbones to the internet.
On the network we have a cluster of Proxmox VM hosts with around a 100 VMs.

We have the need to do traffic accounting, per VM, in the most efficient manner.
We don't need to count protocol traffic flowing via internet, but merely the bytes-in and bytes-out.

My switch specialist recommended VXLAN port accounting instead of Netflow, as Netflow appears to be very resource heavy when you're doing bulk.

I need to do this en-mass to assist with network balancing - we need to understand which clients are using a lot more gigabytes/terabytes than others so that we can route their traffic properly.

I am open to using any tools. I'm happy to use Zabbix, Nagios, Cacti, LibreNMS, etc.

I just need basic traffic accounting for many VMs. Any tips will be appreciated.

 

[whitemeercat68]

If you have a dedicated interface per-vm on the upstream host/gw (e.g. a sub-interface). The interface will be exposed via SNMP. And you could use any monitoring solution to moniter on per-interface basis. (e.g. https://docs.librenms.org/Extensions/Billing-Module/)

On Proxmox VE. Does your VMs not have a tap interface attached you could monitor the counters on?

tap<[SIZE=14px][FONT=Open Sans]VM_ID[/FONT][/SIZE]>i<[SIZE=14px][FONT=Open Sans]INT_ID[/FONT][/SIZE]>

• VM_ID - PVE assigned ID to the VM in the cluster
• INT_ID - The `netX` number as seen in the PVE Web-GUI.

 

[eugenevdm]

Hi @whitemeercat68 ,

Thank you! You reminded me what a great product LibreNMS so I revived an old installation. It turns for now at least LibreNMS perfectly suits my need. As an aside I also use PRTG which has exceptional reporting and push (incessant) notifications.

In summary:

• LibreNMS already has a Proxmox interface: https://docs.librenms.org/Extensions/Proxmox/
• The extension run this script:

root@proxmox-ve-host-X:~# /usr/local/bin/proxmox
<<<app-proxmox>>>
cluster
101/net0/1651196487/308551337/mail.example.com
104/net0/9336041233/3055200977/mssql./mail.example.com
108/net0/3126480056/110328490/example.com
118/net0/8420242931/1998319860/accounts.example.com
121/net0/8648026247/61587408740/host2.example.com

• These stats are then viewable as an App:

This is giving me the power to quickly see some of our busy users and think about the next steps. I guess technically the next amazing thing would be percentile integration but that can wait.