Pulling network traffic data from RRD

[stavros]

Hello all. We have a number of VMs on a Proxmox cluster and are very interested in getting network traffic data for specific time periods (usually per month). Using the proc/sysfs system is not an option, since traffic data restarts if a vm is live-migrated (and for specific reasons, we do that relatively often). I know that in previous versions of Proxmox this was not an option, but I see that in 2.x the rrdcached daemon is used.

As far as I can see, this daemon caches the RRD data and moves them along with the configuration of the vm when it is live migrated, and it already stores the info we need (network i/o). I was therefore wondering where this data resides, and if it is possible to read it using RRDtool from another host.

 

[dietmar]

RRD data is replicated to all nodes. You can use rrdtools to extract data from /var/lib/rrdcached/db/...

 

[stavros]

RRD data is replicated to all nodes. You can use rrdtools to extract data from /var/lib/rrdcached/db/...

Hey, thanks for the info. I was able to read the info from it, unfortunately it looks like individual network interfaces are not monitored, so I will have to find another way. Has anybody had experience with migration-compatible monitoring of vm interfaces without the use of SNMP on the guest?

 

[huseinb]

Hey, thanks for the info. I was able to read the info from it, unfortunately it looks like individual network interfaces are not monitored, so I will have to find another way. Has anybody had experience with migration-compatible monitoring of vm interfaces without the use of SNMP on the guest?

Can you tell me how you did this, please?

EDIT: Why can't rrdtool write the total on the bottom of the graph it generates?

 

[abuhome]

Use cacti with snmp. Than you have all statistics from the interface incl inbound/outbound traffic.

 

[stavros]

Can you tell me how you did this, please?

EDIT: Why can't rrdtool write the total on the bottom of the graph it generates?

I assume by your edit that you already found you can do it with rrdtool...in case you haven't, basically install rrdtool, use "rrdtool info /var/lib/rrdcached/{pve-vm}/{vmid}" where {pve-vm} and {vmid} and it will show you all the data it stores. You can then use cacti to load and graph it. Indeed it also does not write totals on the graph, but cacti can produce total reports so I am not bothered too much with it. Would be nice to have in a later version of Proxmox though.

 

[stavros]

Use cacti with snmp. Than you have all statistics from the interface incl inbound/outbound traffic.

abuhome, if this is a reply to huseinb, you are right. However, it does not solve my problem since if you live migrate the vm to a new host, cacti still tries to query the interface on the old host, where it does not exist anymore. And I do not think cacti has a way of retrieving the all the interfaces from different hosts and merging them, does it?

 

[abuhome]

@stavros: If you live migrate a vm, the interface change. Thats true. But that was not my approach. I monitor all my proxmox server and vm's with cacti and snmp. If i do a live migration ill change "by hand" the interface in cacti so that i get the snmp data from the new interface. But to be honest. Live migration in my environment happens the last 6 months 2 times.

 

[stavros]

@stavros: If you live migrate a vm, the interface change. Thats true. But that was not my approach. I monitor all my proxmox server and vm's with cacti and snmp. If i do a live migration ill change "by hand" the interface in cacti so that i get the snmp data from the new interface. But to be honest. Live migration in my environment happens the last 6 months 2 times.

I agree with all you said. It is simply that because of our (currently) small cluster we sometimes do live migrations to accomodate new vms so we do them once a week or so, which makes the manual process on cacti more cumbersome. If the RRD data would accomodate traffic per interface though, our problems would be solved.

Thanks for your reply.

Stavros