[SOLVED] Logging outgoing IP traffic

[jordaneb]

Hello,

I have various bits of software running in virtual machines, 1 running Windows 10 and the rest Debian 10.

I'd like to monitor what's going on so that I can occasionally manually audit what IP addresses my virtual machines are communicating with from outside of the virtual machine so that it's less likely a bit of rogue software could operate without me knowing.

I would like to log every single IP address communicated with (preferrably which protocol too) and which VM initiated the connection.

I had a look at enabling logging in the firewall settings but from the looks of things it doesn't include all outgoing connections.

What would be the best solution to obtain this information and would there be any negative impact on performance?

Thanks

 

[sg90]

Hello,

I have various bits of software running in virtual machines, 1 running Windows 10 and the rest Debian 10.

I'd like to monitor what's going on so that I can occasionally manually audit what IP addresses my virtual machines are communicating with from outside of the virtual machine so that it's less likely a bit of rogue software could operate without me knowing.

I would like to log every single IP address communicated with (preferrably which protocol too) and which VM initiated the connection.

I had a look at enabling logging in the firewall settings but from the looks of things it doesn't include all outgoing connections.

What would be the best solution to obtain this information and would there be any negative impact on performance?

Thanks

Do you want to know just VM -> Outside world or any traffic leaving the VM Interface even if locally?

 

[jordaneb]

Do you want to know just VM -> Outside world or any traffic leaving the VM Interface even if locally?

I would just like to know VM -> Outside world. I think I'm happy to assume that connections on local interfaces is fine for now.

 

[sg90]

I would just like to know VM -> Outside world. I think I'm happy to assume that connections on local interfaces is fine for now.

Then its probably something youd want to setup on your main gateway / firewall, and not within Proxmox itself.

 

[jordaneb]

Sorry I forgot to mention that I don't have access to any networking equipment outside of the Proxmox host as they're both rented bare metal machines.

I would absolutely agree though if this were on a network I could control myself.

My thinking was that at the very least monitoring these connections on the host meant it would be somewhat unlikely that a VM could remain under the radar unless the hypervisor were compromised (in which case I'd have bigger problems.)

 

[jordaneb]

So I've come up with a solution which I'm happy with at the moment. Just in case this is helpful to anyone else I wanted to post this here.

I've gone with tcpdump since it's installed by default on Proxmox.

I've created a systemd service file called /etc/systemd/system/tcpdumpd.service with the following contents:

[Unit]
After=network.target

[Service]
Restart=always
RestartSec=30
ExecStartPre=/bin/mkdir -p /var/log/tcpdumpd/
ExecStart=/bin/bash -c '/sbin/tcpdump -i enp35s0 -n ip | cut -d \' \' -f 3-5 | tr -d \':\' > /var/log/tcpdumpd/connections.log'
ExecStop=/bin/kill -s QUIT $MAINPID

[Install]
WantedBy=multi-user.target

If we've got a VM with the IP `192.168.1.20` and a web server not on the Proxmox host with the IP `192.168.1.100` and the VM performs an HTTPS request, log entries like this will be created:

192.168.1.20.49054 > 192.168.1.100.443
192.168.1.100.443 > 192.168.1.20.49054

So if I run run this through sort | uniq I can get an idea of what VMs are up to and (hopefully) spot whether any software I'm running in these VMs is phoning some foreign server somewhere.

I've used logrotate to keep the log files in order with the following config:

/var/log/tcpdumpd/*.log {
        missingok
        size 100M
        rotate 30
        compress
        notifempty
        copytruncate
}

The files do get pretty large, but once logrotate compresses them they immediately take up very little space.

 

[supremasi]

So I've come up with a solution which I'm happy with at the moment. Just in case this is helpful to anyone else I wanted to post this here.

I've gone with tcpdump since it's installed by default on Proxmox.

I've created a systemd service file called /etc/systemd/system/tcpdumpd.service with the following contents:

[Unit]
After=network.target

[Service]
Restart=always
RestartSec=30
ExecStartPre=/bin/mkdir -p /var/log/tcpdumpd/
ExecStart=/bin/bash -c '/sbin/tcpdump -i enp35s0 -n ip | cut -d \' \' -f 3-5 | tr -d \':\' > /var/log/tcpdumpd/connections.log'
ExecStop=/bin/kill -s QUIT $MAINPID

[Install]
WantedBy=multi-user.target

If we've got a VM with the IP `192.168.1.20` and a web server not on the Proxmox host with the IP `192.168.1.100` and the VM performs an HTTPS request, log entries like this will be created:

192.168.1.20.49054 > 192.168.1.100.443
192.168.1.100.443 > 192.168.1.20.49054

So if I run run this through sort | uniq I can get an idea of what VMs are up to and (hopefully) spot whether any software I'm running in these VMs is phoning some foreign server somewhere.

I've used logrotate to keep the log files in order with the following config:

/var/log/tcpdumpd/*.log {
        missingok
        size 100M
        rotate 30
        compress
        notifempty
        copytruncate
}

The files do get pretty large, but once logrotate compresses them they immediately take up very little space.

Can you give step by step to do it?
I'm new in proxmox and linux