Mirroring incoming/outgoing traffic to a ntopng (security analysis) VM on same node - best practices

[victorhooi]

Hi,

I'm setting up a single Proxmox node with multiple Windows VMs. The physical server has multiple onboard NICs (i.e. on the motherboard).

I'd like to use ntopng to monitor all incoming/outgoing traffic from those Windows VMs.

Ideally, I'd like to run a Linux instance with ntopng on the same Proxmox node.

The two ways I can see to do this are:

1. Configure Proxmox somehow to mirror traffic within itself, and pass outgoing traffic from those Windows VMs back to another VM. Is this possible?
2. Setup port mirroring on the physical switch, and plug that back into a second NIC on the server

For 1. I saw this earlier post which talks about either setting up the bridge as a hub, or using tc. However, is there any performance advantage of doing it this way, versus mirroring on the physical switch and plugging it back in? It's not clear how I would go about actually doing this either.

For 2. I can setup mirroring on the switch - however, can I pass through that NIC directly to the ntopng Linux VM? I saw this wiki article but it's not clear if this works with on-board motherboard NICs. Also it says at top "PCI passthrough is an experimental feature in Proxmox VE" - is this still the case in late 2018?

Thanks,
Victor

 

[victorhooi]

I'm keen to take another shot at this. What do you guys think of this?


I have a main Proxmox server that runs all my normal VMs.

Then I have a separate server, also running Proxmox that will be used for network analysis.

What I'm thinking of is - I will have one cable for the normal VM traffic (and to manage Proxmox). Then we'll have a second cable for sniffing traffic - the network port can be setup as a SPAN port (that will mirror all network traffic).

Should this be plugged into a separate NIC, that we then do PCIe pass-through to Proxmox? Or will just a normal network interface that we assign to a VM work here?

 

[Stoiko Ivanov]

For 1. I saw this earlier post which talks about either setting up the bridge as a hub, or using tc. However, is there any performance advantage of doing it this way, versus mirroring on the physical switch and plugging it back in? It's not clear how I would go about actually doing this either.

This might have a negative impact on the performance (I know about the way by setting the mac-aging to zero with '/etc/network/interface' or with the `bridge` util) - because then the bridge behaves as a hub an all traffic is broadcasted to all member-interfaces.

The version with the SPAN-port sounds reasonable - if possible I would use a dedicated NIC with PCI-passthrough for the SPAN-port (less overhead and translations - you want to get the traffic as close to the wire as possible).

I hope this helps!

 

[victorhooi]

I'm going to try the SPAN port, based on your recommendation =).

I don't have a separate NIC available.

Hence, I have a single server running Proxmox, with two 10Gbase-LR cables going into it.

The plan is to use the first network connection as the main VM bridge interface, and the second network connection will be connected to the SPAN port on the switch

My question is - how do I pass that second port into the VM, and have it work for sniffing all the traffic? How do I add, and configure that interface in Proxmox?

 

[Stoiko Ivanov]

My question is - how do I pass that second port into the VM, and have it work for sniffing all the traffic? How do I add, and configure that interface in Proxmox?

Use PCI(e)- passthrough - see our reference documentation: https://pve.proxmox.com/pve-docs/chapter-qm.html#qm_pci_passthrough

I hope this helps!

 

[victorhooi]

Thanks Stoiko for getting back! =)

However, *both* ports are plugged into the same NIC card. (Intel X520-DA2, I believe)

(The machine is remote - it's actually on another continent, lol - I'm in Australia, machine is in US).

Can I still use PCI passthrough in that case?

I assumed I'd still need port 1 available to Proxmox, since that will be the management interface, and normal VM traffic.

 

[Stoiko Ivanov]

both ports are usually 2 separate pci-devices - whether or not you can pass one through depends on whether the pci-devices are in separate IOMMU groups - the docs should explain that well.

apart from that you could also, depending on the network-card, consider using SR-IOV and passing one virtual function (a virtual network-port) to the VM - you should find quite a few tutorials online for this

I hope this helps!