Setting up VM with two NICs, to act as bridge?

[victorhooi]

I'm looking at installing Ntopng Edge in inline mode on Proxmox 5.2

My server has one Mellanox ConnectX-3 card, and four on-board NICs.

I was thinking of using the Mellanox card for the management interface, and then setting up a VM with two of the NICs, to act in pass-through (or inline) mode.

Is there a way to dedicate those two NICs to that VM, and have it operate in this way?

 

[Richard]

I was thinking of using the Mellanox card for the management interface, and then setting up a VM with two of the NICs, to act in pass-through (or inline) mode.

For passthrough see https://pve.proxmox.com/wiki/Pci_passthrough

Note that passthrough is experimental - may work but no guarantee.

Is there a way to dedicate those two NICs to that VM, and have it operate in this way?

You can assign the NICs to a bridge (for each of them a different bridge, don't need an IP address in the host) and then assign the VM's NICs to those bridges.

 

[victorhooi]

I'm sorry, but I'm still finding this really confusing =(.

If we use PCI passthrough fo the NICs - do we still need to assign them to a bridgein Proxmox?

Or are they essentially "invisible" to Proxmox?

There are two separate ways we were thinking of setting this up:

First, for other physical hosts plugged into a switch, the idea is to run Proxmox with ntopng Edge in a VM, and have all network traffic pass through that:

Physical hosts -> Network Switch -> ntopng Edge (running under Proxmox) -> Router (WAN)

(Not sure how to get VMs on Proxmox to also go through this - although they would be using separate NICs on the Proxmox host)

Secondly, for other VMs on the same Proxmox host - is there a way to have their traffic pass through the ntopng Edge VM, before passing out to WAN?

 

[Richard]

If we use PCI passthrough fo the NICs - do we still need to assign them to a bridgein Proxmox?

No

Or are they essentially "invisible" to Proxmox?

Yes

Secondly, for other VMs on the same Proxmox host - is there a way to have their traffic pass through the ntopng Edge VM, before passing out to WAN?

Yes - assuming you use bridges (and not passthrough) e.g.:

* one bridge (vmbr0) connected to WAN (Router)

* one bridge (vmbr1) connected to VMs with applications

* Edge VM ist connected to both vmbr0 and vmbr1

Packet from a VM to WAN will be transported as follows:

VM-virtual_NIC -> vmbr1 -> virtual_NIC-EdgeVM-virtual_NIC -> vmbr0 -> physical_NIC-WAN-connection

 

[spirit]

I think you also need a bridge inside the vm, if you really want it in transparent (layer2) mode.


Another way could be to install Ntopng Edge directly on the proxmox host, configure it with nfqueue mode,
then configure proxmox firewall ips
(https://pve.proxmox.com/wiki/Firewall, see suricata ips section, it's the same config)

 

[victorhooi]

@spirit - To clarify, when you ay you need a bridge in VM - this only applies when you're using ntopng Edge inside a VM, to filter for other VMs, right?

As in, you don't need this bridge if you are simply passing through the NIC direct to the VM, right?

The nfqueue and Proxmox firewall sounds interesting - I can't seem to find much on Ntopng Edge and nfqueue mode. Do you have any docs on this?

 

[spirit]

@spirit - To clarify, when you ay you need a bridge in VM - this only applies when you're using ntopng Edge inside a VM, to filter for other VMs, right?

I don't known how Ntopng Edge is working, but it seem that it's doing it from it's gui
https://www.ntop.org/guides/nedge/bridging.html

(so you only need to have both interfaces in the vm, and ntppng do the bridge internaly in the vm)

As in, you don't need this bridge if you are simply passing through the NIC direct to the VM, right?

no, it's not related. You can passthrough nic directly in the vm if you want.

The nfqueue and Proxmox firewall sounds interesting - I can't seem to find much on Ntopng Edge and nfqueue mode. Do you have any docs on this?

I only have seen this
https://www.ntop.org/guides/ntopng/cli_options.html

Available interfaces (-i <interface index>):
...
13. nfqueue


So it seem possible to make it listen to a nfqueue , and then send traffic from iptables to this nfqueue.