4 Port NIC card : 3 port on a bridge and 1 port passthrough - not possible ?

[gianry]

I've a 4 port Realtek Nic card . I've tried to setup three port on a bridge and 1 port as passthrough with static address.
with this configuration it is not possible to reach the Proxmox node neither with port in bridge than port in passthrough .

does it possible or it is an hardware limitation ?

thanks

 

[gfngfn256]

Have you checked what IOMMU groups they are in:

#check all IOMMU groups with:
for d in /sys/kernel/iommu_groups/*/devices/*; do n=${d#*/iommu_groups/*}; n=${n%%/*}; printf 'IOMMU group %s ' "$n"; lspci -nns "${d##*/}"; done

#check ethernet only with:
for d in /sys/kernel/iommu_groups/*/devices/*; do n=${d#*/iommu_groups/*}; n=${n%%/*}; printf 'IOMMU group %s ' "$n"; lspci -nns "${d##*/}"; done | grep Ethernet

 

[gianry]

Have you checked what IOMMU groups they are in:

IOMMU group 2 03:00.0 Ethernet controller [0200]: Realtek Semiconductor Co., Ltd. RTL8125 2.5GbE Controller [10ec:8125] (rev 05)
IOMMU group 2 04:00.0 Ethernet controller [0200]: Realtek Semiconductor Co., Ltd. RTL8125 2.5GbE Controller [10ec:8125] (rev 05)
IOMMU group 2 05:00.0 Ethernet controller [0200]: Realtek Semiconductor Co., Ltd. RTL8125 2.5GbE Controller [10ec:8125] (rev 05)
IOMMU group 2 06:00.0 Ethernet controller [0200]: Realtek Semiconductor Co., Ltd. RTL8125 2.5GbE Controller [10ec:8125] (rev 05)
IOMMU group 8 00:1f.6 Ethernet controller [0200]: Intel Corporation Ethernet Connection (7) I219-LM [8086:15bb] (rev 1

 

[gfngfn256]

So as you can see - they are all in IOMMU group 2 (together with their on-board packet controllers). This is as expected from a single (physical) PCI card. So it would appear you cannot selectively passthrough a single NIC/s. On other PCI devices there are sometimes some workarounds - but I'm guessing on a single PCI card - this is not going to be possible.

Also note that not all PCI devices allow passthrough in general.

 

[gianry]

ok,
I tried the passthrough for all 4 port and it works in CHR routerOs .

How should I set up the PVE node to link on the LAN all the other VMs ?

 

[LnxBil]

How should I set up the PVE node to link on the LAN all the other VMs ?

Do you have a VLAN and LACP capable switch? If so, just setup it up and do everything else via VLAN. Much easier setup and you have theoretical 10 GBE throughput.
If not, create another bridge with only the one NIC you wanted to seperate and configure it in your VM.

 

[gianry]

If not, create another bridge with only the one NIC you wanted to seperate and configure it in your VM.

Hi, thanks for your reply
No, I don't have a switch .I don't need to separate the LAN.

My idea was to use the multiport Nic in pass through in a Router VM like CHR or Vyos but the other VMs should be on the same lan of the router.

 

[gfngfn256]

Assuming you only have the 4-port NICS card (no other NICs on server), I'd do the following:

1. Don't passthrough the NIC card at all.

2. Create bridge VMBR0 from port-1 (on the card) for main PVE host node access (with its address etc. correctly set).
Connect this port physically to your router.

3. Create bridge VMBR1 from port-2 (on the card) for other VMs LXCs traffic.
Connect this port physically to your router.

4. Create bridge VMBR2 from port-3 (on the card) for WAN side of routerOS VM.
Connect this port physically to your router.

5. Create bridge VMBR3 from port-4 (on the card) for LAN side of routerOS VM.

This way you can connect anything you wish to run from the routerOS by connecting it to the LAN side of it.



I'm not exactly sure what you require & I have also never run a config like this.