Can't communicate with guest using certain NICs

C

chrisb81

New Member
Mar 16, 2021
6
0
1
43
Hi all,

Sorry - first post on here.. I've tried to figure this out, but I'm totally stumped.

I have a server I'm using as a testbed for some VMs, just testing things out using Proxmox at home. The machine only had one on-board NIC, and a Broadcom gigabit PCIE NIC. All worked straight away out-of-the-box.

Now, I added a Realtek 8111/8168/8411 Quad port gigabit PCI-E card, not realising this card doesn't seem to be well supported. After multiple false starts, I managed to get this driver compiled and added to the kernel: https://github.com/mtorromeo/r8168

Now, the NICs all show up in the Proxmox GUI - it all looks great. However, when I try to use any of these new NICs with a guest (through a Linux Bridge), the NIC appears on the guest but it basically has no connectivity whatsoever. I tried giving the bridge itself an IP address, and confirmed I could ping the bridge on my network, so it seems like the card is working, but it's like no network traffic actually passes through from the host to the guest, or vice-versa. I'm baffled, I feel like I must have forgotten something on the setup, but my two working bridges are set up identically, but the four I've added simply do nothing.

Is it due to the R8168 driver? I think I probably should have bought a I350 card instead, but now I've got this hardware and it seems so close to working, I'm really keen to try to figure this out.

lspci | grep -i eth:

Code: 
00:1f.6 Ethernet controller: Intel Corporation Ethernet Connection (2) I219-V
03:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 0c)
04:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 0c)
05:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 0c)
06:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 0c)
09:00.0 Ethernet controller: Broadcom Limited NetXtreme BCM5721 Gigabit Ethernet PCI Express (rev 21)

The Intel and Broadcom NICs work perfectly.

/etc/network/interfaces:

Code: 
auto lo
iface lo inet loopback

iface enp0s31f6 inet manual

iface enp3s0 inet manual

iface enp4s0 inet manual

iface enp5s0 inet manual

iface enp6s0 inet manual

iface enp9s0 inet manual

auto vmbr0
iface vmbr0 inet static
        address 192.168.0.194/24
        gateway 192.168.0.1
        bridge-ports enp0s31f6
        bridge-stp off
        bridge-fd 0

auto vmbr5
iface vmbr5 inet manual
        bridge-ports enp9s0
        bridge-stp off
        bridge-fd 0
        
auto vmbr1
iface vmbr1 inet manual
        bridge-ports enp3s0
        bridge-stp off
        bridge-fd 0

auto vmbr2
iface vmbr2 inet manual
        bridge-ports enp4s0
        bridge-stp off
        bridge-fd 0

auto vmbr3
iface vmbr3 inet manual
        bridge-ports enp5s0
        bridge-stp off
        bridge-fd 0
        
auto vmbr4
iface vmbr4 inet manual
        bridge-ports enp6s0
        bridge-stp off
        bridge-fd 0

Example lines from a VM .conf file:

Code: 
net0: e1000=62:CC:F3:49:DB:E2,bridge=vmbr0
net1: e1000=76:C3:4D:C9:2E:8D,bridge=vmbr1
net2: e1000=FE:00:ED:BA:BC:36,bridge=vmbr2

net0 will work, net1 and net2 will have no connectivity.

ip -br -c link show:

Code: 
lo               UNKNOWN        00:00:00:00:00:00 <LOOPBACK,UP,LOWER_UP> 
enp9s0           UP             00:10:18:33:c4:4d <BROADCAST,MULTICAST,UP,LOWER_UP> 
enp3s0           UP             00:0a:cd:39:7a:98 <BROADCAST,MULTICAST,UP,LOWER_UP> 
enp4s0           UP             00:0a:cd:39:7a:99 <BROADCAST,MULTICAST,UP,LOWER_UP> 
enp5s0           UP             00:0a:cd:39:7a:9a <BROADCAST,MULTICAST,UP,LOWER_UP> 
enp6s0           UP             00:0a:cd:39:7a:9b <BROADCAST,MULTICAST,UP,LOWER_UP> 
enp0s31f6        UP             4c:cc:6a:d2:c5:e4 <BROADCAST,MULTICAST,UP,LOWER_UP> 
vmbr0            UP             4c:cc:6a:d2:c5:e4 <BROADCAST,MULTICAST,UP,LOWER_UP> 
vmbr5            UP             00:10:18:33:c4:4d <BROADCAST,MULTICAST,UP,LOWER_UP> 
tap100i0         UNKNOWN        d6:c5:8e:b7:1c:67 <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> 
vmbr1            UP             00:0a:cd:39:7a:98 <BROADCAST,MULTICAST,UP,LOWER_UP> 
tap100i1         UNKNOWN        c2:2f:b3:c5:04:90 <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> 
vmbr2            UP             00:0a:cd:39:7a:99 <BROADCAST,MULTICAST,UP,LOWER_UP> 
vmbr3            UP             00:0a:cd:39:7a:9a <BROADCAST,MULTICAST,UP,LOWER_UP> 
vmbr4            UP             00:0a:cd:39:7a:9b <BROADCAST,MULTICAST,UP,LOWER_UP> 
tap100i2         UNKNOWN        8e:f9:62:e2:b6:e7 <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> 
tap100i3         UNKNOWN        6e:3c:85:0d:0b:9e <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> 
tap100i4         UNKNOWN        be:cc:97:d4:87:16 <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP>

Not sure if any of that is any help, but as I say I can actually communicate with vmbr1 - 4 if I give them static IP addresses and ping them, so connectivity from the network to the host seems fine. It's just passing it through from host to guest that seems a problem.

Any ideas?

Thanks for any help,

Chris.
 
Another thing to add, I tried using PCI passthrough to actually pass the Realtek card through to a Windows guest, and connectivity was fine then as well.
 
Just to update this - have been trying for several hours over the past few days. My best guess is that the drivers or the card itself doesn't support bridge mode connections. I can't find anything to support this theory but it's literally just bridging the connection that doesn't work. In all other respects, the card seems to be working.

I'm pretty much ready to give up now, I've ordered a different quad-ethernet card and I guess I'll repurpose this one!

On a side-note - I thought about passing through the NICs to the VMs, but I've only been successfuly in passing the entire card through to one VM - I can't pass through each individual connection to a different VM. Any ideas if this should be possible?

Thanks,

Chris.
 
Yep, so I plugged in an Intel I350-based card and it worked straight away! Connections are now bridging correctly. No idea what is wrong with the Realtek, but I think I'll avoid that chipset in the future.
 
You must log in or register to reply here.
Top Bottom