[SOLVED] Network traffic in LXC brings PVE host down

[gcsecsey]

Hi there!

I have intalled Proxmox VE on a Beelink S12 Pro Mini PC with Intel N100. It is connected through an Apple Airport Extreme working as an unmanaged switch, to an Eero 6 router, which is the DHCP server.

I have an issue when starting a task in an LXC which uses the network, eg. downloading a torrent or just wgetting a bigger file. Both the LXC and the PVE web UIs become unresponsive and then disconnect. Over SSH, the input first becomes very much delayed then the host also drops any existing connections, and won't respond to pings. When the network task is done, usually the host recovers after a while, and I can connect to it again.

I'm using the default bridged configurations, this is the contents of my /etc/network/interfaces file:

auto lo
iface lo inet loopback

iface enp1s0 inet manual

auto vmbr0
iface vmbr0 inet static
        address 192.168.4.100/22
        gateway 192.168.4.1
        bridge-ports enp1s0
        bridge-stp off
        bridge-fd 0

iface wlo1 inet manual

source /etc/network/interfaces.d/*

The LXC gets an IP address from the DHCP server within the range. I also tried setting a static IP on the LXC, but the same issue occurs.

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host noprefixroute
       valid_lft forever preferred_lft forever
2: eth0@if6: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether bc:24:11:4d:7c:78 brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet 192.168.4.103/22 brd 192.168.7.255 scope global eth0
       valid_lft forever preferred_lft forever
    inet6 2a0a:ef40:649:ef00:be24:11ff:fe4d:7c78/64 scope global dynamic mngtmpaddr
       valid_lft 85974sec preferred_lft 3174sec
    inet6 fdfd:29cb:5152:1:be24:11ff:fe4d:7c78/64 scope global dynamic mngtmpaddr
       valid_lft 2591818sec preferred_lft 604618sec
    inet6 fe80::be24:11ff:fe4d:7c78/64 scope link
       valid_lft forever preferred_lft forever

Could you please help me figure out what's wrong with my configuration? Thanks!

 

[gfngfn256]

I'm no NW guru but I notice:

address 192.168.4.100/22

are you sure about that 22 bit mask?

This is probably causing:

inet 192.168.4.103/22 brd 192.168.7.255 scope global eth0

that 192.168.7.xxx brd (broadcasting address) to be considered in the same subnet as 192.168.4.xxx

Is this intentional seeing that as far as I could make out your above HW probably does not support/designed for this.

Maybe I'm missing something here.

 

[gcsecsey]

Thanks for the quick reply!

are you sure about that 22 bit mask?

Yes, this is the default subnet mask of my eero routers. AFAIK the broadcast address on the network is 192.168.7.255 and this has been working well so far. All the LXCs or VMs I create, and all other devices on my network have been receiving IP addresses in the 192.168.4.2-192.168.7.254 range.

Is this intentional seeing that as far as I could make out your above HW probably does not support/designed for this.

Could you elaborate on this a bit please? Do you mean that some of hardware may not support subnet masks other than /24?

In the meantime I did some more testing. I tried downloading the Ubuntu Server ISO both via wget and torrent in the LXC again. I'm starting to suspect that the issue might be related to the number of TCP connections the torrent client is setting up, and not the DHCP server. The last time I tested this I might have tried wgetting too soon, as running it after killing the torrent client within the LXC didn't produce the same issue.

I'll report back once I know anything more specific.

Thanks!

 

[alexskysilk]

are you sure about that 22 bit mask?

nothing wrong or unsupported/unsupportable with a /22.

Whats NIC is it in your host (lspci | grep net) and what driver is it using (lsmod,modinfo)

 

[gcsecsey]

Whats NIC is it in your host (lspci | grep net)

root@beelink:~# lspci | grep net
01:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 15)

and what driver is it using (lsmod,modinfo)

root@beelink:~# ethtool -i enp1s0
driver: r8169
version: 6.8.12-5-pve
firmware-version: rtl8168h-2_0.0.2 02/26/15
expansion-rom-version:
bus-info: 0000:01:00.0
supports-statistics: yes
supports-test: no
supports-eeprom-access: no
supports-register-dump: yes
supports-priv-flags: no

root@beelink:~# lsmod | grep r8169
r8169                 110592  0

Let me know if I can povide any more details.

 

[alexskysilk]

would be good to get the actual vid for the nic, but in general RTL nics are a pain in the ass on linux, and proxmox specifically. I dont use any so I cant help directly but I can suggest that its the likely cause of your issues. search the forums for realtek and there may be some workarounds (external firmware/drivers, kernel rollback, kernel boot line entries, etc.)

 

[gfngfn256]

While I agree with alexskysilk above that Realtek NICs can be a problem in the Linux world, I still believe that your NW infrastructure could be a possible bottleneck.

3 tests I can think of:

1. Do you have another device connected in exactly the same way. (i.e. Device -->Apple Airport --> Eero 6), make a heavy download with it, what happens to the PVE on the Beelink. Same thing the other way round - what happens to that device when you make a heavy download on the Beelink.

2. Connect the PVE Beelink directly to the --> Eero 6 & test your scenario.

3. Connect another USB to ETH adapter to the PVE Beelink & test your scenario.

Good luck.

 

[gcsecsey]

Thanks a lot @alexskysilk and @gfngfn256 for your helpful suggestions!

2. Connect the PVE Beelink directly to the --> Eero 6 & test your scenario.

I tested this scenario and everything started working perfectly! I even brought up multiple LXCs doing downloads in parallel and could experience no issues at all.

I can't troubleshoot the Apple Airport Extreme because it uses proprietary software, and within the Airport Utility tool, I can only set it up to work as a switch, there's no more settings. I think I'll just replace it with a gigabit switch and move it under that to work as a backup destination.

 

[gfngfn256]

Happy you got it solved.

I think I'll just replace it with a gigabit switch

Be aware it maybe that Eero 6 router not playing well with another switch out of it.
I'd first test with another switch before investing.