Network speed sometimes slows down to about 3 MB/s, system-wide, including host. Other devices in network unaffected.

[Lucavon]

Hello everyone, first of all, happy new year!

I've been facing a certain problem with Proxmox for the past few months: every now and then - as far as I can tell, not following any pattern - the network speed slows down significantly for a few minutes. This affects the host itself and the containers and VMs. Sometimes, however, only certain containers are affected, while others are not. It's always the same containers, reinstalling them does not fix the issue. Not even leaving the container fresh - e.g. installing the Debian 11 template and doing nothing else but testing the speed - fixes the issue.
In this case, containers 106 and 110 are usually the only ones affected. I have checked the traffic, CPU, memory and disk IO graphs for both the host and the affected containers, and they are all far below 100%. I have also monitored the network traffic for the entire network, and there is no especially high load on the network while the problem is there.

So, to summarize:
- Some containers (106, 110) and the host have slow (0.5 - 3 MB/s) network speeds (only tested connecting to the internet, not between containers or to the local network)
- RAM, CPU, disk IO and network usage for both the VMs and host are not abnormal and not close to their limits
- Network itself is not under high load
- Other devices in the network get the full speed (~60 MB/s) on speed tests
- The issue comes up randomly, persists for a few minutes, then goes away again
- There is no (known) trigger for this issue

There are no suspicious dmesg messages, journalctl -xe does not show anything out of the ordinary. I'm on the latest Proxmox version, everything is fully up to date and was rebooted after installing updates.

Hardware:
CPU: i3 10100
RAM: 4x 16 GB DDR4 @ 2666 MHz, CL19
Disks: 1x 1 TB SATA SSD, 1x 500 GB SATA SSD forming an LV of 1.5TB, 1x 120 GB SATA SSD as the boot drive, 1x 1 TB NVMe SSD forming an LV.
PCIe Cards: Network Card with 4 Gigabit ports, passed to a VM

The network port used for Proxmox is the motherboard's 1 Gbit port.

Do you have any ideas what could be causing these very inconsistent network speeds that seemingly only affect certain containers?

Thank you!

 

[Dunuin]

Hello everyone, first of all, happy new year!

I've been facing a certain problem with Proxmox for the past few months: every now and then - as far as I can tell, not following any pattern - the network speed slows down significantly for a few minutes. This affects the host itself and the containers and VMs. Sometimes, however, only certain containers are affected, while others are not. It's always the same containers, reinstalling them does not fix the issue. Not even leaving the container fresh - e.g. installing the Debian 11 template and doing nothing else but testing the speed - fixes the issue.
In this case, containers 106 and 110 are usually the only ones affected. I have checked the traffic, CPU, memory and disk IO graphs for both the host and the affected containers, and they are all far below 100%. I have also monitored the network traffic for the entire network, and there is no especially high load on the network while the problem is there.

So, to summarize:
- Some containers (106, 110) and the host have slow (0.5 - 3 MB/s) network speeds (only tested connecting to the internet, not between containers or to the local network)
- RAM, CPU, disk IO and network usage for both the VMs and host are not abnormal and not close to their limits
- Network itself is not under high load
- Other devices in the network get the full speed (~60 MB/s) on speed tests
- The issue comes up randomly, persists for a few minutes, then goes away again
- There is no (known) trigger for this issue

There are no suspicious dmesg messages, journalctl -xe does not show anything out of the ordinary. I'm on the latest Proxmox version, everything is fully up to date and was rebooted after installing updates.

Hardware:
CPU: i3 10100
RAM: 4x 16 GB DDR4 @ 2666 MHz, CL19
Disks: 1x 1 TB SATA SSD, 1x 500 GB SATA SSD forming an LV of 1.5TB, 1x 120 GB SATA SSD as the boot drive, 1x 1 TB NVMe SSD forming an LV.
PCIe Cards: Network Card with 4 Gigabit ports, passed to a VM

The network port used for Proxmox is the motherboard's 1 Gbit port.

Do you have any ideas what could be causing these very inconsistent network speeds that seemingly only affect certain containers?

Thank you!

So the LXCs that are causing problems are connected to the onboard Gbit NIC or are you using something like a OPNsense/pfsense VM that is routing the guests traffic with your 4x Gbit passed through NIC?
What are the chipsets of these NICs?

 

[Lucavon]

So the LXCs that are causing problems are connected to the onboard Gbit NIC or are you using something like a OPNsense/pfsense VM that is routing the guests traffic with your 4x Gbit passed through NIC?
What are the chipsets of these NICs?

The LXCs are using vmbr0, which uses the motherboard's LAN port. The traffic then goes through 2 switches and back into the server - this time the network card OPNsense has - and from there to my modem and the internet.

So basically, the router for the entire network is the OPNSense VM. I've tested to ensure that this router VM is not the one causing the problem - and it's not. While the issue occurs, other devices can use the full bandwidth I have.

The motherboard's network chip is Some generic Intel GbE chip (Mainboard is the Gigabyte B460M-D3H) and the PCIe card has an Intel 82580 chip, it's an NC365T.
I can't find any details about the internal network card of the Mainboard, I tried a bunch of commands like hwinfo and tried googling but I just can't find the specific model name.

 

[mspinolo]

Hi,

I have a similar issue: did you find a solution for your case?
what do you mean by "PCIe Cards: Network Card with 4 Gigabit ports, passed to a VM"?