LXC container networking is unusably slow and unstable

[ksten]

Starting today, any new LXC containers I make have terrible network performance. Any downloads I make start out full speed, and then over the next 30 seconds they peter out to nothing and eventually time out.

Example download on the PVE host:

root@nuc13:~# wget https://cdimage.debian.org/debian-cd/current/amd64/iso-cd/debian-12.7.0-amd64-netinst.iso
--2024-09-21 17:30:57-- https://cdimage.debian.org/debian-cd/current/amd64/iso-cd/debian-12.7.0-amd64-netinst.iso
Resolving cdimage.debian.org (cdimage.debian.org)... 194.71.11.163, 194.71.11.173, 2001:6b0:19::163, ...
Connecting to cdimage.debian.org (cdimage.debian.org)|194.71.11.163|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://chuangtzu.ftp.acc.umu.se/debian-cd/current/amd64/iso-cd/debian-12.7.0-amd64-netinst.iso [following]
--2024-09-21 17:30:58-- https://chuangtzu.ftp.acc.umu.se/debian-cd/current/amd64/iso-cd/debian-12.7.0-amd64-netinst.iso
Resolving chuangtzu.ftp.acc.umu.se (chuangtzu.ftp.acc.umu.se)... 194.71.11.167, 2001:6b0:19::167
Connecting to chuangtzu.ftp.acc.umu.se (chuangtzu.ftp.acc.umu.se)|194.71.11.167|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 661651456 (631M) [application/x-iso9660-image]
Saving to: ‘debian-12.7.0-amd64-netinst.iso.1’

debian-12.7.0-amd64-netinst.i 100%[==============================================>] 631.00M 8.03MB/s in 81s

2024-09-21 17:32:19 (7.79 MB/s) - ‘debian-12.7.0-amd64-netinst.iso.1’ saved [661651456/661651456]

Same download from the container:

root@template-tx:~# wget https://cdimage.debian.org/debian-cd/current/amd64/iso-cd/debian-12.7.0-amd64-netinst.iso
--2024-09-21 17:34:06-- https://cdimage.debian.org/debian-cd/current/amd64/iso-cd/debian-12.7.0-amd64-netinst.iso
Resolving cdimage.debian.org (cdimage.debian.org)... 194.71.11.173, 194.71.11.163, 2001:6b0:19::173, ...
Connecting to cdimage.debian.org (cdimage.debian.org)|194.71.11.173|:443... connected.
HTTP request sent, awaiting response... Read error (Error in the pull function.) in headers.
Retrying.

--2024-09-21 17:34:08-- (try: 2) https://cdimage.debian.org/debian-cd/current/amd64/iso-cd/debian-12.7.0-amd64-netinst.iso
Connecting to cdimage.debian.org (cdimage.debian.org)|194.71.11.173|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://chuangtzu.ftp.acc.umu.se/debian-cd/current/amd64/iso-cd/debian-12.7.0-amd64-netinst.iso [following]
--2024-09-21 17:34:08-- https://chuangtzu.ftp.acc.umu.se/debian-cd/current/amd64/iso-cd/debian-12.7.0-amd64-netinst.iso
Resolving chuangtzu.ftp.acc.umu.se (chuangtzu.ftp.acc.umu.se)... 194.71.11.167, 2001:6b0:19::167
Connecting to chuangtzu.ftp.acc.umu.se (chuangtzu.ftp.acc.umu.se)|194.71.11.167|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 661651456 (631M) [application/x-iso9660-image]
Saving to: ‘debian-12.7.0-amd64-netinst.iso.1’

debian-12.7.0-amd64-netinst.i 1%[ ] 9.69M --.-KB/s eta 2h 3m

Copying from the host to the container works fine:

root@template-tx:~# scp root@nuc13:~/*.iso .
root@nuc13's password:
debian-12.7.0-amd64-netinst.iso 100% 72MB 415.2MB/s 00:00

[B]Copying between two physical hosts works as well:[/B]

root@thor:~# scp nuc13:~/*.iso .
debian-12.7.0-amd64-netinst.iso 100% 72MB 111.1MB/s 00:00

[B]Copying from another machine on the local network to the container doesn't:[/B]

root@template-tx:~# scp root@thor:~/*.iso .
root@thor's password:
debian-12.7.0-amd64-netinst.iso 50% 37MB 21.0MB/s 00:01 ETAclient_loop: send disconnect: Broken pipe
scp: Connection closed

I'm not really sure how to diagnose this...