Host network flaps/is temporary disabled by VM? Very strange!

bleomycin

Renowned Member
Mar 20, 2011
49
7
73
I'm having this extremely strange issue I have never seen before and unfortunately have been unable to debug. I've installed photoprism (self hosted google photos competitor, kind of amazing) in a ubuntu 20.04 VM. Inside this vm I'm running docker from the docker repository. I don't know if docker is somehow to blame for this but it is the only thing I can think of (photoprism is installed via docker)? When photoprism is indexing photo's it will use as many cpu cores as you can throw at it and is incredibly intensive, however it barely requires any network access while it is doing this.

This is what happens after allowing the photoprism VM to index photo's on the proxmox host, it takes a couple minutes of processing then this happens:

Code:
[  556.967492] vmbr0: port 1(eth2) entered disabled state
[  586.503584] ixgbe 0000:04:00.0 eth2: NIC Link is Up 10 Gbps, Flow Control: RX/TX
[  586.503675] vmbr0: port 1(eth2) entered blocking state
[  586.503679] vmbr0: port 1(eth2) entered forwarding state
[  586.615641] ixgbe 0000:04:00.0 eth2: NIC Link is Down
[  587.519728] vmbr0: port 1(eth2) entered disabled state
[  588.639611] ixgbe 0000:04:00.0 eth2: NIC Link is Up 10 Gbps, Flow Control: RX/TX
[  588.639700] vmbr0: port 1(eth2) entered blocking state
[  588.639704] vmbr0: port 1(eth2) entered forwarding state
[  591.084207] ixgbe 0000:04:00.0 eth2: NIC Link is Down
[  591.084386] vmbr0: port 1(eth2) entered disabled state
[  592.767676] ixgbe 0000:04:00.0 eth2: NIC Link is Up 10 Gbps, Flow Control: RX/TX
[  592.767771] vmbr0: port 1(eth2) entered blocking state
[  592.767775] vmbr0: port 1(eth2) entered forwarding state
[  912.701799] vmbr1: port 11(tap134i0) entered disabled state
[  920.111666] device tap134i0 entered promiscuous mode
[  920.120260] vmbr1: port 11(tap134i0) entered blocking state
[  920.120261] vmbr1: port 11(tap134i0) entered disabled state
[  920.120346] vmbr1: port 11(tap134i0) entered blocking state
[  920.120347] vmbr1: port 11(tap134i0) entered forwarding state


All connectivity to the host and all of it's guests is lost for a few seconds to a minute while these interfaces are flapping. I'm at a loss, any help is greatly appreciated!


photoprism guest .conf

Code:
agent: 1
balloon: 2000
bootdisk: scsi0
cores: 10
cpu: host
cpuunits: 500
ide2: local:iso/ubuntu-20.04-live-server-amd64.iso,media=cdrom
memory: 8000
name: photoprism
net0: e1000=5A:F5:E6:D4:AE:F0,bridge=vmbr1
numa: 0
ostype: l26
scsi0: zfsp3600:vm-149-disk-0,backup=0,discard=on,size=400G,ssd=1
scsihw: virtio-scsi-pci
smbios1: uuid=96a0a9b6-e709-400b-879a-d12bed9a7e84
sockets: 1
vmgenid: 93f20aac-18c4-47aa-8c28-d937b5c9eee2
Initially tried with virtio network, switched to e1000 for testing, made no difference.

interfaces.conf - wan ip's changed to bogus ones.

Code:
auto lo
iface lo inet loopback

iface eth0 inet manual

iface eth1 inet manual

iface eth2 inet manual

auto vmbr0
iface vmbr0 inet static
        address  35.74.27.242
        netmask  255.255.255.240
        gateway  35.74.27.241
        bridge-ports eth2
        bridge-stp off
        bridge-fd 0

auto vmbr1
iface vmbr1 inet static
        address  192.168.2.3
        netmask  255.255.255.0
        bridge-ports none
        bridge-stp off
        bridge-fd 0
        post-up ip route add 192.168.2.0/24 dev vmbr1 src 192.168.2.3 table rt2
        post-up ip route add default via 192.168.2.1 dev vmbr1 table rt2
        post-up ip rule add from 192.168.2.3/32 table rt2
        post-up ip rule add to 192.168.2.3/32 table rt2
        post-up ip route add 192.168.1.0/24 via 192.168.2.1 dev vmbr1

Code:
pveversion -v
proxmox-ve: 6.2-1 (running kernel: 5.4.44-2-pve)
pve-manager: 6.2-10 (running version: 6.2-10/a20769ed)
pve-kernel-5.4: 6.2-4
pve-kernel-helper: 6.2-4
pve-kernel-5.4.44-2-pve: 5.4.44-2
pve-kernel-5.4.41-1-pve: 5.4.41-1
pve-kernel-4.15: 5.4-18
pve-kernel-4.13: 5.2-2
pve-kernel-4.15.18-29-pve: 4.15.18-57
pve-kernel-4.15.18-9-pve: 4.15.18-30
pve-kernel-4.13.16-4-pve: 4.13.16-51
pve-kernel-4.13.16-2-pve: 4.13.16-48
pve-kernel-4.13.4-1-pve: 4.13.4-26
pve-kernel-4.4.83-1-pve: 4.4.83-96
pve-kernel-4.4.40-1-pve: 4.4.40-82
pve-kernel-4.4.35-2-pve: 4.4.35-79
pve-kernel-4.4.35-1-pve: 4.4.35-77
pve-kernel-4.4.19-1-pve: 4.4.19-66
pve-kernel-4.4.13-2-pve: 4.4.13-58
pve-kernel-4.4.13-1-pve: 4.4.13-56
pve-kernel-4.4.10-1-pve: 4.4.10-54
pve-kernel-4.4.6-1-pve: 4.4.6-48
pve-kernel-4.2.8-1-pve: 4.2.8-41
pve-kernel-4.2.6-1-pve: 4.2.6-36
pve-kernel-4.2.2-1-pve: 4.2.2-16
ceph-fuse: 12.2.11+dfsg1-2.1+b1
corosync: 3.0.4-pve1
criu: 3.11-3
glusterfs-client: 5.5-3
ifupdown: 0.8.35+pve1
ksm-control-daemon: 1.3-1
libjs-extjs: 6.0.1-10
libknet1: 1.16-pve1
libproxmox-acme-perl: 1.0.4
libpve-access-control: 6.1-2
libpve-apiclient-perl: 3.0-3
libpve-common-perl: 6.1-5
libpve-guest-common-perl: 3.1-1
libpve-http-server-perl: 3.0-6
libpve-storage-perl: 6.2-5
libqb0: 1.0.5-1
libspice-server1: 0.14.2-4~pve6+1
lvm2: 2.03.02-pve4
lxc-pve: 4.0.2-1
lxcfs: 4.0.3-pve3
novnc-pve: 1.1.0-1
proxmox-mini-journalreader: 1.1-1
proxmox-widget-toolkit: 2.2-9
pve-cluster: 6.1-8
pve-container: 3.1-11
pve-docs: 6.2-5
pve-edk2-firmware: 2.20200531-1
pve-firewall: 4.1-2
pve-firmware: 3.1-1
pve-ha-manager: 3.0-9
pve-i18n: 2.1-3
pve-qemu-kvm: 5.0.0-11
pve-xtermjs: 4.3.0-1
pve-zsync: 2.0-3
qemu-server: 6.2-10
smartmontools: 7.1-pve2
spiceterm: 3.1-1
vncterm: 1.6-1
zfsutils-linux: 0.8.4-pve1


I've been running proxmox for many many years trouble free, thank you for creating such amazing software!
 
Hi,

I guess you have problems with Flow Control.
Try to disable Flow Control this can be done with "ethtool -a"
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!