VirtIO Network Adapters lose configuration after PVE reboot

[seteq]

Hi,

I'm unsure if this is the correct place to ask this, but I have a very strange problem and no idea about the cause and how to fix this.

In the last few weeks my 3 node test cluster (PVE 6.3-6) had several external caused outages (power and heartbeat network issues).
As a result I had some KVM-Windows-Machines mysteriously lose their network configuration which has never happened before.
The affected Guest-OS versions are Server 2008R2 (i know!), Server 2019 and even Windows 10 20H2.

There was a new network adapter appearing inside the Windows-Guest (Red Hat VirtIO Ethernet Adapter #2 or even #3), which was set to DHCP.
After I reconfigured the adapter to it's previous fixed IPv4-Address, it said that there already is another adapter with this IP address.
When I check the nonpresent devices in Device Manager the additional VirtIO-Adapters are still there but disconnected.

I'm using VirtIO-Drivers 0.1.171 and did not change anything on the KVM-Guest's or PVE-host's configuration.
That happened three times now and I have no idea why this is happening or which logfiles to check.

Does anyone have an idea how to track this down?

Thank you

 

[t.lamprecht]

Hi,

You were/are affected by this bug:
https://forum.proxmox.com/threads/w...s-6-3-4-patch-inside.84915/page-2#post-373380

Then you worked around it and re-added the lost configuration to the new network device before we could provide a fix for pve-no-subscription, described in:
https://forum.proxmox.com/threads/w...s-6-3-4-patch-inside.84915/page-3#post-374993

So for you the following sentence from above linked thread applied:

for people on pve-subscription which either adapted their Windows VMs to the changes or installed new Windows VMs with the 5.2.0-2 QEMU version: you should set the machine type (q35 or pc-i440fx) to 5.2.

You adapted already, so you can set the machine version to 5.2 for those VMs via the webinterface.

To confirm this post your currently used versions: pveversion -v

 

[seteq]

Hello Thomas,

Thank you very much - I really did not expect such a quick reply in a free support forum
The mentioned bug indeed sounds like to be the issue for this problem.

The hardware version of the affected VMs is now set to 5.1
I assume when I change the version to 5.2 manually, the network magic will happen once again.

Some of my VMs also have a "Latest" option in the dropdown menu while others do not have this option available.

Thank you!

# pveversion -v

proxmox-ve: 6.3-1 (running kernel: 5.4.103-1-pve)
pve-manager: 6.3-6 (running version: 6.3-6/2184247e)
pve-kernel-5.4: 6.3-7
pve-kernel-helper: 6.3-7
pve-kernel-5.3: 6.1-6
pve-kernel-5.4.103-1-pve: 5.4.103-1
pve-kernel-5.4.98-1-pve: 5.4.98-1
pve-kernel-5.3.18-3-pve: 5.3.18-3
ceph-fuse: 12.2.11+dfsg1-2.1+b1
corosync: 3.1.0-pve1
criu: 3.11-3
glusterfs-client: 5.5-3
ifupdown: residual config
ifupdown2: 3.0.0-1+pve3
ksm-control-daemon: 1.3-1
libjs-extjs: 6.0.1-10
libknet1: 1.20-pve1
libproxmox-acme-perl: 1.0.7
libproxmox-backup-qemu0: 1.0.3-1
libpve-access-control: 6.1-3
libpve-apiclient-perl: 3.1-3
libpve-common-perl: 6.3-5
libpve-guest-common-perl: 3.1-5
libpve-http-server-perl: 3.1-1
libpve-storage-perl: 6.3-7
libqb0: 1.0.5-1
libspice-server1: 0.14.2-4~pve6+1
lvm2: 2.03.02-pve4
lxc-pve: 4.0.6-2
lxcfs: 4.0.6-pve1
novnc-pve: 1.1.0-1
proxmox-backup-client: 1.0.10-1
proxmox-mini-journalreader: 1.1-1
proxmox-widget-toolkit: 2.4-6
pve-cluster: 6.2-1
pve-container: 3.3-4
pve-docs: 6.3-1
pve-edk2-firmware: 2.20200531-1
pve-firewall: 4.1-3
pve-firmware: 3.2-2
pve-ha-manager: 3.1-1
pve-i18n: 2.2-2
pve-qemu-kvm: 5.2.0-3
pve-xtermjs: 4.7.0-3
qemu-server: 6.3-8
smartmontools: 7.2-pve2
spiceterm: 3.1-1
vncterm: 1.6-2
zfsutils-linux: 2.0.3-pve2

 

[Ramalama]

@t.lamprecht
My vm's were on 5.1 too... After i adopted the change in the beginning and upgraded through all versions xD
The behavior should have been that i should had 5.2.

Just a side note, but probably you should check it once before releasing in enterprise.

@seteq
Correct, you should change to 5.2 and in theory it should be fine again. (Hopefully)

 

[t.lamprecht]

Some of my VMs also have a "Latest" option in the dropdown menu while others do not have this option available.

WIndows VMs won't have the latest option available, as for them such HW change tend to be way more problematic and result in breakage sooner or later; that's why we always pin the VMs with Windows as OS type.

My vm's were on 5.1 too... After i adopted the change in the beginning and upgraded through all versions xD
The behavior should have been that i should had 5.2.

Just a side note, but probably you should check it once before releasing in enterprise.

What? The problematic version never hit Enterprise, so there nobody was affected and thus nobody had adapted, and so they will require the pinned 5.1 version, as that is now the last version before the ACPI change.