Windows System lost every connection after while after upgrade to 7

[Florian Reuland]

Hallo,

a few days ago I upgraded proxmox to 7.1 from the latest 6.x version.
Now I got the problem that my windows machine (Windows 2012 R2 with guest agent active and installed (newest stable version).
After a random time the communication ist completly lost to the machine. The agent give retry error in syslog No console is possible and also all other connections are completly lost. After stop and Start everything is doing well until it happens again. I tried to install a second server with same os and I getting the same problem there. There is also an Linux cm on the system and there is no problem.
My tries:
I moved the disc to different location also with different filesystem( raw and qcow)
Reinstalled agenda on windows
Tried without agent


All logs are showing nothing. Windows logs just stopping when this happens and daemon or other proxmox logs are also silent.
Do you guys have an ideas where I and how can check anything about this problem?

My last chance is to complete reinstall the proxmox and hope that after restore this problem stops but I think this is a not safe scenario.

Just tell me if more information are needed.

Thanks

Answer in eng and ger are welcome

 

[mira]

Please provide the output of pveversion -v and the VM config qm config <VMID>.

 

[Florian Reuland]

Please provide the output of pveversion -v and the VM config qm config <VMID>.

Hi Mira,

thanks for your questions - here are the results:

root@virtual:~# pveversion -v
proxmox-ve: 7.1-1 (running kernel: 5.13.19-2-pve)
pve-manager: 7.1-8 (running version: 7.1-8/5b267f33)
pve-kernel-helper: 7.1-6
pve-kernel-5.13: 7.1-5
pve-kernel-5.4: 6.4-11
pve-kernel-5.13.19-2-pve: 5.13.19-4
pve-kernel-5.4.157-1-pve: 5.4.157-1
pve-kernel-5.4.143-1-pve: 5.4.143-1
pve-kernel-5.4.73-1-pve: 5.4.73-1
ceph-fuse: 14.2.21-1
corosync: 3.1.5-pve2
criu: 3.15-1+pve-1
glusterfs-client: 9.2-1
ifupdown: 0.8.36+pve1
ksm-control-daemon: 1.4-1
libjs-extjs: 7.0.0-1
libknet1: 1.22-pve2
libproxmox-acme-perl: 1.4.0
libproxmox-backup-qemu0: 1.2.0-1
libpve-access-control: 7.1-5
libpve-apiclient-perl: 3.2-1
libpve-common-perl: 7.0-14
libpve-guest-common-perl: 4.0-3
libpve-http-server-perl: 4.0-4
libpve-storage-perl: 7.0-15
libqb0: 1.0.5-1
libspice-server1: 0.14.3-2.1
lvm2: 2.03.11-2.1
lxc-pve: 4.0.11-1
lxcfs: 4.0.11-pve1
novnc-pve: 1.2.0-3
proxmox-backup-client: 2.1.2-1
proxmox-backup-file-restore: 2.1.2-1
proxmox-mini-journalreader: 1.3-1
proxmox-widget-toolkit: 3.4-4
pve-cluster: 7.1-2
pve-container: 4.1-3
pve-docs: 7.1-2
pve-edk2-firmware: 3.20210831-2
pve-firewall: 4.2-5
pve-firmware: 3.3-3
pve-ha-manager: 3.3-1
pve-i18n: 2.6-2
pve-qemu-kvm: 6.1.0-3
pve-xtermjs: 4.12.0-1
qemu-server: 7.1-4
smartmontools: 7.2-pve2
spiceterm: 3.2-2
swtpm: 0.7.0~rc1+2
vncterm: 1.7-1
zfsutils-linux: 2.1.1-pve3


root@virtual:~# qm config 101
agent: 0
balloon: 1024
bootdisk: sata0
cores: 4
ide2: none,media=cdrom
memory: 8192
name: win.birekoven.de
net0: e1000=D6:63:48:1B:4AF,bridge=vmbr0
numa: 0
onboot: 1
ostype: win8
sata0: Backup:101/vm-101-disk-0.qcow2,size=300G,ssd=1
scsihw: virtio-scsi-pci
smbios1: uuid=b6f71dd6-f2a4-4a77-a4a3-d7522e2fde84
sockets: 1
unused0: Storage:101/vm-101-disk-0.raw

 

[Falk R.]

Hallo,

a few days ago I upgraded proxmox to 7.1 from the latest 6.x version.
Now I got the problem that my windows machine (Windows 2012 R2 with guest agent active and installed (newest stable version).
After a random time the communication ist completly lost to the machine. The agent give retry error in syslog No console is possible and also all other connections are completly lost. After stop and Start everything is doing well until it happens again. I tried to install a second server with same os and I getting the same problem there. There is also an Linux cm on the system and there is no problem.
My tries:
I moved the disc to different location also with different filesystem( raw and qcow)
Reinstalled agenda on windows
Tried without agent


All logs are showing nothing. Windows logs just stopping when this happens and daemon or other proxmox logs are also silent.
Do you guys have an ideas where I and how can check anything about this problem?

My last chance is to complete reinstall the proxmox and hope that after restore this problem stops but I think this is a not safe scenario.

Just tell me if more information are needed.

Thanks

Answer in eng and ger are welcome

Zufällig deutsches Windows? Bei 2016+ gibts ja hier im Forum schon ganz viel dazu.
z.B. https://forum.proxmox.com/threads/windows-server-2022-virtio.97887/

 

[Florian Reuland]

Zufällig deutsches Windows? Bei 2016+ gibts ja hier im Forum schon ganz viel dazu.
z.B. https://forum.proxmox.com/threads/windows-server-2022-virtio.97887/

Hi,

hatte ich auch schon gesehen und die Erfahrung hatte ich selber gemacht mit dem Netzwerkadaptern.
Aber nein hier ist es Windows Server 2012 R2.

Aber danke.

 

[mira]

Let's continue this in English

Is there anything at all in the journal when the VM basically `times out`?
Please provide the journal if possible.

 

[Florian Reuland]

Hi Mira,

of course.
Here is a part of the log when the connection lost happens:

09:55:01 virtual CRON[28307]: (root) CMD (cd ~/scripts && bash check_vpn.sh > check_vpn.log 2>&1)
Dec 8 09:58:13 virtual pvestatd[1130]: VM 101 qmp command failed - VM 101 qmp command 'query-proxmox-support' failed - got timeout
Dec 8 09:58:13 virtual pvestatd[1130]: status update time (6.379 seconds)
Dec 8 09:58:23 virtual pvestatd[1130]: VM 101 qmp command failed - VM 101 qmp command 'query-proxmox-support' failed - unable to connect to VM 101 qmp socket - timeout after 31 retries
Dec 8 09:58:24 virtual smartd[673]: Device: /dev/sda [SAT], SMART Usage Attribute: 194 Temperature_Celsius changed from 110 to 109
Dec 8 09:58:24 virtual pvestatd[1130]: status update time (6.376 seconds)
Dec 8 09:58:33 virtual pvestatd[1130]: VM 101 qmp command failed - VM 101 qmp command 'query-proxmox-support' failed - unable to connect to VM 101 qmp socket - timeout after 31 retries
Dec 8 09:58:33 virtual pvestatd[1130]: status update time (6.399 seconds)
Dec 8 09:58:43 virtual pvestatd[1130]: VM 101 qmp command failed - VM 101 qmp command 'query-proxmox-support' failed - unable to connect to VM 101 qmp socket - timeout after 31 retries
Dec 8 09:58:43 virtual pvestatd[1130]: status update time (6.371 seconds)
Dec 8 09:58:53 virtual pvestatd[1130]: VM 101 qmp command failed - VM 101 qmp command 'query-proxmox-support' failed - unable to connect to VM 101 qmp socket - timeout after 31 retries
Dec 8 09:58:54 virtual pvestatd[1130]: status update time (6.371 seconds)
Dec 8 09:59:03 virtual pvestatd[1130]: VM 101 qmp command failed - VM 101 qmp command 'query-proxmox-support' failed - unable to connect to VM 101 qmp socket - timeout after 31 retries

You see everything is fine until 09:58 then the system got timeouts. This is the syslog.
There es no error or any reaction that result in this problem.

 

[mira]

Any backups running during that time? Or any other I/O intensive work?

Could you provide the ps aux output?

 

[Florian Reuland]

Any backups running during that time? Or any other I/O intensive work?

Could you provide the ps aux output?

No sadly not. When i stop and start the vm then everything works for a random time.
It was a few minutes to a few hours.

Here is the current ps aux:

See attachment

I added also summary pictures while the machine 101 is lost

The machine 102 is the new installed windows vm with same issue

 

[Florian Reuland]

No sadly not. When i stop and start the vm then everything works for a random time.
It was a few minutes to a few hours.

Here is the current ps aux:

See attachment

I added also summary pictures while the machine 101 is lost

The machine 102 is the new installed windows vm with same issue

By the way the machine 100 is ubuntu and dont have the issue

 

[mira]

The screenshot shows that the guest agent isn't configured, but in your first post you mentioned it to be installed.
Did you disable it in PVE?

Was the `ps aux` output taken when the VM was stuck? If not, please provide it when that's the case.
Sorry that I wasn't clear on when to run the `ps aux` command.

 

[Florian Reuland]

The screenshot shows that the guest agent isn't configured, but in your first post you mentioned it to be installed.
Did you disable it in PVE?

Was the `ps aux` output taken when the VM was stuck? If not, please provide it when that's the case.
Sorry that I wasn't clear on when to run the `ps aux` command.

Hey,

yes i deactived the guest agent in the options to test. I can activate it we just get more timeout errors.

The ps aux is when the vm stuck.

The interessting about that is thats it seems like everything is doing normal when you check ps aux

 

[mira]

Could you disable swap for testing? This can sometimes slow everything down.
swapoff -a to disable it until the next reboot.

 

[Florian Reuland]

Could you disable swap for testing? This can sometimes slow everything down.
swapoff -a to disable it until the next reboot.

Check is off.
I also activated guest agent again

Should i bring the machine back?

 

[ferdikilic]

Hello, this problem is with me proxmox 7.1.7 guest os server 2016 no quemu tools