Linux guests thawed from pause/hibernate not fixing time despite guest-agent installed

[tconnors]

All of my Debian/stable and Centos/7.8 guests (on proxmox 6.2 host) are having their time paused for the duration of any pause/hibernate.

The guests have agents installed:

[root@north-jump ~]# rpm -qa | grep guest
qemu-guest-agent-2.12.0-3.el7.x86_64
49096,1> dpkg --get-selections | grep guest
qemu-guest-agent install
(Version: 1:3.1+dfsg-8+deb10u5)

But while the machine is paused, the guest clock stops ticking, and does not get reset when resumed.

I *can* run manually from the host foreach guest being resumed:

sudo socat unix-connect:/var/run/qemu-server/100.qga stdin <<< '{"execute":"guest-set-time"}'

and that works, but this seems strongly like the kind of job I'd expect the hypervisor to do for me?

The results are independent of whether ntp/chrony is running or not.

Am I missing something obvious, or is this a bug? Google is returning nothing relevant about timekeeping in linux guests compared to the copious documentation about how to configure ntp and guest agents for vmware! But nor is it returning results that anyone else is having trouble with timekeeping!

 

[tconnors]

Oh yeah, and host:/etc/default/libvirt-guests
has SYNC_TIME=1

 

[Stoiko Ivanov]

if you let timesyncd run that should take care of the large jump after pause/hibernate - would this work in your environment?

 

[tconnors]

> if you let timesyncd run that should take care of the large jump after pause/hibernate - would this work in your environment?

Alas not, most of my systems don't run systemd (I've tried, and failed to migrate most of them).

But now I've heard of /dev/ptp0, perhaps chrony will still work, and Azure have offered this advice:

https://docs.microsoft.com/en-us/azure/virtual-machines/linux/time-sync

https://opensource.com/article/17/6/timekeeping-linux-vms

It's odd that I had to set ntp.conf's "tinker panic 0" on vmware clusters 10 years ago, but ntp didn't seem to panic on me here when time was vastly out (it also didn't do anything to fix it). I'd much rather stay with ntp than chrony (I've come to understand it quite well over the past 20 years), so might try experimenting with "tinker panic 0 ; tinker step 1 ; tinker stepout 0"

It'd be much more elegant if proxmox/kvm told the guests to guest-set-time when appropriate though.

 

[tconnors]

timesyncd etc won't fundamentally solve the issue, because they can't know that the hypervisor have put them to sleep and woken them up again. At best they can do is poll at some interval, not so short that they overwhelm the upstream servers, and hope the network is up when it finally gets around to querying upstream. Meanwhile, the kernel and ntp doesn't know why the time jumped, and might conclude the best thing to do is to slew back to the correct time if the pause was only brief (eg, a freeze resulting from a backup), rather than correctly jumping. All the while, the host is right there underneath the VM, with the right time, and with the knowledge that it just woke up the VM.

I have looked into hooks - but they only get invoked at startup and shutdown - and resuming from hibernation does count as startup. From there, I can run `socat unix-connect:/var/run/qemu-server/100.qga stdin <<< '{"execute":"guest-set-time"}'`

But there's no hook for pause and resume. Looks like it would be easy to run a hook as part of QemuServer:vm_resume(), will automatically take care of all the cases that might cause a VM to be paused. Probably need a hook for vm_suspend too. If this works out, I suggest there should be an option (default off is fine) for VMs to automatically run guest-set-time if an agent is running. Looks like we'd need a new API call in API2/Qemu/Agent.pm - it's only got get-time currently.

 

[tconnors]

I've just submitted bug 3482, along with a fix that works for me.