Synchronization in PMG stops working

[vladosubu]

Good afternoon!
After switching from version 7 to version 8, a problem appeared:
Both clusters started to fall into the syncing status, a reboot solves this problem, but after 3-4 days the error reappears. Tell me, please, what is this about? And how can it be solved? How can I view the synchronization logs?

 

[Stoiko Ivanov]

please share the journal from both nodes (`journalctl --since today`) - make sure to anonymize the information you don't want to share publicly.

 

[vladosubu]

Thanks a lot, thanks to the logs we found the reason, but we don't know how to fix it. There are different times on the clusters, because of this they cannot synchronize, please tell me how to correct the time?

Time Service – chronyd

Access to the time server is allowed.

Sep 04 08:35:44 mx1 pmgmirror[8627]: starting cluster synchronization
Sep 04 08:35:45 mx1 pmgmirror[8627]: database sync 'mx2' failed - large time difference (> 6 seconds) - not syncing
Sep 04 08:35:46 mx1 pmgmirror[8627]: cluster synchronization finished (1 errors, 1.94 seconds (files 0.00, database 1.35, config 0.59))

MX1 and MX2 are virtual machines in PVE.
There are options in the settings of these virtual machines:
Use local time for RTC = Default (Enabled for Windows)
RTC start date: now

It is likely that these options may affect.

And now you can see that the time on both MX differs by about 10 seconds.

 

[Stoiko Ivanov]

are both running on the same PVE-node?
are the PVE-nodes in sync?
are the PMG installations containers of Qemu-VMs?

 

[vladosubu]

1. No on different
2. Yes, both nodes have the exact time (second per second)
3. No, these are virtual machines

 

[Stoiko Ivanov]

3. No, these are virtual machines

check the journal on the PVE-nodes maybe they mention why the clock is different?

also check the journal since boot of the PMG nodes `journalctl -b`

 

[vladosubu]

On PVE nodes, the time is synchronized normally. On one mx, the time is like on PVE nodes, and on the other mx, the time is lagging.

 

[Stoiko Ivanov]

as said - check the logs/journal of the pve-nodes (and the one from the PMG node) - there might be some context there ...

 

[vladosubu]

I looked at the logs, there are no errors in PVE

 

[Stoiko Ivanov]

maybe try shutting down the PMG VM that lags behind completely and start it fresh - check the journal from its boot - feel free to share it here, and also keep an eye on the journal of the PVE node

usually time-drift inside qemu-guests is nothing that happen without any notice in the logs

 

[vladosubu]

Good afternoon! I checked the logs for PNG and PVE, turned off the virtual machine and turned it on, I didn't see anything normal, everything is in normal mode. What other options are there?

 

[vladosubu]

Good day, I raise the question

 

[Stoiko Ivanov]

Share the journals from the pve-nodes and the pmg machines - maybe we can see something off there ...