What is the VMs n hosts syslog showing at that time? Did you check the hosts syslog for "oom" messages? Did you check the hosts and VMs systemd timers and crontab if there is a job/schedule running at that time?
To check the PVE host for oom messages you could run cat /ver/log/syslog | grep oom. To chec the syslogd timers you could run systemctl list-timers. To check the crontab you can have a look at cat /etc/crontab as well as crontab -e.