[SOLVED] random reboots

[udotirol]

I only learned today that my home proxmox server reboots frequently.

Sometimes it's sufficient to just start it to have the proxmox server go down an reboot. Unfortunately, I only activated the journal today, so I can't say how long this has been happening. After activating persistent logging, I see that the server goes down almost as if the reboot was planned:

Dec 12 17:30:23 foundation kernel: Command line: BOOT_IMAGE=/vmlinuz-5.4.78-2-pve root=UUID=6ad2877d-c58a-4337-8968-b432410fb376 ro quiet intel_iommu=on
Dec 12 17:30:23 foundation kernel: Linux version 5.4.78-2-pve (build@pve) (gcc version 8.3.0 (Debian 8.3.0-6)) #1 SMP PVE 5.4.78-2 (Thu, 03 Dec 2020 14:26:17 +0100) ()
-- Reboot --
Dec 12 17:29:59 foundation systemd-journald[385]: Journal stopped
Dec 12 17:29:59 foundation sshd[3905]: pam_unix(sshd:session): session closed for user root
Dec 12 17:29:59 foundation sshd[3905]: Exiting on signal 15
Dec 12 17:29:59 foundation systemd-shutdown[1]: Sending SIGTERM to remaining processes...
Dec 12 17:29:59 foundation systemd-shutdown[1]: Syncing filesystems and block devices.
Dec 12 17:29:59 foundation kernel: printk: systemd-shutdow: 45 output lines suppressed due to ratelimiting

So far, I've narrowed it down to my sons Minecraft VM: sometimes it's sufficient to start it up, sometimes it takes a while. Neither the VM nor the proxmox server itself have anything close to high load (plenty of CPU, RAM and I/O still available).

Today, I've seen it happening within shortest time, but like now, the physical server has been up and running for some hours.

One of the things I tried, was to update the server form 6.2 to the latest 6.3, but all to no avail.

I've attached the log, see around 17:30 for the reboot to happen.

As per hardware, this is my single node home server, so it hasn't got a lot of fancy things in it:

Intel(R) Core(TM) i7-8700T CPU @ 2.40GHz
32GB RAM
on-board SATA (baseboard GA-IMB310TN)
mdadm RAID-1 with two Samsung SSD 860 EVO 1TB
LVM-thin for the VMs

Any ideas what might be causing this and/or how to debug it?

 

[udotirol]

it has just happened 5 times within shortest time, and in total 8 times this day ...

Dec 13 11:00:58 foundation kernel: Command line: BOOT_IMAGE=/vmlinuz-5.4.78-2-pve root=UUID=6ad2877d-c58a-4337-8968-b432410fb376 ro quiet intel_iommu=on
Dec 13 11:00:58 foundation kernel: Linux version 5.4.78-2-pve (build@pve) (gcc version 8.3.0 (Debian 8.3.0-6)) #1 SMP PVE 5.4.78-2 (Thu, 03 Dec 2020 14:26:17 +0100) ()
-- Reboot --
Dec 13 11:00:14 foundation kernel: vmbr0: port 4(tap101i0) entered forwarding state
Dec 13 11:00:14 foundation kernel: vmbr0: port 4(tap101i0) entered blocking state
Dec 13 11:00:14 foundation kernel: vmbr0: port 4(tap101i0) entered disabled state
Dec 13 11:00:14 foundation kernel: vmbr0: port 4(tap101i0) entered blocking state
Dec 13 11:00:14 foundation kernel: device tap101i0 entered promiscuous mode
Dec 13 11:00:14 foundation systemd-udevd[1482]: Could not generate persistent MAC address for tap101i0: No such file or directory
Dec 13 11:00:14 foundation systemd-udevd[1482]: link_config: autonegotiation is unset or enabled, the speed and duplex are not writable.
Dec 13 11:00:14 foundation systemd-udevd[1482]: Using default interface naming scheme 'v240'.
Dec 13 11:00:14 foundation systemd[1]: Started 101.scope.
Dec 13 11:00:14 foundation pve-guests[1815]: start VM 101: UPID:foundation:00000717:000019C9:5FD5E62E:qmstart:101:root@pam:
Dec 13 11:00:14 foundation pve-guests[1203]: <root@pam> starting task UPID:foundation:00000717:000019C9:5FD5E62E:qmstart:101:root@pam:
Dec 13 11:00:14 foundation pvesh[1202]: Starting VM 101
Dec 13 11:00:13 foundation pmxcfs[1049]: [status] notice: RRDC update error /var/lib/rrdcached/db/pve2-storage/foundation/local: -1
Dec 13 11:00:13 foundation pmxcfs[1049]: [status] notice: RRDC update error /var/lib/rrdcached/db/pve2-storage/foundation/local-thin: -1

For the time being, I've disabled my son's VM, but I'm more than surprized to see this happening. One of the core concepts of "virtual" machines is that they are supposed to be so isolated that they cannot bring down the physical host.

I've uploaded some logs again, only showing the 3 lines before and after a reboot, but I don't see a pattern there unfortunately.

 

[udotirol]

final update: this has all turned out to be completely non-proxmox related. Instead my nice little home server has died ungracefully from a hardware failure ...

Sorry for the noise

 

[lps90]

what was the hardware problem?

 

[udotirol]

It was the power supply failing. While the server is not even close to high load, occasional spikes were enough to trigger the failure.

 

[apoc]

Sadly i know what you are talking about. PSUs tend to age. After 3 or More years I had issues mainly with consumer grade PSUs.
My ablecom/supermicro PSU running currently on the pve server is 8 years old