Automatic Proxmox server reboot

Medvenda

Member
Oct 5, 2016
40
0
6
35
So really?
This is the line look like system starting up:

Jan 13 08:11:56 proxmoxch03-hp rsyslogd: [origin software="rsyslogd" swVersion="8.4.2" x-pid="2690" x-info="http://www.rsyslog.com"] start

And you say this are the lines before?

Jan 13 16:47:17 proxmoxch03-hp systemd[1]: Started udev Kernel Device Manager.
Jan 13 16:47:17 proxmoxch03-hp systemd[1]: Starting LSB: Set preliminary keymap...
Jan 13 16:47:17 proxmoxch03-hp hdparm[546]: Setting parameters of disc: (none).

This is more later! This must be something like:

Jan 13 08:06:00 ...
Jan 13 07:05:13 proxmoxch03-hp pvedaemon[11567]: <root@pam> successful auth for user 'root@pam'
Jan 13 07:05:13 proxmoxch03-hp pvedaemon[11567]: <root@pam> successful auth for user 'root@pam'
Jan 13 07:05:14 proxmoxch03-hp pvedaemon[10345]: <root@pam> successful auth for user 'root@pam'
Jan 13 07:05:14 proxmoxch03-hp pvedaemon[58802]: <root@pam> successful auth for user 'root@pam'
Jan 13 07:10:52 proxmoxch03-hp systemd-timesyncd[2303]: interval/delta/delay/jitter/drift 2048s/-0.000s/0.020s/0.001s/+34ppm
Jan 13 07:17:01 proxmoxch03-hp CRON[64927]: (root) CMD ( cd / && run-parts --report /etc/cron.hourly)
Jan 13 07:21:17 proxmoxch03-hp smartd[2639]: Device: /dev/sda, failed to read Temperature
Jan 13 07:21:17 proxmoxch03-hp rrdcached[2689]: flushing old values
Jan 13 07:21:17 proxmoxch03-hp rrdcached[2689]: rotating journals
Jan 13 07:21:17 proxmoxch03-hp rrdcached[2689]: started new journal /var/lib/rrdcached/journal/rrd.journal.1484288477.535610
Jan 13 07:21:17 proxmoxch03-hp rrdcached[2689]: removing old journal /var/lib/rrdcached/journal/rrd.journal.1484281277.535583
Jan 13 07:25:29 proxmoxch03-hp pvedaemon[10345]: <root@pam> successful auth for user 'root@pam'
Jan 13 07:25:30 proxmoxch03-hp pvedaemon[11567]: <root@pam> successful auth for user 'root@pam'
Jan 13 07:25:30 proxmoxch03-hp pvedaemon[10345]: <root@pam> successful auth for user 'root@pam'

........

Jan 13 08:04:41 proxmoxch03-hp pvedaemon[11567]: <root@pam> successful auth for user 'root@pam'
Jan 13 08:04:41 proxmoxch03-hp pvedaemon[11567]: <root@pam> successful auth for user 'root@pam'
Jan 13 08:04:44 proxmoxch03-hp pvedaemon[11567]: <root@pam> successful auth for user 'root@pam'
Jan 13 08:04:45 proxmoxch03-hp pvedaemon[10345]: <root@pam> successful auth for user 'root@pam'
Jan 13 08:04:45 proxmoxch03-hp pvedaemon[58802]: <root@pam> successful auth for user 'root@pam'
Jan 13 08:04:46 proxmoxch03-hp pvedaemon[10345]: <root@pam> successful auth for user 'root@pam'
Jan 13 08:04:46 proxmoxch03-hp pvedaemon[11567]: <root@pam> successful auth for user 'root@pam'
Jan 13 08:04:47 proxmoxch03-hp pvedaemon[58802]: <root@pam> successful auth for user 'root@pam'
Jan 13 08:04:48 proxmoxch03-hp pvedaemon[11567]: <root@pam> successful auth for user 'root@pam'
Jan 13 08:04:48 proxmoxch03-hp pvedaemon[58802]: <root@pam> successful auth for user 'root@pam'
Jan 13 08:04:49 proxmoxch03-hp pvedaemon[11567]: <root@pam> successful auth for user 'root@pam'
Jan 13 08:04:49 proxmoxch03-hp pvedaemon[58802]: <root@pam> successful auth for user 'root@pam'
Jan 13 08:04:50 proxmoxch03-hp pvedaemon[11567]: <root@pam> successful auth for user 'root@pam'
Jan 13 08:04:51 proxmoxch03-hp pvedaemon[58802]: <root@pam> successful auth for user 'root@pam'
Jan 13 08:04:51 proxmoxch03-hp pvedaemon[11567]: <root@pam> successful auth for user 'root@pam'
Jan 13 08:04:52 proxmoxch03-hp pvedaemon[58802]: <root@pam> successful auth for user 'root@pam'
Jan 13 08:04:52 proxmoxch03-hp pvedaemon[10345]: <root@pam> successful auth for user 'root@pam'
Jan 13 08:04:53 proxmoxch03-hp pvedaemon[11567]: <root@pam> successful auth for user 'root@pam'
Jan 13 08:04:54 proxmoxch03-hp pvedaemon[58802]: <root@pam> successful auth for user 'root@pam'
Jan 13 08:04:54 proxmoxch03-hp pvedaemon[11567]: <root@pam> successful auth for user 'root@pam'
Jan 13 08:04:57 proxmoxch03-hp pvedaemon[58802]: <root@pam> successful auth for user 'root@pam'
Jan 13 08:04:57 proxmoxch03-hp pvedaemon[58802]: <root@pam> successful auth for user 'root@pam'
Jan 13 08:05:00 proxmoxch03-hp pvedaemon[58802]: <root@pam> successful auth for user 'root@pam'
Jan 13 08:05:01 proxmoxch03-hp pvedaemon[10345]: <root@pam> successful auth for user 'root@pam'
Jan 13 08:05:02 proxmoxch03-hp pvedaemon[11567]: <root@pam> successful auth for user 'root@pam'
Jan 13 08:05:02 proxmoxch03-hp pvedaemon[10345]: <root@pam> successful auth for user 'root@pam'
Jan 13 08:05:03 proxmoxch03-hp pvedaemon[58802]: <root@pam> successful auth for user 'root@pam'
Jan 13 08:05:03 proxmoxch03-hp pvedaemon[11567]: <root@pam> successful auth for user 'root@pam'
Jan 13 08:05:04 proxmoxch03-hp pvedaemon[58802]: <root@pam> successful auth for user 'root@pam'
Jan 13 08:05:05 proxmoxch03-hp pvedaemon[58802]: <root@pam> successful auth for user 'root@pam'
Jan 13 08:05:05 proxmoxch03-hp pvedaemon[11567]: <root@pam> successful auth for user 'root@pam'
Jan 13 08:05:06 proxmoxch03-hp pvedaemon[58802]: <root@pam> successful auth for user 'root@pam'
Jan 13 08:05:06 proxmoxch03-hp pvedaemon[11567]: <root@pam> successful auth for user 'root@pam'
Jan 13 08:05:07 proxmoxch03-hp pvedaemon[10345]: <root@pam> successful auth for user 'root@pam'

Jan 13 08:05:06 proxmoxch03-hp pvedaemon[11567]: <root@pam> successful auth for user 'root@pam'
Jan 13 08:05:07 proxmoxch03-hp pvedaemon[10345]: <root@pam> successful auth for user 'root@pam'
Jan 13 08:05:07 proxmoxch03-hp pvedaemon[58802]: <root@pam> successful auth for user 'root@pam'
Jan 13 08:05:08 proxmoxch03-hp pvedaemon[58802]: <root@pam> successful auth for user 'root@pam'
Jan 13 08:05:11 proxmoxch03-hp pvedaemon[11567]: <root@pam> successful auth for user 'root@pam'
Jan 13 08:05:11 proxmoxch03-hp pvedaemon[11567]: <root@pam> successful auth for user 'root@pam'
Jan 13 08:05:12 proxmoxch03-hp pvedaemon[11567]: <root@pam> successful auth for user 'root@pam'
Jan 13 08:05:12 proxmoxch03-hp pvedaemon[58802]: <root@pam> successful auth for user 'root@pam'
Jan 13 08:05:13 proxmoxch03-hp pvedaemon[10345]: <root@pam> successful auth for user 'root@pam'
Jan 13 08:05:13 proxmoxch03-hp pvedaemon[10345]: <root@pam> successful auth for user 'root@pam'
Jan 13 08:05:14 proxmoxch03-hp pvedaemon[58802]: <root@pam> successful auth for user 'root@pam'
Jan 13 08:05:15 proxmoxch03-hp pvedaemon[11567]: <root@pam> successful auth for user 'root@pam'
 

fireon

Famous Member
Oct 25, 2010
3,910
325
103
40
Austria/Graz
iteas.at
Thanks for the logs. This is really strange. Nothing is really see for the reboot... strange is also this lot logins... hmm. What model of HP Server to you have. Have you installed the HP tools?

Have you found the ASR/Watchdog in the HP BIOS settings and disable it too?
 

Medvenda

Member
Oct 5, 2016
40
0
6
35
Thanks for the logs. This is really strange. Nothing is really see for the reboot... strange is also this lot logins... hmm. What model of HP Server to you have. Have you installed the HP tools?

Have you found the ASR/Watchdog in the HP BIOS settings and disable it too?
CPUs
32 x Intel(R) Xeon(R) CPU E5-2670 0 @ 2.60GHz (2 Sockets)

I have no idea to stop the watchdog BIOS, just for nmi_watchdog i
I just update the /etc/default/grub with
GRUB_CMDLINE_LINUX_DEFAULT="quiet nmi_watchdog=0"
AND
# update-grub
 

fireon

Famous Member
Oct 25, 2010
3,910
325
103
40
Austria/Graz
iteas.at
So you don't know the model of you Server??? No sticker on it? Enter the bios and search for the watchdog/asr option, put on a monitor, enter with ILO... I thing that this "GRUB_CMDLINE_LINUX_DEFAULT="quiet nmi_watchdog=0"" have nothing to do with hardwarewatchdog. Don't forget to upgrade your bios firmware.
 

SilverNodashi

Active Member
Jul 30, 2017
128
4
38
42
I just had the same happen to me. The server rebooted as soon as we pulled the network cable out (had to swap them out for CAT 6E cables):

Here's the log just before the reboot:

Code:
Nov  7 12:14:34 virt1 pvedaemon[45503]: <root@pam> successful auth for user 'root@pam'
Nov  7 12:14:54 virt1 pvedaemon[46219]: <root@pam> starting task UPID:virt1:0000B3E7:008A5FEF:5A01879E:srvstop:eek:sd.0:root@pam:
Nov  7 12:14:58 virt1 pvedaemon[46219]: <root@pam> end task UPID:virt1:0000B3E7:008A5FEF:5A01879E:srvstop:eek:sd.0:root@pam: OK
Nov  7 12:17:26 virt1 pvedaemon[45503]: <root@pam> starting task UPID:virt1:0000B9A4:008A9B69:5A018836:srvstart:eek:sd.0:root@pam:
Nov  7 12:17:26 virt1 pvedaemon[45503]: <root@pam> end task UPID:virt1:0000B9A4:008A9B69:5A018836:srvstart:eek:sd.0:root@pam: OK
Nov  7 12:17:30 virt1 pvedaemon[12347]: <root@pam> starting task UPID:virt1:0000BA1B:008A9CFD:5A01883A:srvstart:eek:sd.0:root@pam:
Nov  7 12:17:30 virt1 pvedaemon[12347]: <root@pam> end task UPID:virt1:0000BA1B:008A9CFD:5A01883A:srvstart:eek:sd.0:root@pam: OK
Nov  7 12:17:50 virt1 pvedaemon[45503]: <root@pam> starting task UPID:virt1:0000BC09:008AA4D3:5A01884E:srvstart:eek:sd.0:root@pam:
Nov  7 12:17:50 virt1 pvedaemon[45503]: <root@pam> end task UPID:virt1:0000BC09:008AA4D3:5A01884E:srvstart:eek:sd.0:root@pam: OK
Nov  7 12:25:26 virt1 pvedaemon[45503]: <root@pam> successful auth for user 'root@pam'
Nov  7 12:29:35 virt1 pvedaemon[46219]: <root@pam> successful auth for user 'root@pam'
Nov  7 12:38:46 virt1 kernel: [92116.879909] i40e 0000:81:00.3 ens2f3: NIC Link is Down
Nov  7 12:38:46 virt1 kernel: [92116.879976] vmbr0: port 1(ens2f3) entered disabled state
Nov  7 12:39:20 virt1 kernel: [92150.793576] i40e 0000:81:00.0 ens2f0: NIC Link is Down
Nov  7 12:41:27 virt1 kernel: [    0.000000] random: get_random_bytes called from start_kernel+0x42/0x4f3 with crng_init=0
Nov  7 12:41:27 virt1 kernel: [    0.000000] Linux version 4.13.4-1-pve (root@nora) (gcc version 6.3.0 20170516 (Debian 6.3.0-18)) #1 SMP PVE 4.13.4-25 (Fri, 13 Oct 2017 08:59:53 +0200) ()
Nov  7 12:41:27 virt1 kernel: [    0.000000] Command line: BOOT_IMAGE=/boot/vmlinuz-4.13.4-1-pve root=/dev/mapper/pve-root ro quiet
Nov  7 12:41:27 virt1 kernel: [    0.000000] KERNEL supported cpus:
Nov  7 12:41:27 virt1 kernel: [    0.000000]   Intel GenuineIntel
Nov  7 12:41:27 virt1 kernel: [    0.000000]   AMD AuthenticAMD
Nov  7 12:41:27 virt1 kernel: [    0.000000]   Centaur CentaurHauls
Nov  7 12:41:27 virt1 kernel: [    0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'
Nov  7 12:41:27 virt1 kernel: [    0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'
Nov  7 12:41:27 virt1 kernel: [    0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'
Nov  7 12:41:27 virt1 kernel: [    0.000000] x86/fpu: xstate_offset[2]:  576, xstate_sizes[2]:  256
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get your own in 60 seconds.

Buy now!