Automatic Proxmox server reboot

Medvenda · Jan 14, 2017

fireon said:
So really?
This is the line look like system starting up:

Jan 13 08:11:56 proxmoxch03-hp rsyslogd: [origin software="rsyslogd" swVersion="8.4.2" x-pid="2690" x-info="http://www.rsyslog.com"] start

And you say this are the lines before?

Jan 13 16:47:17 proxmoxch03-hp systemd[1]: Started udev Kernel Device Manager.
Jan 13 16:47:17 proxmoxch03-hp systemd[1]: Starting LSB: Set preliminary keymap...
Jan 13 16:47:17 proxmoxch03-hp hdparm[546]: Setting parameters of disc: (none).

This is more later! This must be something like:

Jan 13 08:06:00 ...

Jan 13 07:05:13 proxmoxch03-hp pvedaemon[11567]: <root@pam> successful auth for user 'root@pam'
Jan 13 07:05:13 proxmoxch03-hp pvedaemon[11567]: <root@pam> successful auth for user 'root@pam'
Jan 13 07:05:14 proxmoxch03-hp pvedaemon[10345]: <root@pam> successful auth for user 'root@pam'
Jan 13 07:05:14 proxmoxch03-hp pvedaemon[58802]: <root@pam> successful auth for user 'root@pam'
Jan 13 07:10:52 proxmoxch03-hp systemd-timesyncd[2303]: interval/delta/delay/jitter/drift 2048s/-0.000s/0.020s/0.001s/+34ppm
Jan 13 07:17:01 proxmoxch03-hp CRON[64927]: (root) CMD ( cd / && run-parts --report /etc/cron.hourly)
Jan 13 07:21:17 proxmoxch03-hp smartd[2639]: Device: /dev/sda, failed to read Temperature
Jan 13 07:21:17 proxmoxch03-hp rrdcached[2689]: flushing old values
Jan 13 07:21:17 proxmoxch03-hp rrdcached[2689]: rotating journals
Jan 13 07:21:17 proxmoxch03-hp rrdcached[2689]: started new journal /var/lib/rrdcached/journal/rrd.journal.1484288477.535610
Jan 13 07:21:17 proxmoxch03-hp rrdcached[2689]: removing old journal /var/lib/rrdcached/journal/rrd.journal.1484281277.535583
Jan 13 07:25:29 proxmoxch03-hp pvedaemon[10345]: <root@pam> successful auth for user 'root@pam'
Jan 13 07:25:30 proxmoxch03-hp pvedaemon[11567]: <root@pam> successful auth for user 'root@pam'
Jan 13 07:25:30 proxmoxch03-hp pvedaemon[10345]: <root@pam> successful auth for user 'root@pam'

........

Jan 13 08:04:41 proxmoxch03-hp pvedaemon[11567]: <root@pam> successful auth for user 'root@pam'
Jan 13 08:04:41 proxmoxch03-hp pvedaemon[11567]: <root@pam> successful auth for user 'root@pam'
Jan 13 08:04:44 proxmoxch03-hp pvedaemon[11567]: <root@pam> successful auth for user 'root@pam'
Jan 13 08:04:45 proxmoxch03-hp pvedaemon[10345]: <root@pam> successful auth for user 'root@pam'
Jan 13 08:04:45 proxmoxch03-hp pvedaemon[58802]: <root@pam> successful auth for user 'root@pam'
Jan 13 08:04:46 proxmoxch03-hp pvedaemon[10345]: <root@pam> successful auth for user 'root@pam'
Jan 13 08:04:46 proxmoxch03-hp pvedaemon[11567]: <root@pam> successful auth for user 'root@pam'
Jan 13 08:04:47 proxmoxch03-hp pvedaemon[58802]: <root@pam> successful auth for user 'root@pam'
Jan 13 08:04:48 proxmoxch03-hp pvedaemon[11567]: <root@pam> successful auth for user 'root@pam'
Jan 13 08:04:48 proxmoxch03-hp pvedaemon[58802]: <root@pam> successful auth for user 'root@pam'
Jan 13 08:04:49 proxmoxch03-hp pvedaemon[11567]: <root@pam> successful auth for user 'root@pam'
Jan 13 08:04:49 proxmoxch03-hp pvedaemon[58802]: <root@pam> successful auth for user 'root@pam'
Jan 13 08:04:50 proxmoxch03-hp pvedaemon[11567]: <root@pam> successful auth for user 'root@pam'
Jan 13 08:04:51 proxmoxch03-hp pvedaemon[58802]: <root@pam> successful auth for user 'root@pam'
Jan 13 08:04:51 proxmoxch03-hp pvedaemon[11567]: <root@pam> successful auth for user 'root@pam'
Jan 13 08:04:52 proxmoxch03-hp pvedaemon[58802]: <root@pam> successful auth for user 'root@pam'
Jan 13 08:04:52 proxmoxch03-hp pvedaemon[10345]: <root@pam> successful auth for user 'root@pam'
Jan 13 08:04:53 proxmoxch03-hp pvedaemon[11567]: <root@pam> successful auth for user 'root@pam'
Jan 13 08:04:54 proxmoxch03-hp pvedaemon[58802]: <root@pam> successful auth for user 'root@pam'
Jan 13 08:04:54 proxmoxch03-hp pvedaemon[11567]: <root@pam> successful auth for user 'root@pam'
Jan 13 08:04:57 proxmoxch03-hp pvedaemon[58802]: <root@pam> successful auth for user 'root@pam'
Jan 13 08:04:57 proxmoxch03-hp pvedaemon[58802]: <root@pam> successful auth for user 'root@pam'
Jan 13 08:05:00 proxmoxch03-hp pvedaemon[58802]: <root@pam> successful auth for user 'root@pam'
Jan 13 08:05:01 proxmoxch03-hp pvedaemon[10345]: <root@pam> successful auth for user 'root@pam'
Jan 13 08:05:02 proxmoxch03-hp pvedaemon[11567]: <root@pam> successful auth for user 'root@pam'
Jan 13 08:05:02 proxmoxch03-hp pvedaemon[10345]: <root@pam> successful auth for user 'root@pam'
Jan 13 08:05:03 proxmoxch03-hp pvedaemon[58802]: <root@pam> successful auth for user 'root@pam'
Jan 13 08:05:03 proxmoxch03-hp pvedaemon[11567]: <root@pam> successful auth for user 'root@pam'
Jan 13 08:05:04 proxmoxch03-hp pvedaemon[58802]: <root@pam> successful auth for user 'root@pam'
Jan 13 08:05:05 proxmoxch03-hp pvedaemon[58802]: <root@pam> successful auth for user 'root@pam'
Jan 13 08:05:05 proxmoxch03-hp pvedaemon[11567]: <root@pam> successful auth for user 'root@pam'
Jan 13 08:05:06 proxmoxch03-hp pvedaemon[58802]: <root@pam> successful auth for user 'root@pam'
Jan 13 08:05:06 proxmoxch03-hp pvedaemon[11567]: <root@pam> successful auth for user 'root@pam'
Jan 13 08:05:07 proxmoxch03-hp pvedaemon[10345]: <root@pam> successful auth for user 'root@pam'

Jan 13 08:05:06 proxmoxch03-hp pvedaemon[11567]: <root@pam> successful auth for user 'root@pam'
Jan 13 08:05:07 proxmoxch03-hp pvedaemon[10345]: <root@pam> successful auth for user 'root@pam'
Jan 13 08:05:07 proxmoxch03-hp pvedaemon[58802]: <root@pam> successful auth for user 'root@pam'
Jan 13 08:05:08 proxmoxch03-hp pvedaemon[58802]: <root@pam> successful auth for user 'root@pam'
Jan 13 08:05:11 proxmoxch03-hp pvedaemon[11567]: <root@pam> successful auth for user 'root@pam'
Jan 13 08:05:11 proxmoxch03-hp pvedaemon[11567]: <root@pam> successful auth for user 'root@pam'
Jan 13 08:05:12 proxmoxch03-hp pvedaemon[11567]: <root@pam> successful auth for user 'root@pam'
Jan 13 08:05:12 proxmoxch03-hp pvedaemon[58802]: <root@pam> successful auth for user 'root@pam'
Jan 13 08:05:13 proxmoxch03-hp pvedaemon[10345]: <root@pam> successful auth for user 'root@pam'
Jan 13 08:05:13 proxmoxch03-hp pvedaemon[10345]: <root@pam> successful auth for user 'root@pam'
Jan 13 08:05:14 proxmoxch03-hp pvedaemon[58802]: <root@pam> successful auth for user 'root@pam'
Jan 13 08:05:15 proxmoxch03-hp pvedaemon[11567]: <root@pam> successful auth for user 'root@pam'

fireon · Jan 14, 2017

Thanks for the logs. This is really strange. Nothing is really see for the reboot... strange is also this lot logins... hmm. What model of HP Server to you have. Have you installed the HP tools?

Have you found the ASR/Watchdog in the HP BIOS settings and disable it too?

Medvenda · Jan 14, 2017

fireon said:
Thanks for the logs. This is really strange. Nothing is really see for the reboot... strange is also this lot logins... hmm. What model of HP Server to you have. Have you installed the HP tools?

Have you found the ASR/Watchdog in the HP BIOS settings and disable it too?

CPUs
32 x Intel(R) Xeon(R) CPU E5-2670 0 @ 2.60GHz (2 Sockets)

I have no idea to stop the watchdog BIOS, just for nmi_watchdog i
I just update the /etc/default/grub with
GRUB_CMDLINE_LINUX_DEFAULT="quiet nmi_watchdog=0"
AND
# update-grub

fireon · Jan 14, 2017

So you don't know the model of you Server??? No sticker on it? Enter the bios and search for the watchdog/asr option, put on a monitor, enter with ILO... I thing that this "GRUB_CMDLINE_LINUX_DEFAULT="quiet nmi_watchdog=0"" have nothing to do with hardwarewatchdog. Don't forget to upgrade your bios firmware.

Magneto · Nov 7, 2017

I just had the same happen to me. The server rebooted as soon as we pulled the network cable out (had to swap them out for CAT 6E cables):

Here's the log just before the reboot:

Code:

Nov  7 12:14:34 virt1 pvedaemon[45503]: <root@pam> successful auth for user 'root@pam'
Nov  7 12:14:54 virt1 pvedaemon[46219]: <root@pam> starting task UPID:virt1:0000B3E7:008A5FEF:5A01879E:srvstop:eek:sd.0:root@pam:
Nov  7 12:14:58 virt1 pvedaemon[46219]: <root@pam> end task UPID:virt1:0000B3E7:008A5FEF:5A01879E:srvstop:eek:sd.0:root@pam: OK
Nov  7 12:17:26 virt1 pvedaemon[45503]: <root@pam> starting task UPID:virt1:0000B9A4:008A9B69:5A018836:srvstart:eek:sd.0:root@pam:
Nov  7 12:17:26 virt1 pvedaemon[45503]: <root@pam> end task UPID:virt1:0000B9A4:008A9B69:5A018836:srvstart:eek:sd.0:root@pam: OK
Nov  7 12:17:30 virt1 pvedaemon[12347]: <root@pam> starting task UPID:virt1:0000BA1B:008A9CFD:5A01883A:srvstart:eek:sd.0:root@pam:
Nov  7 12:17:30 virt1 pvedaemon[12347]: <root@pam> end task UPID:virt1:0000BA1B:008A9CFD:5A01883A:srvstart:eek:sd.0:root@pam: OK
Nov  7 12:17:50 virt1 pvedaemon[45503]: <root@pam> starting task UPID:virt1:0000BC09:008AA4D3:5A01884E:srvstart:eek:sd.0:root@pam:
Nov  7 12:17:50 virt1 pvedaemon[45503]: <root@pam> end task UPID:virt1:0000BC09:008AA4D3:5A01884E:srvstart:eek:sd.0:root@pam: OK
Nov  7 12:25:26 virt1 pvedaemon[45503]: <root@pam> successful auth for user 'root@pam'
Nov  7 12:29:35 virt1 pvedaemon[46219]: <root@pam> successful auth for user 'root@pam'
Nov  7 12:38:46 virt1 kernel: [92116.879909] i40e 0000:81:00.3 ens2f3: NIC Link is Down
Nov  7 12:38:46 virt1 kernel: [92116.879976] vmbr0: port 1(ens2f3) entered disabled state
Nov  7 12:39:20 virt1 kernel: [92150.793576] i40e 0000:81:00.0 ens2f0: NIC Link is Down
Nov  7 12:41:27 virt1 kernel: [    0.000000] random: get_random_bytes called from start_kernel+0x42/0x4f3 with crng_init=0
Nov  7 12:41:27 virt1 kernel: [    0.000000] Linux version 4.13.4-1-pve (root@nora) (gcc version 6.3.0 20170516 (Debian 6.3.0-18)) #1 SMP PVE 4.13.4-25 (Fri, 13 Oct 2017 08:59:53 +0200) ()
Nov  7 12:41:27 virt1 kernel: [    0.000000] Command line: BOOT_IMAGE=/boot/vmlinuz-4.13.4-1-pve root=/dev/mapper/pve-root ro quiet
Nov  7 12:41:27 virt1 kernel: [    0.000000] KERNEL supported cpus:
Nov  7 12:41:27 virt1 kernel: [    0.000000]   Intel GenuineIntel
Nov  7 12:41:27 virt1 kernel: [    0.000000]   AMD AuthenticAMD
Nov  7 12:41:27 virt1 kernel: [    0.000000]   Centaur CentaurHauls
Nov  7 12:41:27 virt1 kernel: [    0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'
Nov  7 12:41:27 virt1 kernel: [    0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'
Nov  7 12:41:27 virt1 kernel: [    0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'
Nov  7 12:41:27 virt1 kernel: [    0.000000] x86/fpu: xstate_offset[2]:  576, xstate_sizes[2]:  256

Search

Search

Automatic Proxmox server reboot

Medvenda

Member

fireon

Distinguished Member

Medvenda

Member

fireon

Distinguished Member

Magneto

Well-Known Member