Constant Kernel Panics on PVE 9 fresh install

P

pakyrs

Active Member
Jan 12, 2020
25
0
41
45
Hi All,

I keep on getting Kernel Panics and Hangs with the Latest PVE 9 freshly installed on my server.



1763133713208.png

Journalctl doesn't give me any info, this is the log from an hour ago, altough it crashed 10min ago, I have omitted the new boot logs.

Bash: 
journalctl --since "1 hour ago"
Nov 14 14:35:01 nibbler CRON[78122]: pam_unix(cron:session): session opened for user root(uid=0) by root(uid=0)
Nov 14 14:35:01 nibbler CRON[78124]: (root) CMD (command -v debian-sa1 > /dev/null && debian-sa1 1 1)
Nov 14 14:35:01 nibbler CRON[78122]: pam_unix(cron:session): session closed for user root
Nov 14 14:45:01 nibbler CRON[82544]: pam_unix(cron:session): session opened for user root(uid=0) by root(uid=0)
Nov 14 14:45:01 nibbler CRON[82546]: (root) CMD (command -v debian-sa1 > /dev/null && debian-sa1 1 1)
Nov 14 14:45:01 nibbler CRON[82544]: pam_unix(cron:session): session closed for user root
Nov 14 14:55:01 nibbler CRON[86916]: pam_unix(cron:session): session opened for user root(uid=0) by root(uid=0)
Nov 14 14:55:01 nibbler CRON[86918]: (root) CMD (command -v debian-sa1 > /dev/null && debian-sa1 1 1)
Nov 14 14:55:01 nibbler CRON[86916]: pam_unix(cron:session): session closed for user root
Nov 14 14:59:21 nibbler pvedaemon[3106]: <root@pam> successful auth for user 'root@pam'
Nov 14 14:59:22 nibbler pvedaemon[3106]: <root@pam> starting task UPID:nibbler:00015AF7:00106197:691743CA:vncproxy:107:root@pam:
Nov 14 14:59:22 nibbler pvedaemon[88823]: starting vnc proxy UPID:nibbler:00015AF7:00106197:691743CA:vncproxy:107:root@pam:
Nov 14 14:59:35 nibbler pvedaemon[3107]: worker exit
Nov 14 14:59:37 nibbler pvedaemon[3105]: worker 3107 finished
Nov 14 14:59:37 nibbler pvedaemon[3105]: starting 1 worker(s)
Nov 14 14:59:37 nibbler pvedaemon[3105]: worker 88919 started
Nov 14 15:00:12 nibbler pvedaemon[3108]: worker exit
Nov 14 15:00:12 nibbler pvedaemon[3105]: worker 3108 finished
Nov 14 15:00:12 nibbler pvedaemon[3105]: starting 1 worker(s)
Nov 14 15:00:12 nibbler pvedaemon[3105]: worker 89181 started
Nov 14 15:00:38 nibbler pvedaemon[3106]: worker exit
Nov 14 15:00:38 nibbler pvedaemon[3105]: worker 3106 finished
Nov 14 15:00:38 nibbler pvedaemon[3105]: starting 1 worker(s)
Nov 14 15:00:38 nibbler pvedaemon[3105]: worker 89355 started
Nov 14 15:00:39 nibbler smartd[2598]: Device: /dev/sde [SAT], SMART Usage Attribute: 204 Soft_ECC_Correction changed from 95 to 96
Nov 14 15:05:01 nibbler CRON[91763]: pam_unix(cron:session): session opened for user root(uid=0) by root(uid=0)
Nov 14 15:05:01 nibbler CRON[91765]: (root) CMD (command -v debian-sa1 > /dev/null && debian-sa1 1 1)
Nov 14 15:05:01 nibbler CRON[91763]: pam_unix(cron:session): session closed for user root
Nov 14 15:05:04 nibbler pvedaemon[88919]: <root@pam> update VM 107: -balloon 4096 -delete shares -memory 10240
Nov 14 15:05:04 nibbler pvedaemon[88919]: cannot delete 'shares' - not set in current configuration!
 
Hi!
pakyrs said:
Journalctl doesn't give me any info, this is the log from an hour ago, altough it crashed 10min ago, I have omitted the new boot logs.
Click to expand...
journalctl with PAGER set will usually open in that pager (typically it's less) and therefore you need to page down for journalctl --since="1 hour ago". For a better view of the log from the end of the last boot, you can run journalctl -b -1 -e. What is the output there?
 
Either way, I'd first check if your hardware is alright, e.g. all cables and the RAM modules are connected correctly and aren't loose, there BIOS configuration is reset (no overlocking, etc.), and so forth. If it doesn't fix the problem, then a full boot log would be beneficial in finding the cause of the problem.
 
Bash: 
journalctl -b -1 -e
Nov 14 13:27:06 nibbler pveproxy[3121]: worker 47899 started
Nov 14 13:27:07 nibbler pveproxy[47898]: worker exit
Nov 14 13:27:46 nibbler pveproxy[3121]: worker 19895 finished
Nov 14 13:27:46 nibbler pveproxy[3121]: starting 1 worker(s)
Nov 14 13:27:46 nibbler pveproxy[3121]: worker 48239 started
Nov 14 13:27:49 nibbler pveproxy[48237]: got inotify poll request in wrong process - disabling inotify
Nov 14 13:30:39 nibbler smartd[2598]: Device: /dev/sdd [SAT], SMART Usage Attribute: 194 Temperature_Celsius changed from 63 to 62
Nov 14 13:30:39 nibbler smartd[2598]: Device: /dev/sde [SAT], SMART Usage Attribute: 204 Soft_ECC_Correction changed from 93 to 94
Nov 14 13:32:41 nibbler pvedaemon[3108]: <root@pam> successful auth for user 'root@pam'
Nov 14 13:35:01 nibbler CRON[51405]: pam_unix(cron:session): session opened for user root(uid=0) by root(uid=0)
Nov 14 13:35:01 nibbler CRON[51407]: (root) CMD (command -v debian-sa1 > /dev/null && debian-sa1 1 1)
Nov 14 13:35:01 nibbler CRON[51405]: pam_unix(cron:session): session closed for user root
Nov 14 13:37:29 nibbler pveproxy[25196]: worker exit
Nov 14 13:37:29 nibbler pveproxy[3121]: worker 25196 finished
Nov 14 13:37:29 nibbler pveproxy[3121]: starting 1 worker(s)
Nov 14 13:37:29 nibbler pveproxy[3121]: worker 52516 started
Nov 14 13:44:40 nibbler pvedaemon[3106]: <root@pam> end task UPID:nibbler:000066ED:00040021:69172418:vncproxy:107:root@pam: OK
Nov 14 13:44:40 nibbler pveproxy[48237]: worker exit
Nov 14 13:45:01 nibbler CRON[55986]: pam_unix(cron:session): session opened for user root(uid=0) by root(uid=0)
Nov 14 13:45:01 nibbler CRON[55991]: (root) CMD (command -v debian-sa1 > /dev/null && debian-sa1 1 1)
Nov 14 13:45:01 nibbler CRON[55986]: pam_unix(cron:session): session closed for user root
Nov 14 13:55:01 nibbler CRON[60284]: pam_unix(cron:session): session opened for user root(uid=0) by root(uid=0)
Nov 14 13:55:01 nibbler CRON[60286]: (root) CMD (command -v debian-sa1 > /dev/null && debian-sa1 1 1)
Nov 14 13:55:01 nibbler CRON[60284]: pam_unix(cron:session): session closed for user root
Nov 14 14:00:39 nibbler smartd[2598]: Device: /dev/sdd [SAT], SMART Usage Attribute: 194 Temperature_Celsius changed from 62 to 63
Nov 14 14:00:39 nibbler smartd[2598]: Device: /dev/sde [SAT], SMART Usage Attribute: 194 Temperature_Celsius changed from 26 to 25
Nov 14 14:00:39 nibbler smartd[2598]: Device: /dev/sde [SAT], SMART Usage Attribute: 204 Soft_ECC_Correction changed from 94 to 95
Nov 14 14:05:01 nibbler CRON[64773]: pam_unix(cron:session): session opened for user root(uid=0) by root(uid=0)
Nov 14 14:05:01 nibbler CRON[64775]: (root) CMD (command -v debian-sa1 > /dev/null && debian-sa1 1 1)
Nov 14 14:05:01 nibbler CRON[64773]: pam_unix(cron:session): session closed for user root
Nov 14 14:15:01 nibbler CRON[69076]: pam_unix(cron:session): session opened for user root(uid=0) by root(uid=0)
Nov 14 14:15:01 nibbler CRON[69078]: (root) CMD (command -v debian-sa1 > /dev/null && debian-sa1 1 1)
Nov 14 14:15:01 nibbler CRON[69076]: pam_unix(cron:session): session closed for user root
Nov 14 14:17:01 nibbler CRON[69991]: pam_unix(cron:session): session opened for user root(uid=0) by root(uid=0)
Nov 14 14:17:01 nibbler CRON[69993]: (root) CMD (cd / && run-parts --report /etc/cron.hourly)
Nov 14 14:17:01 nibbler CRON[69991]: pam_unix(cron:session): session closed for user root
Nov 14 14:25:01 nibbler CRON[73752]: pam_unix(cron:session): session opened for user root(uid=0) by root(uid=0)
Nov 14 14:25:01 nibbler CRON[73754]: (root) CMD (command -v debian-sa1 > /dev/null && debian-sa1 1 1)
Nov 14 14:25:01 nibbler CRON[73752]: pam_unix(cron:session): session closed for user root
Nov 14 14:35:01 nibbler CRON[78122]: pam_unix(cron:session): session opened for user root(uid=0) by root(uid=0)
Nov 14 14:35:01 nibbler CRON[78124]: (root) CMD (command -v debian-sa1 > /dev/null && debian-sa1 1 1)
Nov 14 14:35:01 nibbler CRON[78122]: pam_unix(cron:session): session closed for user root
Nov 14 14:45:01 nibbler CRON[82544]: pam_unix(cron:session): session opened for user root(uid=0) by root(uid=0)
Nov 14 14:45:01 nibbler CRON[82546]: (root) CMD (command -v debian-sa1 > /dev/null && debian-sa1 1 1)
Nov 14 14:45:01 nibbler CRON[82544]: pam_unix(cron:session): session closed for user root
Nov 14 14:55:01 nibbler CRON[86916]: pam_unix(cron:session): session opened for user root(uid=0) by root(uid=0)
Nov 14 14:55:01 nibbler CRON[86918]: (root) CMD (command -v debian-sa1 > /dev/null && debian-sa1 1 1)
Nov 14 14:55:01 nibbler CRON[86916]: pam_unix(cron:session): session closed for user root
Nov 14 14:59:21 nibbler pvedaemon[3106]: <root@pam> successful auth for user 'root@pam'
Nov 14 14:59:22 nibbler pvedaemon[3106]: <root@pam> starting task UPID:nibbler:00015AF7:00106197:691743CA:vncproxy:107:root@pam:
Nov 14 14:59:22 nibbler pvedaemon[88823]: starting vnc proxy UPID:nibbler:00015AF7:00106197:691743CA:vncproxy:107:root@pam:
Nov 14 14:59:35 nibbler pvedaemon[3107]: worker exit
Nov 14 14:59:37 nibbler pvedaemon[3105]: worker 3107 finished
Nov 14 14:59:37 nibbler pvedaemon[3105]: starting 1 worker(s)
Nov 14 14:59:37 nibbler pvedaemon[3105]: worker 88919 started
Nov 14 15:00:12 nibbler pvedaemon[3108]: worker exit
Nov 14 15:00:12 nibbler pvedaemon[3105]: worker 3108 finished
Nov 14 15:00:12 nibbler pvedaemon[3105]: starting 1 worker(s)
Nov 14 15:00:12 nibbler pvedaemon[3105]: worker 89181 started
Nov 14 15:00:38 nibbler pvedaemon[3106]: worker exit
Nov 14 15:00:38 nibbler pvedaemon[3105]: worker 3106 finished
Nov 14 15:00:38 nibbler pvedaemon[3105]: starting 1 worker(s)
Nov 14 15:00:38 nibbler pvedaemon[3105]: worker 89355 started
Nov 14 15:00:39 nibbler smartd[2598]: Device: /dev/sde [SAT], SMART Usage Attribute: 204 Soft_ECC_Correction changed from 95 to 96
Nov 14 15:05:01 nibbler CRON[91763]: pam_unix(cron:session): session opened for user root(uid=0) by root(uid=0)
Nov 14 15:05:01 nibbler CRON[91765]: (root) CMD (command -v debian-sa1 > /dev/null && debian-sa1 1 1)
Nov 14 15:05:01 nibbler CRON[91763]: pam_unix(cron:session): session closed for user root
Nov 14 15:05:04 nibbler pvedaemon[88919]: <root@pam> update VM 107: -balloon 4096 -delete shares -memory 10240
Nov 14 15:05:04 nibbler pvedaemon[88919]: cannot delete 'shares' - not set in current configuration!

Thanks for stepping in, I removed basically every device attached to the system that I thought could add a layer of problems. I also tested the ram with memtest86 and it is sound.

I don't see anything in the logs that point me to that. Before I reinstalled proxmox fresh, I had a kdump from previous crash they are rather large not sure how to share these.
 
You must log in or register to reply here.
Top Bottom