Server Crashing since upgrade

[John Allison]

[SOLVED]

One of our Dell Proxmox servers has crashed 3 times over the last couple of weeks, following the application of some recent(ish) updates. Prior to this the server was working fine.

Its a poweredge R630 with:
CPU(s): 64 x Intel(R) Xeon(R) CPU E5-2683 v4 @ 2.10GHz (2 Sockets)
Kernel Version: Linux 4.15.18-14-pve #1 SMP PVE 4.15.18-39 (Wed, 15 May 2019 06:56:23 +0200)
PVE Manager Version: pve-manager/5.4-6/aa7856c5

The BIOS reports:
CPU 1 machine check error detected.

The linux log files dont contain any clues.

How can i find out whats happening and get it fixed?
I dont want to have to reinstall as this machine contains some pretty big VM's.

Thanks

 

[oguz]

hi,

is your firmware / bios up to date?

how often do the crashes happen?

i've also found this on the intel site[0], which looks similar

[0]: https://www.intel.com/content/www/u...server-products/rack_scale_design_direct.html

 

[John Allison]

Thanks for the reply, firmware was up to date, creashing only happened once a week or so, seems like when the server was under heavy load.
I think its all sorted now tho, contacted Dell support and they analysed the bios logs and identified an issue with one of the 32GB ram chip. Was advised to reseat the chip, and thats when i noticed it wasnt clipped into place correctly! doh!

 

[oguz]

i'm glad the issue is gone. you can mark the thread [SOLVED] if you edit your first post.