VM freezes irregularly

LiFE1688 said:
There really isn't a need to get a CWWK N5105/N6005 box, you are better off with the 8505 for better performance, and around the same price, or you can look at the i3/i5/i7 12th gen. The only thing to note is Promox, you need Kernel 6.1 or above.

The only advantage for Protectli is Coreboot, imo, but the processor is usually older. So no N5105/N6005/8505 or the i3/5/7 12th gen. If you are not on the Coreboot BIOS and using the AMI one, you should consider switching. Just make sure to install Proxmox using UEFI and not Legacy BIOS. Since Coreboot is an Opensource modular BIOS, you will find BIOS upgrades easily even when the manufacturer stops the BIOS development. (maybe).

N5105 v3/v4/v5 BIOS should work with the N6005 v3/v4/v5 CWWK boxes as well.

@AdriftAtlas on the STH forums, "chicken blood" version is not the complete slang, The whole slang is "chicken blood all open" which basically means, all functions enabled for changing without regards to the wisdom/need of it. I can't read / understand Chinese, but that's what a Mandarin speaking person doing the translation for me, explained.
Click to expand...


I'm Chinese.

chicken blood,It means unlocking all performance，After upgrading this bios。No matter how many degrees, the cpu will maintain the highest frequency，Even if the cpu temperature reaches 100, it can keep operating at 2.8ghz。
 
leslieqwq said:
I'm Chinese.

chicken blood,It means unlocking all performance，After upgrading this bios。No matter how many degrees, the cpu will maintain the highest frequency，Even if the cpu temperature reaches 100, it can keep operating at 2.8ghz。
Click to expand...
That sounds pretty dangerous. So, it's probably best to configure the BIOS with some reasonable safety settings. Anyone knows which settings are most important?
 
yup sounds like grilled cpu :) a disaster waiting to happen. throttling and similar measures are there for a reason, otherwise you get such things as a cpu unsoldering itself :)
 
Unfortunately it looks like yesterday my Debian 11 Docker VM froze again after almost a month even though I have the newest microcode... :(
 
I'm glad that I found this thread, hopefully somewhere in here I can solve the crashes I'm seeing on a N5105 in a Beelink U59.

Before finding this thread I'd already opted into Kernel 6.2, thanks PVE team, but I was still seeing VMs hang with 100% usage of a single core. The Proxmox host stayed fine, and there was nothing getting logged before the VMs hang.

I updated to intel-microcode from Bullseye nonfree, I had been on:

Code: 
cat /proc/cpuinfo | grep micro
microcode    : 0x1b
microcode    : 0x1b
microcode    : 0x1b
microcode    : 0x1b

before. Fingers crossed that just a microcode update fixes it.

Edit: Oh, I'm using the "powersave" governor and "host" CPU type on the guests. If this doesn't fix it I'm just going to make this a physical box instead and not use KVM on it.
 
Last edited:
FWIW, I just upgraded my bios to the latest from CWWK (N5105-V3-V4-V5 microcode update released on 2023-04-18 (chicken blood version)).

I made the following tweaks to improve power consumption and to place some limits:
-changed Platform PL1 in Turbo Settings to 10000 (from 50000)
-changed Platform PL2 in Turbo Settings to 22000 (from 65000)
-changed PL1 in Turbo Settings to 10000 (from 50000)
-changed PL2 in Turbo Settings to 22000 (from 65000)
-changed ASPM to Auto (PCH-IO->PCIe Config, Root Ports 1,4,5,6,7,8)

I'm running Proxmox with OPNsense and a few small LXC containers. CPU is usually ~3%. Power draw is around 8.5W.

Everything seems stable but it's only been a few hours.
 
  • Like
Reactions: Neuer_User and MrHello
Using N5105 CPU with kernel 5.15.104-1-pve.
After updating the microcode to 0x24000024, I have 4 Linux VMs running flawlessly for 3 weeks. It's my longest record until now.
 
kociubin said:
FWIW, I just upgraded my bios to the latest from CWWK (N5105-V3-V4-V5 microcode update released on 2023-04-18 (chicken blood version)).

I made the following tweaks to improve power consumption and to place some limits:
-changed Platform PL1 in Turbo Settings to 10000 (from 50000)
-changed Platform PL2 in Turbo Settings to 22000 (from 65000)
-changed PL1 in Turbo Settings to 10000 (from 50000)
-changed PL2 in Turbo Settings to 22000 (from 65000)
-changed ASPM to Auto (PCH-IO->PCIe Config, Root Ports 1,4,5,6,7,8)

I'm running Proxmox with OPNsense and a few small LXC containers. CPU is usually ~3%. Power draw is around 8.5W.

Everything seems stable but it's only been a few hours.
Click to expand...
What about thermal limits? Anything restricted here?
 
apowers said:
My uptime is currently "up 2 weeks, 3 days, 2 hours, 55 minutes" on a Beelink U59 (with N5105). Previous uptime prior to the changes below was never more than 6 hours.

Changes I made:
  1. Upgrade Ubuntu VM kernel to 6.2
    1. wget https://kernel.ubuntu.com/~kernel-p...0-generic_6.2.0-060200.202302191831_amd64.deb
    2. wget https://kernel.ubuntu.com/~kernel-p....2.0-060200_6.2.0-060200.202302191831_all.deb
    3. wget https://kernel.ubuntu.com/~kernel-p...0-generic_6.2.0-060200.202302191831_amd64.deb
    4. wget https://kernel.ubuntu.com/~kernel-p...0-generic_6.2.0-060200.202302191831_amd64.deb
    5. sudo dpkg -i *.deb
  2. Update Microcode on Proxmox Debian
    1. wget https://r-1.ch/intel-microcode_3.20221108.2_amd64.deb
    2. dpkg -i intel-microcode_3.20221108.2_amd64.deb
    3. optional
      1. cat /proc/cpuinfo
  3. Update Ubuntu VM GRUB
    1. Edit /etc/default/grub and modified the value of GRUB_CMDLINE_LINUX_DEFAULT to be "intel_pstate=disable quiet"
    2. Save the changes and run the update-grub command
    3. reboot
  4. Update Ubuntu VM Microcode Blacklist
    1. cd /etc/modprobe.d
    2. mv intel-microcode-blacklist.conf intel-microcode-blacklist.conf~
I don't believe #3 or #4 were necessary, but I haven't tested rolling them back yet.

Thanks to everyone for the help on this.
Click to expand...

Thank you Apowers for the detailed write up!

I have 2 VM's running one with steps 1 3 & 4 and one without. Both seem to no longer crash. So likely only getting updated microcode will fix it.
 
I'm having the same problem with a J4125 (from a GMK Nucbox)

I've tryied to update the microcode to the latest version, but even after getting the newest version it still shows the 2022 version:

Code: 
wget http://ftp.debian.org/debian/pool/non-free-firmware/i/intel-microcode/intel-microcode_3.20230214.1_amd64.deb
dpkg -i intel-microcode_3.20230214.1_amd64.deb
reboot

dmesg | grep microcode
[    0.000000] microcode: microcode updated early to revision 0x22, date = 2022-09-20
[    1.359281] microcode: sig=0x706a8, pf=0x1, revision=0x22
[    1.359353] microcode: Microcode Update Driver: v2.2.


After adding the debian repo with the microcode and running "apt-get install intel-microcode" i get:
Code: 
intel-microcode is already the newest version (3.20230214.1).

But the "dmesg | grep microcode" still shows revision 0x22
Is my cpu not compatible with the 0x24 microcode?
 
Latest Microcode for Gemini Lake (Pentium J5040/N5030, Celeron J4125/J4025/N4020/N4120) is 0x00000022
Latest Microcode for Jasper Lake (Pentium N6000/N6005, Celeron N4500/N4505/N5100/N5105) is 0x24000024
The microcode is different for every series of processors, so it is not 1 microcode for every Intel processor.

For more information, check
https://github.com/intel/Intel-Linux-Processor-Microcode-Data-Files/releases

Debian has finally released the 3.20230214.1 in main non-free repo, no need to manually update, just add non-free to the apt source and use apt install.

@TiagoTab your VM crashes or reboots randomly in Proxmox? I have a J4125 from CWWK, it works flawlessly, and does not have the same issues as the Jasper Lake in Proxmox.
 
Last edited:
TiagoTab said:
I'm having the same problem with a J4125 (from a GMK Nucbox)

I've tryied to update the microcode to the latest version, but even after getting the newest version it still shows the 2022 version:

Code: 
wget http://ftp.debian.org/debian/pool/non-free-firmware/i/intel-microcode/intel-microcode_3.20230214.1_amd64.deb
dpkg -i intel-microcode_3.20230214.1_amd64.deb
reboot

dmesg | grep microcode
[    0.000000] microcode: microcode updated early to revision 0x22, date = 2022-09-20
[    1.359281] microcode: sig=0x706a8, pf=0x1, revision=0x22
[    1.359353] microcode: Microcode Update Driver: v2.2.


After adding the debian repo with the microcode and running "apt-get install intel-microcode" i get:
Code: 
intel-microcode is already the newest version (3.20230214.1).

But the "dmesg | grep microcode" still shows revision 0x22
Is my cpu not compatible with the 0x24 microcode?
Click to expand...
Note: The date displayed does not correspond to the version of the [intel-microcode] package installed. It does show the last time Intel updated the microcode that corresponds to the specific hardware being updated.
https://wiki.archlinux.org/title/microcode#Verifying_that_microcode_got_updated_on_boot
 
Last edited:
LiFE1688 said:
Latest Microcode for Gemini Lake (Pentium J5040/N5030, Celeron J4125/J4025/N4020/N4120) is 0x00000022
Latest Microcode for Jasper Lake (Pentium N6000/N6005, Celeron N4500/N4505/N5100/N5105) is 0x24000024
The microcode is different for every series of processors, so it is not 1 microcode for every Intel processor.

For more information, check
https://github.com/intel/Intel-Linux-Processor-Microcode-Data-Files/releases

Debian has finally released the 3.20230214.1 in main non-free repo, no need to manually update, just add non-free to the apt source and use apt install.

@TiagoTab your VM crashes or reboots randomly in Proxmox? I have a J4125 from CWWK, it works flawlessly, and does not have the same issues as the Jasper Lake in Proxmox.
Click to expand...

Proxmox itself crashes (Along will all VMs) and requires a reboot, i've tried windows server and had no issues for months, but as i'm running linux vms i want to use proxmox.

Could it be a firmware issue from the manufacturer? At this point, i think i'm forced to replace the machine (its a pitty as the power consumption is super low)
 
If the Host is crashing, it is not the same issues as the people using Jasper Lake processors are facing.

Your issue is definitely something else.
Maybe hardware related (maybe):
  • Over heating (Try putting a fan on top of the device)
  • Faulty Power adapter (Not providing enough power)
  • Faulty RAM
The above are the usual suspects.
 
  • Like
Reactions: nanoken
Glad I found this thread, Upgraded to 6.1 Kernel and Ran the Microcode Command. Its showing this? Am I all good guys?

Had the same issue of VM crashing every 1-5 days but host super stable.

CPU:N5100
Output
[ 0.000000] microcode: microcode updated early to revision 0x24000024, date = 2022-09-02
[ 0.129391] SRBDS: Vulnerable: No microcode
[ 1.260103] microcode: sig=0x906c0, pf=0x1, revision=0x24000024
[ 1.260116] microcode: Microcode Update Driver: v2.2.

+

intel-microcode is already the newest version (3.20230214.1).

7 Days PFSENSE Uptime and Counting since change.
 
Last edited:
i stumbled over this aswell as my opnsense VM keeps crashing on random intervals.

error is VM status internal error and log says:
May 19 22:51:49 pve-m QEMU[976]: KVM internal error. Suberror: 3

i ran the microcode command and got the following output:
dmesg | grep microcode
[ 0.145069] MMIO Stale Data: Vulnerable: Clear CPU buffers attempted, no microcode
[ 0.145070] SRBDS: Vulnerable: No microcode
[ 1.161033] microcode: sig=0x906c0, pf=0x1, revision=0x1d
[ 1.161046] microcode: Microcode Update Driver: v2.2.

I have a Teklager (topton rebranded i guess), N5105 with 4 i226 nics.

is my go to here to update microcode? (if so how do i do that :eek: )
 
Last edited:
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom
Back