Ubuntu 20.04.04 machine freezes

AxisNL

New Member
Jun 4, 2022
12
0
1
Hi, I have proxmox on a few few rented dedicated boxes, and for the first time this morning one of the vms on one of these servers decided to freeze. Full lockup, not responding anymore. According to the PVE gui the cpu continued at around 25% steady, but no network i/o, and no response on the console. Only way was to reset the vm, and it start up normally again.

Incidents happen. Hoewer, an hour later or so, it hang again. Reset it. 5 minutes later, it hang again. Then I rebooted the proxmox machine. It's been up so far, but I wonder the vm will freeze again. Oh, it just did.

PVE is up to date (no-subscription) and the vm is also up to date to the latest version (
Linux server-27.stream-server.nl 5.4.0-122-generic #138-Ubuntu SMP Wed Jun 22 15:00:31 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux)

Nothing in the PVE logs, nothing in the guests's journalctl, nothing. How would i start troubleshooting this problem?
 

tuxis

Well-Known Member
Jan 3, 2014
171
78
48
Ede, NL
www.tuxis.nl
Please post the full config of the VM.

The 25% CPU usage is 'normal' on a crash. You probably have 4 cpu's configured, upon crashing, one vCPU is using 100%.

Do you get something on the console when you keep it open, waiting for it to crash?
 

AxisNL

New Member
Jun 4, 2022
12
0
1
If kept a journalctl -f open, saw absolutely nothing change, it just stops show messages.

Code:
agent: 1
boot: order=scsi0;ide2;net0
cores: 2
ide2: none,media=cdrom
memory: 10240
meta: creation-qemu=6.2.0,ctime=1658180975
name: server-27
net0: virtio=00:0c:29:c0:**:**,bridge=vmbr0
numa: 0
onboot: 1
ostype: l26
scsi0: local-lvm:vm-100-disk-0,size=150G
scsihw: virtio-scsi-pci
smbios1: uuid=b4d7352c-50f3-45b2-8f7b-a9cdb7916810
sockets: 2
vmgenid: dde96147-bfbc-41ec-8fa1-330e9936f571

(i sensored part of the mac)
 

tuxis

Well-Known Member
Jan 3, 2014
171
78
48
Ede, NL
www.tuxis.nl
Is it running (proxmox) backups by any chance? Any other VM's on that node that suffer from issues?
 

AxisNL

New Member
Jun 4, 2022
12
0
1
No backups running at the time (and nothing shows in the vm's task history as well), and no other vm's on the machine. It's been up for a few hours now though, but running on a ticking timebomb is scaring me, not knowing when it will go down again.

In the meanwhile, I've created a second ubuntu 20 vm, and copied all user data there as backup and tonight I'm switching over to the second vm.
 

AxisNL

New Member
Jun 4, 2022
12
0
1
Hmm.. the problematic machine is running:

Code:
Linux tel-pve-27 5.15.39-1-pve #1 SMP PVE 5.15.39-1 (Wed, 22 Jun 2022 17:22:00 +0200) x86_64 GNU/Linux

Most other machines where I don't have a problem run :

Code:
Linux tel-pve-24 5.15.35-3-pve #1 SMP PVE 5.15.35-6 (Fri, 17 Jun 2022 13:42:35 +0200) x86_64 GNU/Linux

But I have a few more of those 5.15.39-1 machines without problems (but perhaps less load).
 

AxisNL

New Member
Jun 4, 2022
12
0
1
And the locking up vm is running:

Code:
Linux server-27.stream-server.nl 5.4.0-122-generic #138-Ubuntu SMP Wed Jun 22 15:00:31 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

where it's working brother is running:

Code:
Linux server-24.stream-server.nl 5.4.0-121-generic #137-Ubuntu SMP Wed Jun 15 13:33:07 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
 

AxisNL

New Member
Jun 4, 2022
12
0
1
@IlyaK, which exact kernel versions of proxmox and ubuntu do you run? Can you remember what you upgraded then?
 

IlyaK

Member
Apr 15, 2019
2
0
6
34
@IlyaK, which exact kernel versions of proxmox and ubuntu do you run? Can you remember what you upgraded then?
I tried many versions of proxmox kernel and didn't succeed. But when i change ubuntu kernel from 5.4.0-122-generic to 5.4.0-121-generic, the problem was gone -i have two days without any freezes.
 

sam007

New Member
Jun 28, 2022
10
1
3
I tried many versions of proxmox kernel and didn't succeed. But when i change ubuntu kernel from 5.4.0-122-generic to 5.4.0-121-generic, the problem was gone -i have two days without any freezes.
updates:
I alos have upgraded my kernel from 5.4.0-122-generic to another, after almost five days , the issues again, freeze,
I DO NOT know what to do next
 

fiona

Proxmox Staff Member
Staff member
Aug 1, 2019
2,196
409
88
Hi,
updates:
I alos have upgraded my kernel from 5.4.0-122-generic to another, after almost five days , the issues again, freeze,
I DO NOT know what to do next
the issue might still be present in that newer kernel. The bug entry linked by @AxisNL is still open and it might be the issue you are having. Did you try to downgrade to 5.4.0-121-generic?
 

sam007

New Member
Jun 28, 2022
10
1
3
What CPU are you running on your host?
Intel(R) Celeron(R) N5105
Proxmox Virtual Environment 7.2-3
ubuntu 20.04 with kernel 5.15.0-43-generic after upgraded, before that is 5.4.0-122-generic
 

sam007

New Member
Jun 28, 2022
10
1
3
Hi,

the issue might still be present in that newer kernel. The bug entry linked by @AxisNL is still open and it might be the issue you are having. Did you try to downgrade to 5.4.0-121-generic?
oh , I have never thought to downgrade kernel
does it work ? if it is I would like to try it
 

fiona

Proxmox Staff Member
Staff member
Aug 1, 2019
2,196
409
88
oh , I have never thought to downgrade kernel
does it work ? if it is I would like to try it
If you have the same issue as @IlyaK (or another issue introduced with 5.4.0-122-generic) it should help:
I tried many versions of proxmox kernel and didn't succeed. But when i change ubuntu kernel from 5.4.0-122-generic to 5.4.0-121-generic, the problem was gone -i have two days without any freezes.
 

gyrex

Member
Jul 19, 2022
85
17
8
If you have the same issue as @IlyaK (or another issue introduced with 5.4.0-122-generic) it should help:
Do you know if this issue is specifically affecting the N5105 CPU? There's a bunch of us in the forums with this CPU who are experiencing kernel freezes and it seems like the common denominator is the N5105.
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get your own in 60 seconds.

Buy now!