[SOLVED] Proxmox unresponsive, VM's inaccessible

L

LooneyTunes

Active Member
Jun 1, 2019
203
16
38
Hi,

just found my installation unresponsive and all my VM's were inaccessible. I run the latest version, 6.4.9.

With unresponsive I mean impossible to reach over network, it does not have any console, nor possible as is now to attach one. I had to power cycle it unfortunately.

I would need some guidance on where to start looking for the cause of the issue. I have made a copy of the /log directory just in case.

From syslog:
"
Jul 3 22:17:00 pve systemd[1]: Starting Proxmox VE replication runner...
Jul 3 22:17:01 pve systemd[1]: pvesr.service: Succeeded.
Jul 3 22:17:01 pve systemd[1]: Started Proxmox VE replication runner.
Jul 3 22:17:01 pve CRON[26072]: (root) CMD ( cd / && run-parts --report /etc/cron.hourly)
Jul 4 08:17:26 pve systemd[1]: Started Create list of required static device nodes for the current kernel.
Jul 4 08:17:26 pve systemd[1]: Starting Flush Journal to Persistent Storage...
"

According to other logs, it seems system went offline/unresponsive at the time of last line from July 3.

It looks like a backup job was initiated (or tried at least)? How can I check what that Cron job is actually doing? I have certainly not issued it.

Thanks
 
Last edited:
For the last week or so my server hosted at OVH has been doing the same thing, just stops working and I didn't even think about backups running. I thought it was a hardware issue, so I got a new server, moved everything over to have it just happen again. I have to hard boot it from the console to bring it back up.
 
JayMan6014 said:
For the last week or so my server hosted at OVH has been doing the same thing, just stops working and I didn't even think about backups running. I thought it was a hardware issue, so I got a new server, moved everything over to have it just happen again. I have to hard boot it from the console to bring it back up.
Click to expand...
I'm sorry to hear that. That was exactly what I had to do as well... Hope they will identify the issue soon
 
Well, checked my settings and the backup job I have set up is set to start at midnight. My "crash" happened hours before that, but not ruling anything out just yet
 
I'm not sure if this is related, but my server has been crashing after just about 2.5 hours of runtime. It seems that RAM utilization on the host slowly rises until the web GUI breaks, then the pmxcfs processes slowly start eating up more and more CPU until my server is at 100% CPU utilization and RAM/swap both fill up to 100%. I have to reset the server through IPMI after that.

Again, not sure if it's a related issue though.
 
Last edited:
Yeah I think I've solved my issue after a week of struggling... Turns out I had a Graphite metrics server configured, but Proxmox wasn't playing nicely for some reason and was spewing out hundreds of error messages per second. Since removing the metrics server, suddenly Proxmox is perfectly stable. Pretty ridiculous problem, but I'm glad it seems to be fixed.
 
I would try examining the logs at the time just before the restart, if they may contain any clues... on linux they're at /var/logs
 
I looked in messages, syslog, debug etc.. They all just show it restarting at 7:54am but nothing before that I can see. Even looked in the .log.1 files.
 
Then I don't know. I would hope someone from Proxmox Staff would take a minute to comment on our issues...
 
Oh, that's much worse than what I experience. I have had one crash so far, hopefully no more... But would love to know the cause. Server has been stable for about a year before this... If it would panic for some reason, wouldn't that have ended up as last entry in the log? And some other clues been recorded?
 
Just upgraded to v7.0-8. Hope this is the last of this issue, checking out for now. Hope you guys find your answers!
 
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom
Back