ZFS Pool issues and Ubuntu VM hangs on boot

Micle

New Member
Apr 25, 2023
1
0
1
Virtual Environment 7.2-3

Hello,
I have a home server running Proxmox without any major issues for the past 1-2 years or so. I have two 4TB HDDs in a mirror raid using ZFS for a total of ~4TB usable storage and this is where I keep all of my VMs, ISOs etc. Proxmox itself is installed on a separate SSD.
Since I'm not sure what exactly my problem is, here is what I did and what I have happening now:
Yesterday, my server has been up for ~56 days, running fine. I decided to try adding more RAM to one of my VMs so I shut the VM down, configure the new RAM amount and click to reboot it, this is where I noticed that the VM was not starting and I wasn't able to see the console. I tried accessing the proxmox shell and I noticed a weird issue where I wasn't able to enter any commands, it would just type out the characters and enter would move onto the next line, I tried ssh'ing into proxmox but that didn't work either so I then tried rebooting it. It was taking very long to reboot so I went to the physical machine and rebooted it manually.
After the reboot I saw an error message along the lines of Failed to start: Import ZFS pool pool1, it proceeded to the login prompt, I log in and I am greeted with the same issue as I had in the Web GUI where I can't actually run any commands. After many reboots and fix attempts using the forums and similar issues and using recovery mode I was able to fix that issue by running dpkg-reconfigure tzdata, selecting none and then UTC, apparently the issue was caused by daylight savings hours... I have no idea why but it finally let me run commands in proxmox again.
Currently I am able to boot into Proxmox, access the Web GUI and my PfSense VM is working. Along the way I believe I also had to fix pve-cluster not starting up and I had to disable my problematic VM from auto-starting on boot as it seemed to be causing Proxmox to hang. Regarding the Failed to start: Import ZFS pool pool1 error on boot, I have tried a few fixes regarding the zfs-import-cache system service, I tried making it run modprobe
and sleep 10 before running and after that I started getting Failed to start: Import ZFS pool from cache after which I tried creating a new cache file etc. which did end up with the a boot without any errors but after logging in Proxmox crashed. Currently I have zfs-import-cache disabled but all that does is occasionally show me Failed to start: Import ZFS pool pool1 already exists.

Without writing more paragraphs, here are some more things to note:
- I get General protection fault what seems like every boot. I have read that this is a bug and not an issue.
- Running zpool status shows my pool as DEGRADED and it shows that there is a scrub in progress (although the scrub seems to get stuck at some point and never progress, even after leaving it for 12h+), I don't think my HDDs are faulty because they are still relatively new and SMART doesn't report any issues.
- I can start my faulty VM but it hangs on boot, I can't restart it, only way for me to restart it is by running qm stop 102 and qm showcmd 102 | bash in the shell. I have also set the RAM to the original amount, still not working. Also tried booting using the ISO but same story.
- I tried cloning my Ubuntu VM template which is how I made my currently broken VM to test but the cloning is even now still ongoing so that doesn't seem to be working.

My hope is to get my VMs up and running normally again without issues, I do not have any backups of Proxmox or my faulty VM unforunately. I am really lost right now and any help would be greatly appreciated.
(Please let me know if I should post any more info/command outputs)
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!