[TUTORIAL] Manually continue ZFS boot when dropped to initramfs

deviantintegral

New Member
Dec 22, 2023
4
3
3
When boot fails and you end up in initramfs, typically the recovery process involves booting off of a USB stick with the Proxmox installer so you can mount the system and fix things up.

I discovered that there's a way you can (often) get into a booted system without needing to go to recovery. In my case, I used this twice: once, when the /etc/kernel/cmdline file has the wrong root pool specified. Second, when I had two pools with the same name and manually needed to import one by number.

First, when you're at the (initramfs) prompt make sure you have zpool, zfs, and other tools available. If what's broken is due to missing zfs modules, or the initramfs not containing the zfs tools, you'll have to go all the way to using the Proxmox installer.

Let's say you need to import a specific pool by pool ID to boot:


Code:
# This will show all discovered pools including their Pool ID.
(initramfs) zpool import
# Import the specific pool, forcing it if it was last mounted on another system.
(initramfs) zpool import [-f] <pool id>
# If the pool is imported but not unlocking, or you need to do any other zfs commands, do them now!
(initramfs) zfs load-key rpool
# Find the process ID of your shell. If you're connecting over ssh, use the lowest process ID as that's the one attached to the console.
(initramfs) ps auxww | grep sh
# If all goes well, this should continue boot and you can fix things permanently in the comfort of your full environment.
(initramfs) kill -9 <pid>

I found this process worked better than trying to use the zfsunlock script when things were broken. Hopefully this helps someone else!
 
Thank you for this, it seems they have changed the zfs structure in Proxmox 8

They now added 'rpool/var-lib-vz' as a separate data set, which had caused 2 datasets I mount under images to block the boot process. Interesting that if a non critical mount fails the entire boot process is killed.

Dont know why continue or exit command just doesnt continue booting on initramfs. Your kill command is what got me recovered.
 
Last edited:
  • Like
Reactions: deviantintegral
When boot fails and you end up in initramfs, typically the recovery process involves booting off of a USB stick with the Proxmox installer so you can mount the system and fix things up.

I discovered that there's a way you can (often) get into a booted system without needing to go to recovery. In my case, I used this twice: once, when the /etc/kernel/cmdline file has the wrong root pool specified. Second, when I had two pools with the same name and manually needed to import one by number.

First, when you're at the (initramfs) prompt make sure you have zpool, zfs, and other tools available. If what's broken is due to missing zfs modules, or the initramfs not containing the zfs tools, you'll have to go all the way to using the Proxmox installer.

Let's say you need to import a specific pool by pool ID to boot:


Code:
# This will show all discovered pools including their Pool ID.
(initramfs) zpool import
# Import the specific pool, forcing it if it was last mounted on another system.
(initramfs) zpool import [-f] <pool id>
# If the pool is imported but not unlocking, or you need to do any other zfs commands, do them now!
(initramfs) zfs load-key rpool
# Find the process ID of your shell. If you're connecting over ssh, use the lowest process ID as that's the one attached to the console.
(initramfs) ps auxww | grep sh
# If all goes well, this should continue boot and you can fix things permanently in the comfort of your full environment.
(initramfs) kill -9 <pid>

I found this process worked better than trying to use the zfsunlock script when things were broken. Hopefully this helps someone else!
Great approach, saved me a lot of time, Thank you very much :)
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!