Booting server on Live OS to retrieve file from VM

[milkmonster]

Hello

I have been the happy user of Proxmox for a year or so since swaping out XCP-NG. I have made one gigantic blunder and lost my access to the server. I stored my KeepassXC file on a vm while changing OS on my daily driver. I had a USB with Pika backup of the file, so guessed I was safe. The VM went down, and Pika was encrypting the backup but never prompted the password. My Proxmox restarted and the VM was not enable to start on start up, now my password file is on a shutdown VM and I can't get into the server! I know, completely stupid amateur.

My setup is two SSD in a ZFS Raid0. I have tried to boot to a ubuntu Live OS:

"zpool import" shows rpool

"zpool import -f rpool" imports correctly and shows ONLINE.

"zfs list" gives me a nice overview

rpool/ROOT w/ mount point
rpool/DATA w/ mount point
vm-100-disk-0 (250G, block device for VM disk)
vm-101-disk-0 (500G, block device for VM disk)
vm-102-disk-0 (100G, block device for VM disk)

It's vm-100-disk-0 that's the important one!

I then tried to mount the VM 100 disk with "kpartx -av /dev/zvol/rpool/data/vm-100-disk-0" but the command just hangs at least for 30 minutes.

What am I missing...? I have spend countless hours on YT and forums

 

[gfngfn256]

kpartx -av /dev/zvol/rpool/data/vm-100-disk-0

First thing I would try is:

kpartx -av /dev/zvol/rpool/DATA/vm-100-disk-0

since as far as I know, ZFS paths are by default case-sensitive & your zfs list shows rpool/DATA

Please note I don't use zfs.

 

[milkmonster]

First thing I would try is:

kpartx -av /dev/zvol/rpool/DATA/vm-100-disk-0

since as far as I know, ZFS paths are by default case-sensitive & your zfs list shows rpool/DATA

Please note I don't use zfs.

Ah, sry! I tab my way through in the actually terminal, so the path is ok.

 

[Kingneutron]

Will be watching this thread with interest, but I would recommend making a "portable PVE" on bootable SD card / external SSD or the like so you have a full Proxmox temporary environment with LVM+ZFS for recovery

You should also be backing up all VM/LXC regular, Proxmox Backup Server on separate PC/laptop is pretty much ideal for this

 

[gfngfn256]

What does this show:

fdisk -l /dev/zvol/rpool/DATA/vm-100-disk-0

 

[gfngfn256]

but I would recommend making a "portable PVE" on bootable SD card / external SSD or the like so you have a full Proxmox temporary environment with LVM+ZFS for recovery

I very much like that idea, better than the Proxmox recovery mode which AFAIK is just the bootloader & then CLI. Would probably need updating periodically.

Maybe someone can build a script/system/ansible to produce an image file based on the current PVE of the user. Would probably need a lot of tweaking to make it as non-intrusive/damaging as possible, but yet be a great management recovery resource for the current system.

Enough of my expectations ranting .....

 

[milkmonster]

Hi guys and girls

I'm still in a lot of trouble. I have found some failure in importing the rpool. I have taken screenshots to document the process.

Fresh from booting up:


I find the zpool:


Importing the rpool gives an error:


I "crtl+c" and the rpool seems imported. But I suspect this is where I have the problem. I have searched forums and found this: https://www.reddit.com/r/zfs/comments/1g49ga7/importing_pool_kills_the_system/. I think it's related, but I nowhere near understanding the issue or fix.

So after crtl+c I just tried, but as expected it didn't work even though zfs shows:


kpartx just hangs and then i exit ctrl+c.

fdisk on the important vm shows like this:




What to do?

 

[gfngfn256]

As I've said above I don't even dabble in ZFS, but the suggestion of the linked post you posted, seems to suggest that the auto mounting of the Proxmox root fs is the culprit you are suffering from.

So maybe this will help:

zfs set canmount=noauto rpool/ROOT

See here what this option does.