Recover data from proxmox VM

[Gianmarco Amici]

Hi all,
from previous IT guy, I inherited a server in a cloud company datacenter.
This server mount a proxmox distribution vith few VM inside it.
Datacenter, due to a problem, turn off the server and let me connect to the sistem using a "rescue" o.s., like a linux live cd.
What I need to do is to mount or be able to read VM disks to recover data inside them.
I succeded mounting proxmox root filesystem on /mnt (with etc, bin , lib...), but I have no idea on how to mount VM disks.
Below what I see with lsblk command

root@rescue:/mnt# lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sdb 8:16 0 1.8T 0 disk
├─sdb4 8:20 0 1.8T 0 part
│ └─md4 9:4 0 1.8T 0 raid1
│ └─pve-data 252:0 0 1.8T 0 lvm
├─sdb2 8:18 0 19.5G 0 part
│ └─md2 9:2 0 19.5G 0 raid1 /mnt
├─sdb3 8:19 0 1023M 0 part
└─sdb1 8:17 0 1004.5K 0 part
sda 8:0 0 1.8T 0 disk
├─sda4 8:4 0 1.8T 0 part
│ └─md4 9:4 0 1.8T 0 raid1
│ └─pve-data 252:0 0 1.8T 0 lvm
├─sda2 8:2 0 19.5G 0 part
│ └─md2 9:2 0 19.5G 0 raid1 /mnt
├─sda3 8:3 0 1023M 0 part
└─sda1 8:1 0 1004.5K 0 part


Do you have any advice on how to proceed?

Thank you.

 

[wolfgang]

Hi,

You have to activate the lv and then you can mount them.

 

[Gianmarco Amici]

Thank you Wolfgang,
I tried, but I'm not sure I'm doing it right.
I manage to mount pve-data.
Inside it there are few folders and an "images" one in which I found some vm-<imagename>-disk-1.raw
Tried to mount them using

qemu-nbd -c /dev/nbd0 /<path>/vm-100-disk-1.raw
(although I didn't know if it's the same to use /devnbd0 or should I use /dev/nbd1 or others)

It works, but mounting this device to a folder I receive

#mount /dev/nbd0 /mou
NTFS signature is missing.
Failed to mount '/dev/nbd0': Invalid argument
The device '/dev/nbd0' doesn't seem to have a valid NTFS.
Maybe the wrong device is used? Or the whole disk instead of a
partition (e.g. /dev/sda, not /dev/sda1)? Or the other way around?

Ok, so doing fdisk -l /dev/nbd0

Disk /dev/nbd0: 1000 GiB, 1073741824000 bytes, 2097152000 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 5F89A76E-BB9B-11E4-9EA2-02353F681C44

Device Start End Sectors Size Type
/dev/nbd0p1 128 4194431 4194304 2G FreeBSD swap
/dev/nbd0p2 4194432 2097151959 2092957528 998G FreeBSD ZFS

Then I tried

#/sbin/modprobe zfs
# mount -t zfs /dev/nbd0p1 /mou
filesystem '/dev/nbd0p1' cannot be mounted, unable to open the dataset
# mount -t zfs /dev/nbd0 /mou
filesystem '/dev/nbd0' cannot be mounted, unable to open the dataset
# mount -t zfs /dev/nbd0p2 /mou
filesystem '/dev/nbd0p2' cannot be mounted, unable to open the dataset

I'm missing something, but I don't know how to proceed.
Any help is really appreciated.

Thanks.

 

[wolfgang]

You need no zfs.
You must mount /dev/nbd0p2
The problem is you try to mount a disk not the partition of the disk.

 

[Gianmarco Amici]

Sorry Wolfgang, I didn't put in previos reploy, but I've tried it before

root@rescue:~# mount /dev/nbd0p2 /mou
mount: special device /dev/nbd0p2 does not exist


then I tried

kpartx -av /dev/nbd0
add map nbd0p1 (252:1): 0 4194304 linear /dev/nbd0 128
add map nbd0p2 (252:2): 0 2092957528 linear /dev/nbd0 4194432

and I saw it creates in /dev/mapper

# ls -l /dev/mapper/
total 0
crw------- 1 root root 10, 236 Jun 26 15:46 control
lrwxrwxrwx 1 root root 7 Jun 27 14:51 nbd0p1 -> ../dm-1
lrwxrwxrwx 1 root root 7 Jun 27 14:51 nbd0p2 -> ../dm-2
lrwxrwxrwx 1 root root 7 Jun 26 15:46 pve-data -> ../dm-0

but then again, trying to mount it

# mount /dev/mapper/nbd0p2 /mou
mount: unknown filesystem type 'zfs_member'

 

[wolfgang]

If this is a zfs you have to import it.

see man zpool
zpool import

 

[Gianmarco Amici]

Ok,
I solved.
I can't mount it because it was part of a pool of disks.
using
zpool import

it said
...
config:

all UNAVAIL missing device
nbd0 ONLINE

So I see there are other 2 raw disks and I did

qemu-nbd -c /dev/ndb1 /media/images/100/vm-100-disk-2.raw
qemu-nbd -c /dev/ndb2 /media/images/100/vm-100-disk-3.raw

and then

zpool import
...
...
config:

all ONLINE
nbd0 ONLINE
nbd2 ONLINE

so doing

zpool import -f all

it mounts all data needed.

Thanks.