See contents of zfs-volumes of virtual machines

[Sir_Max]

Hi all,

I googled this but could not get it to work and currently I am not sure if this is even possible.

The setup is proxmox 5.1-41 with local zfs storage. I created a few virtual machines and now I want to find a possibility to access all files inside a zfs-volume (for example: "rpool/data/vm-100-disk-1") without the vm being booted. I want to be able to access the files in emergency cases and maybe recover specific files from the block-device from outside the vm.

I already tried to just mount /dev/zd64 (which is in my case "rpool/data/vm-100-disk-1") with "-t zfs" to a local folder but with no success. Another idea would be to use dd to write the contents of the block device into an image file. Does somebody know a better solution?

Thank you for your help

 

[morph027]

You might need to address the proper partition (as it's a block device)

~# fdisk -l /dev/zvol/rpool/non-critical-vm/vm-159-disk-1
Disk /dev/zvol/rpool/non-critical-vm/vm-159-disk-1: 128 GiB, 137438953472 bytes, 268435456 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 8192 bytes
I/O size (minimum/optimal): 8192 bytes / 8192 bytes
Disklabel type: dos
Disk identifier: 0xcd87dc13

Device                                          Boot     Start       End   Sectors  Size Id Type
/dev/zvol/rpool/non-critical-vm/vm-159-disk-1p1 *         2048 260046847 260044800  124G 83 Linux
/dev/zvol/rpool/non-critical-vm/vm-159-disk-1p2      260048894 268433407   8384514    4G  5 Extended
/dev/zvol/rpool/non-critical-vm/vm-159-disk-1p5      260048896 268433407   8384512    4G 82 Linux swap / Solaris

Partition 2 does not start on physical sector boundary.

~# mount -o ro /dev/zvol/rpool/non-critical-vm/vm-159-disk-1-part1 /mnt/loop/
~# ls /mnt/loop/
bin  boot  dev  etc  home  initrd.img  initrd.img.old  lib  lib64  lost+found  media  mnt  opt  proc  root  run  sbin  srv  sys  tmp  usr  var  vmlinuz  vmlinuz.old

 

[Sir_Max]

Thank you for your answer. Unfortunately this did not work:

mount -o ro /dev/zvol/rpool/data/vm-102-disk-1 /mnt/loop/

mount: wrong fs type, bad option, bad superblock on /dev/zd16,
       missing codepage or helper program, or other error

       In some cases useful info is found in syslog - try
       dmesg | tail or so.

Then I tried with fs type options:

mount -o ro -t ext4 /dev/zvol/rpool/data/vm-102-disk-1 /mnt/loop/

mount: wrong fs type, bad option, bad superblock on /dev/zd16,
       missing codepage or helper program, or other error

       In some cases useful info is found in syslog - try
       dmesg | tail or so.

mount -o ro -t zfs /dev/zvol/rpool/data/vm-102-disk-1 /mnt/loop/

filesystem '/dev/zd16' cannot be mounted, unable to open the dataset

The test-vm is formatted with ext4

 

[morph027]

Can you list the partitions with fdisk before?

 

[Sir_Max]

sure:

~# fdisk -l /dev/zvol/rpool/data/vm-102-disk-1
Disk /dev/zvol/rpool/data/vm-102-disk-1: 30 GiB, 32212254720 bytes, 62914560 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 8192 bytes
I/O size (minimum/optimal): 8192 bytes / 8192 bytes
Disklabel type: dos
Disk identifier: 0x10d72aca

Device                               Boot    Start      End  Sectors  Size Id Type
/dev/zvol/rpool/data/vm-102-disk-1p1          2048 15624191 15622144  7.5G 82 Linux swap / Solaris
/dev/zvol/rpool/data/vm-102-disk-1p2 *    15624192 62912511 47288320 22.6G 83 Linux

 

[morph027]

Hm ... looks like there is no partition yet. Is there an OS installed already?

 

[Sir_Max]

sry, I forgot to paste the last part. I edited the post before

 

[morph027]

mount -o ro /dev/zvol/rpool/data/vm-102-disk-1-part2 /mnt/loop/

 

[Sir_Max]

~# mount -o ro /dev/zvol/rpool/data/vm-102-disk-1-part2 /mnt/loop/
mount: special device /dev/zvol/rpool/data/vm-102-disk-1-part2 does not exist

 

[morph027]

Hmpf...

ls -la /dev/zvol/rpool/data/vm-102-disk-1*

 

[Sir_Max]

I finally got it! Here is what worked:

vm image mount:
## show vm disks
ls -d -1 /dev/zvol/rpool/data/*
or just
ls -laR /dev/zvol/*

## create loop device for disk
losetup -f {path to vm disk}

## read out partitions
partprobe /dev/loop{N} <- for example "loop0". Just check the timestamp to see which one is the newest loop-device with "ls -la /dev/loop*"

## mount
mount /dev/loop{N} /local/directory

## ls /local/directory/

that's it!

# vm image unmount:
## unmount
umount /local/directory

## loop-device detach
losetup -D /dev/loop{N}

## loop-device delete
rm /dev/loop{N}* <- I attached the wildcard (*) here because it creates more loop-devices for each partition found with partprobe, for example "/dev/loop0p1", "/dev/loop0p2" ...

 

[rodrigoaguilera]

Just in case someone lands here just like I did.

On proxmox 8 seems like is not necessary to setup loop devices since the partitions appear exposed as block devices.

For example an ext4 partition
mount -o ro -t ext4 /dev/zvol/local-zfs/vm-102-disk-1-part1 /mnt/aux/

 

[RolandK]

you don't want to use loop devices for mapping partitions, you want device mapper / kpartx for that ( https://manpages.ubuntu.com/manpages/xenial/man8/kpartx.8.html )