How to mount disk/snapshot read only in other VM (or better approaches)?

[Asano]

I'm currently looking for a way to mount a snapshot (local ZFS storage) of a VM disk read only in another VM. My best find so far is this article about moving virtual disks: https://pve.proxmox.com/wiki/Moving_disk_image_from_one_KVM_machine_to_another . But this involves a lot of renaming, config file changes and I guess I would need to create a copy of the snapshotted state of the VM. Ideally I'd like to skip as much of this overhead as possible and I think it could be possible as the mounted disk/fs should be read only and would not conflict with the running original VM.

The purpose are offsite backups to B2 or similar on file level. If I trigger these from a separate VM and use read only snapshots from the disks which contain the backup source, I already have a consistent state of the file system and by using a separate VM, offsite backup management can be done in one location and not in each VM.
Maybe there is a better approach but I think mounting disk snapshots would do this task quite well.

Can this be done currently and if so what is the best way?

 

[wolfgang]

Hi,

with zfs you have 2 ways.
1.) zfs clone <pool>/<path>@<snapshot> <pool>[/<path]/<vmdisk>
Here you can create a thin clone what could also be written.

2.) Depend on the guest type.
KVM set snapdev to visible.
LXC set snapdir to visible.
If you set snapdir to visible, you get all the snapshots contend in you backup when you use vzdump.

 

[LnxBil]

If you set snapdir to visible, you get all the snapshots contend in you backup when you use vzdump.

Oh, I was not aware of that, but sure, it's 'tar'ed. The question is: Is tar intelligent enough to only store identical data once or will you have a Copy-on-Read replication?

 

[wolfgang]

AFIK tar is a stream packer so i think it is not smart enough.
And finished size was also indicate this.

 

[Asano]

So I think I'll go for the approach with zfs clone, however I'm still struggling a bit with automating it (see below).

Also it would be easier if I could access the VM snapshots on file level (from within their VM, another VM or the Proxmox host; it wouldn't matter) but I think this cannot be achieved easily. Also not with snapdir set to visible as this the mere volume and would have be mounted to get file level access. Or am I missing something here (if I do using the snapdir/snapshot directly would be better I guess).

With `zfs clone` I've made some tests and it looks like this is working as I need it. However when I try to automate things I got a little problem. I can automatically unlink a image from my backup VM by running `qm unlink 102 --idlist scsi1` for example but I haven't found the pendant to add a image on https://pve.proxmox.com/pve-docs/qm.1.html .
When I just edit the config file manually or with `sed -i` it looks like it would work however inside the VM the /dev/sdx is not showing up. When I do it simply via the Proxmox Web GUI the result in the config is the same but the newly attached disk is instantly accessible in the running VM via its /dev/sdx link. The Web GUI is no real option for an automated backup script so any ideas how I can do this from the command line?

Thanks for all the input so far.

 

[wolfgang]

@Asano

You do understand me wrong.
I was talking about zfs operation not VM operation.

zfs clone <image> <clone>
mount /dev/zvol/<clone>
do backup 
umount
zfs destroy <clone>

 

[Asano]

@Asano

You do understand me wrong.
I was talking about zfs operation not VM operation.

zfs clone <image> <clone>
mount /dev/zvol/<clone>
do backup
umount
zfs destroy <clone>

Hmm I'm not sure if I'm missing something but when you suggest doing this form the host, how should it work with file systems the host can't handle? I just tried this really quick for Ubuntu LVM images (which most of my VMs are) and this probably will work (I see the partitions etc) but for my Windows Servers (where I also have some VMs) I'm not even sure if there is a way to mount NTFS/ReFS partitions on Debian. So I thought just adding the clones to "backup VMs" (one for each FS type) is easier and I also get the bonus that my backup logic is capsuled in a VM and not on some host.

And as written before, adding the clone manually to a VM works well, however I haven't found a way to script it yet... If I really don't need one great, but currently I don't see how this should work.

(Btw, the reason I want an approach like this is the offsite backup is paid per data so I really only want to backup the actual files I need without OS overhead and also the ability to download only the exact files I need in case I need them as download is paid as well.)

 

[Asano]

I had some more time for this and found a solution for me.
`pvesh` can be used to script adding the clone (commands can be obtained by watching the PUT request headers of the web GUI for example on /api2/extjs/nodes/node/qemu/102/config).

So after I manually added the clone one time to the config of my VM which should take care of backups (102 in my example) I then use the following script on the host which can run regularly:

#!/bin/bash

myvmmount=$(qm agent 102 get-fsinfo)
if [[ $myvmmount = *"media/my-vm"* ]]; then
    echo "my-vm still mounted, backup cannot be initialized!"
else
    echo "my-vm init backup..."
    qm unlink 102 --idlist scsi1
    zfs destroy -R rpool/data/vm-118-disk-1@backup
    zfs snap rpool/data/vm-118-disk-1@backup
    zfs clone rpool/data/vm-118-disk-1@backup rpool/data/vm-102-disk-2-118
    pvesh set /nodes/mynode/qemu/102/config -scsi1 'local-zfs:vm-102-disk-2-118,backup=no,replicate=no'
fi

Now I can mount and backup specific files/folders inside the VM and umount afterwards. This also works for SQL Servers running on Windows Server (only without the mountcheck). I have such a script for each image where I want to take offsite backups from now and the clones just get added to scsi2, scsi3 and so on.

This is more complex then I would like it but as I just have few VMs still okay. However better approaches would still be welcome.

@wolfgang I'm still not sure about your comment, do you see an easier approach here? Maybe I missing something as I'm new to Proxmox as well as ZFS. But as the VMs OS isn't ZFS aware I wouldn't know how to easily make the snapshot available in it (or a separate VM) other than mounting a clone the way I'm doing it now. And on the host as said before I think it's not easily possible with images from Windows Servers to mount and access them..?