VMA archive restore outside of Proxmox

[XrXca]

Again a bit late, but I had the same type of requirement. Proxmox backups on an alternate server, and needed a couple files from one of the backups.
A bit of duckduckgo and I discovered a python project that extract vma files: VMA Extractor (https://github.com/jancc/vma-extractor)

This fit the bill perfectly for me as I only needed python, and zstd on the backup storage server.
Then uncompress the backup, extract the backup using vma.py and then you have the raw images.

 

[bobi]

Available for Arch through AUR here: https://aur.archlinux.org/packages/vma-git

Maybe the compilation could be made more efficient for vma alone, but at least it’s reliably working.
Though Arch is the best, how to adapt this to other distros can be made out from build function there: https://aur.archlinux.org/cgit/aur.git/tree/PKGBUILD?h=vma-git

 

[Lonnie]

What purpose does VMA serve that isn't served by a raw image file? I seriously don't understand why VMA exists. I speculate that it exists exclusively to deter migration to Proxmox KVM alternatives? If that's the reason, do you see the irony? The only reason I'd ever consider migrating to an alternative would be proprietary tactics. I realize VMA is open source, but that doesn't mean it can't serve a "proprietary tactic".

My speculation is probably completely wrong. Yet, I still don't know the real reason that VMA exists. I'd appreciate someone disabusing my speculation.

@XrXca Thanks for posting that extractor. I bookmarked it; it might come in handy at some point.

A Positive Speculation: Maybe VMA exists to facilitate version control (so that you can do a live backup while the virtual machine is actively running ( on a file system that does not provide version control). Is that the reason?

 

[dcsapak]

What purpose does VMA serve that isn't served by a raw image file? I seriously don't understand why VMA exists. I speculate that it exists exclusively to deter migration to Proxmox KVM alternatives? If that's the reason, do you see the irony? The only reason I'd ever consider migrating to an alternative would be proprietary tactics. I realize VMA is open source, but that doesn't mean it can't serve a "proprietary tactic".

My speculation is probably completely wrong. Yet, I still don't know the real reason that VMA exists. I'd appreciate someone disabusing my speculation.

@XrXca Thanks for posting that extractor. I bookmarked it; it might come in handy at some point.

A Positive Speculation: Maybe VMA exists to facilitate version control (so that you can do a live backup while the virtual machine is actively running ( on a file system that does not provide version control). Is that the reason?

the reason why vma exist is of technical nature. when we make a qemu backup, we want to have snapshot semantics, but to have that for a running vm on storage that does not support snapshots, we must backup the blocks of the disk out-of-order. so to not having to seek around a big backup raw image (which can be costly depending on the storage, etc.) vma was invented with allows to store the blocks in any order. it also saves all disks into a single archive and saves the config,
for more details on how the format works, see https://git.proxmox.com/?p=pve-qemu...f5f9beafecc01a19c3a33a0eb54baf4acf9a0;hb=HEAD and https://git.proxmox.com/?p=pve-qemu...10fc76baaa19716aeb06259c2bcd196e4949c;hb=HEAD

 

[Lonnie]

@dcsapak Thank you putting my concerns to rest. I found this link especially informative. This also sheds light as to why VMAs are not as rsync-friendly as true raw image files are between off-site backups. I understand that PBS addresses this in ways that rsync cannot.

 

[bobi]

I realize VMA is open source, but that doesn't mean it can't serve a "proprietary tactic".

Being only distributed as a dpkg patch for upstream qemu and only compilable as a by-product of the Proxmox flavour of qemu, one could argue it just barely qualifies as open source.

It’s not a huge surprise then why nobody got it working outside of pve-qemu for almost nine years.

 

[t.lamprecht]

Being only distributed as a dpkg patch for upstream qemu and only compilable as a by-product of the Proxmox flavour of qemu, one could argue it just barely qualifies as open source.

What do you mean? It's fully open source and we accepted patches from contributors under the normal conditions for it.

It’s not a huge surprise then why nobody got it working outside of pve-qemu for almost nine years.

The vma CLI tool is trivial to build, at least is one has basic development experience. The format also has a clear and concise spec, so easy to rebuild too, there even exist third party implementations, e.g.: https://github.com/jancc/vma-extractor

The integration in QEMU is not that simple, but that has its reasons, mostly that we require all disks to have their state saved at the same time to avoid inconsitency, upstream always supported only one disk, Proxmox tried to upstream their solution way back in 2011 IIRC, the reason it didn't go in was that some devs thought that the existing NBD stack should/could (maybe) be used with less code, but that never materialized so we continue to maintain our own downstream integration and only report and fix bugs that it allows to find in QEMU to upstream.

 

[bobi]

What do you mean? It's fully open source and we accepted patches from contributors under the normal conditions for it.

I only mean that having access to vma actual sources (vma.h, vma.c & vma-reader.c, I reckon?) only through patching qemu in a dpkg step really does not seem the easiest way to access said sources.
But maybe I have just missed where the CLI tool sources are directly accessible.

Once you got that and from this on, I completely agree that the compilation is trivial, but a little bit overkill if you only want to compile vma alone and not the whole pve-qemu, unless you go beyond the trivial entrypoints and go through both the downstream and upstream build stack.

 

[oemb1905]

I'm new to proxmux but a veteran and qemu/kvm and virsh. For anyone needing to do this that's coming by this post, just run:

vzdump 101
vma extract vzdump-qemu-101-2021_07_23-14_17_40.vma /mnt/point/vma

Replace "101" with the vmid of your virtual machine, and replace the directory target with a name of a new directory of your choosing. This command will make a .raw image file and that can easily be brought into virsh, converted at will with qemu-img convert and so on. Not sure what all the fuss and muss is about in here. So long as you can get it from within PVE to raw easily, you have all you need to convert it to any target format imho. Hope this helps stop the next person new to PVE from having to read this whole forum to get a VM out of PVE.

Kindly,
oemb1905