Best way to backup VMs

[Alessandro 123]

In our XenServer (cursed the day i choose to use that rubbish) node, we backup (sometimes) VMs directly exporting them from a snapshot. It's pretty useful because a restore can be mede directly, without resyncing a lot (millions) of small files.

In Google Cloud, we do the same by only using shapshots. No export at all. Snapshot are available even if the virtualmachine is destroyed and we can recover from a snapshot almost in 0 time.

On PVE, which is the best way to do the same, backing up from a remote server ? Currently we use BackupPC to backups files from the inside. I would like to add something extra and backup the same from the outside, so from Proxmox and not from the VM operating system. Also, only using backuppc means that in case of failure I have to reconfigure the VM manually, add disk, partitions, and so on, then restore everything. A huge amount of work.

Any faster, safer and reliable solution ?
I know that is not possible to export a VM via API call, it would be great, because i could call something: https://my.pve.node/backup?vmid=101 on my backup server and download an image ready to be restored.

 

[Stefan_R]

You can't download it via the API, but you sure can trigger a backup that way. See our VZDump backup documentation here.

 

[Alessandro 123]

I don't think I can use VZDump because it need a storage on pve node, and my backup server are not exposed to virtualization nodes. They are on a different natted subnet. Exposing an NFS share would be a mess.

 

[Stefan_R]

You can only create backups on storages that can be mounted on your PVE host, yes. You can build yourself a "downloadable backup" system pretty easily though: Trigger a backup to a local directory storage, then download it via 'scp' or what have you, and remove it afterwards to not fill up the disk.

 

[Alessandro 123]

You can only create backups on storages that can be mounted on your PVE host, yes. You can build yourself a "downloadable backup" system pretty easily though: Trigger a backup to a local directory storage, then download it via 'scp' or what have you, and remove it afterwards to not fill up the disk.

This means to have , at minimum, free space not less than the biggest VM i have to backup.
The one and only interesting feature of xenserver is backups via api with direct download. (yes, doing so on a running VM means creating snapshot and so having free space available not less than the VM i have to backup. Absoluty stupid thing to preallocata a snapshot with the same size of a VM, i have to snaptshot to hold 100MB on a 1TB machine, I need 1TB free)

 

[Stefan_R]

You can use the --stdout parameter of vzdump to avoid allocating a file on the source.

E.g.:

vzdump <vmid> --stdout 1 --compress zstd | ssh user@target 'cat > /path/to/backup.vma'

(or pipe it to wherever you need)

 

[Alessandro 123]

I have SSH access from backup to pve (not the opposite) , can I run the vzdump from the backup node and stream ?

something like :

ssh pvenode 'vzdump --stdout 1.........' > mybackup.vma"

I can run vzdump on a running VM and interrupt the process safely, to do some tests ? I have a very bed experience with XenServer with stale tasks, stale LVM snapshots and so on , so I prefere to ask



EDIT: i'm trying right now. seems to work when triggered from the backup node. This could be an interesting solution.

Some questions:
1) how can i check if the written image is correct ? Any qemu check or similiar ?
2) can I trigger some command *inside* the VM when running a backup ? IIRC, qemu-agent has a sort of trigger called when shapshotting....
3) backup server is ZFS. vzdump is the only solution or can I use ZFS to backup directly ?

 

[Stefan_R]

I have SSH access from backup to pve (not the opposite) , can I run the vzdump from the backup node and stream ?

something like :

ssh pvenode 'vzdump --stdout 1.........' > mybackup.vma"

I can run vzdump on a running VM and interrupt the process safely, to do some tests ? I have a very bed experience with XenServer with stale tasks, stale LVM snapshots and so on , so I prefere to ask

Exactly like that. And yes, you can interrupt the process with a simple ctrl-c, although if you do the transfer via SSH you probably have to delete the 'mybackup.vma' yourself afterwards.

For your questions:

1. The 'vma' binary has a 'verify' subcommand, but keep in mind that it is only available on PVE (no standalone package). Otherwise just restore the backup to a different VMID to see if it restores correctly (i.e. ssh pvenode 'qm create <new_vmid> --archive -' < mybackup.vma).
2. The guest agent does a "fs-freeze" and "fs-thaw" during the QEMU snapshot (not to be confused with a file-system snapshot), so the image is consistent. Other than that I don't think anything is triggered.
3. vzdump is the only solution that includes VM details (i.e. CPU options, disk configuration, networks, names, etc...). If you just need the raw data, you can of course just do a ZFS snapshot on the source and use zfs-send to back up to your target node.

 

[Alessandro 123]

Any suggestion on how to connect, with an unprivileged users, from backup server to pve and still run vzdump ? The only solution i have is to create an unprivileged user with password-less ssh, then using sudo to grand usage to "vzdump" to that user.

Any better way ?

 

[Stefan_R]

That's a reasonable solution, since the vzdump binary can only be run as root. On the API you can assign finer-grained permissions, however, you can not print to stdout then.

 

[Alessandro 123]

I've fixed by allowing vzdump and

qm list

, just the "list" because qm could be dangerous, to a backup user with password-less login from my backup server.

I'm backing up right now one test server in a totally scripted way.
Other solution to read the VMID and VM Names without using "qm" command ? Just to be as much as safe as possible

 

[Stefan_R]

Short of hardcoding, no, 'qm list' is the way to go.

 

[Alessandro 123]

Are you aware of some "scheduler" able to rotate backups ? Backing up remotely from command line with vzdump via ssh works perfectly, but I need something to rotate backups.

Should I create it on my own ? I could use logrotate but looks like a workaround.

 

[Stefan_R]

Our new Proxmox Backup Server (released as beta) has a prune feature working somewhat like logrotate, and the integrated backup can at least limit the amount of backups by deleting the oldest one if a certain count is reached. But doing it yourself like this, I'm not aware of any scheduler that could easily be used.

 

[Alessandro 123]

Backup Server is a standalone server or could be used easily on an existing server without installing tons of sofware ?

 

[Stefan_R]

See our announcement post and our documentation.