Proxmox Backup Strategies > Better than Acronis...

[paxmobile]

I worked with computers for more than 30 years and with Linux since the first Covid wave in 2019. Since my first Proxmox installation (v.6.1) I've solved the backup issue turning off the server monthly, booting from an USB pendrive with Acronis True Image 2020 and cloning the main SSD on another 1TB USB SSD. It's proven: the restore works everytime. Is it elegant ? Noway... Can't wait for a future where maybe ProxMox VE 9 integrates that feature, I mean for my uptime needs Acronis could be ok but productive environments require higher availability.
I tried to run Clonezilla on both Proxmox servers i run for testing and then connect in master/slave mode but ended asking myself what I was doing, anyway the remote connection only offered me to backup a 12Gb Quemu that cannot be sda cuase of its size.
Clonezilla on Proxmox
I'm asking the *nux experts that roams this forum: is there any Debian app that can make the cloning job at least without involving the bootable USB drive ? Could this be automated, I mean including the shutdown and reboot of the system ?
I'm asking out of curiosity since I've very limited theoretical and Linux cli knowledge.

 

[ShaneJ]

You wont get the replication feature you are looking for but you could perhaps try Veeam agent for Linux.

 

[esi_y]

is there any Debian app that can make the cloning job at least without involving the bootable USB drive ? Could this be automated, I mean including the shutdown and reboot of the system ?
I'm asking out of curiosity since I've very limited theoretical and Linux cli knowledge.

I think this is the wrong approach, i.e. "cloning" anything. If I wanted to be cheeky, the reason I never need Acronis is that e.g. there's dd [1]. Does it make sense here? No.

This has been discussed all over the forum for a while now, e.g.:
https://forum.proxmox.com/threads/official-way-to-backup-proxmox-ve-itself.126469/

As I have mentioned there, the nice thing with Linux is you can just copy the files. You can then even put them on a different system, different partitioning, as long as mountpoints are the same (or you have to adapt them). You just need a bootloader and good to go. Disk whatever size.

For me, the only valuable part (besides VM images, which I suppose you are already backing up) is configuration, that's literally few MBs at most:
https://forum.proxmox.com/threads/backup-cluster-config-pmxcfs-etc-pve.154569/#post-704816

But you should read the other thread(s) here on the forum where people point out what else they like to keep. I personally feel most of what you would be backing up is basically content of ISO install, so what's the point of backing it up? By the time you want to recover, you want fresh/updated install anyways.

I do not remember when was the last time I was doing DD on drives to copy them, it just makes no sense, unless in some recovery jobs.

[1] https://manpages.debian.org/bookworm/coreutils/dd.1.en.html

 

[esi_y]

You wont get the replication feature you are looking for but you could perhaps try Veeam agent for Linux.

NB Sorry, nothing personal, but for me, "agents" like these are solutions looking for a problem that does not exist. It is there, IMO, for people migrating from existing backups or Windows guests, etc.

Reinventing wheels whilst failing at that:
https://forum.proxmox.com/threads/veeam-silent-data-corruption.155212/

 

[ShaneJ]

NB Sorry, nothing personal, but for me, "agents" like these are solutions looking for a problem that does not exist. It is there, IMO, for people migrating from existing backups or Windows guests, etc.

Reinventing wheels whilst failing at that:
https://forum.proxmox.com/threads/veeam-silent-data-corruption.155212/

The agent suggestion was for the OP who was asking for an app.

The link you reference to in your post is for VBR, not for the agent!

 

[esi_y]

The agent suggestion was for the OP who was asking for an app.

I am not saying you should not be posting it, I just want to react on it - that does not exclude each other.

The link you reference to in your post is for VBR, not for the agent!

I know, but you basically start installing extras on your host which you would be better off without. Do I really need extra kernel module and all that when I have filesystem that can do snapshots, etc.? Also, it would capture e.g. that config.db mid-write while doing backup, that's SQLite database in write-ahead-log mode...

 

[paxmobile]

Well with a decent Proxmox setup I totally agree to your points, would be a shame to do backups phisically working on a 19" rack with USB pendrives and USB hard disks, but...
Proxmox All Flash setup
I got into Proxmox cause I needed to run Meshcentral as a remote desktop solution to teach some of my clients. Ended up selling my first Proxmox Server with preconfigured domain with SSL and Meshcentral. As Meshcentral it's so lightweight I never felt necessity to buy a stronger machine nor to sell expensive server for that purpose.
Of course I got fascinated by all the other VMs and LXCs backends, so I added a 16GB module and installed a dozen of them.
My Proxmox app list

 

[esi_y]

Well with a decent Proxmox setup I totally agree to your points, would be a shame to do backups phisically working on a 19" rack with USB pendrives and USB hard disks, but...

I am not sure if we are on the same page, it does not really matter where you are dumping your backups, for all I know I can even e.g. have machine reboot somewhere, boot of PXE, auto-restore from NFS and good to go. It is more like the form in which you are doing this. It is e.g. much easier to simply reinstall (there's auto-intaller) completely unattended and then Ansible configures the rest. Uptime lost is very little, there's no skeletons in the wardrobe. Now of course you do not need to be using any of that, you can totally have your auto-installer with scripts on your flashdrive sticking out of your magic box.

I mean for my uptime needs Acronis could be ok but productive environments require higher availability.

Circling back to this - in a cluster, you really do not have downtime, you just kill that node, install new one, join the cluster, let some VMs move over to it. Job done. Config is cluster-wide, auto-distributed on join. Of course it's not a backup, for that though, see above.

 

[paxmobile]

I am not sure if we are on the same page, it does not really matter where you are dumping your backups, for all I know I can even e.g. have machine reboot somewhere, boot of PXE, auto-restore from NFS and good to go. It is more like the form in which you are doing this. It is e.g. much easier to simply reinstall (there's auto-intaller) completely unattended and then Ansible configures the rest. Uptime lost is very little, there's no skeletons in the wardrobe. Now of course you do not need to be using any of that, you can totally have your auto-installer with scripts on your flashdrive sticking out of your magic box.



Circling back to this - in a cluster, you really do not have downtime, you just kill that node, install new one, join the cluster, let some VMs move over to it. Job done. Config is cluster-wide, auto-distributed on join. Of course it's not a backup, for that though, see above.

So you mean an Ansible setup makes a backup of the Proxmox node configuration so I can restore it easily after a new Proxmox installation, and than copy the VMs and LXCs ?

 

[esi_y]

So you mean an Ansible setup makes a backup of the Proxmox node configuration so I can restore it easily after a new Proxmox installation, and than copy the VMs and LXCs ?

I meant it more like other way around, I do not need to be flushing out "clones", I only need to add my configs onto otherwise stock install. The auto-installer + Ansible can get me the rapid recover, it gets me current packages and everything in little time, then put in custom networking, etc. The VM configs are basically just the backed up database. How often you want to be backing it up is up to you. This all of course assumes you are backing up your VM disk images (not what you were asking about in the OP, so leaving it out of scope here).

 

[LnxBil]

The infrastructure-as-code setup like @esi_y describes is very good. It's not what most people have in mind when it comes to backups, because they're used to what has been done for decades, yet it is the most effective one and scales beautifully.

If you're using PBS, you're also good to go. If you want to store your OS disk and /etc/pve, just do it with PBS and have file restore over PBS. That's how we do it. Also, having a cluster and loosing one node is also very easy to restore, just reinstall the node with a different name, add/join it to the cluster and it'll work.

There is also ReaR, which works quite well with non-ZFS system in order to have a bootable restore medium for you PVE, yet I don't use it personally.

 

[esi_y]

just reinstall the node with a different name

This is still a problem, btw? Genuinely asking, I thought the SSH bugfix did away with it.

There is also ReaR, which works quite well with non-ZFS system in order to have a bootable restore medium for you PVE, yet I don't use it personally.

+1 But I would want to test if it recovers the config.db well. But that could be offset by SQlite dump prior to running it.

 

[LnxBil]

This is still a problem, btw? Genuinely asking, I thought the SSH bugfix did away with it.

That's a good question. I haven't tried in a long time.

 

[paxmobile]

I meant it more like other way around, I do not need to be flushing out "clones", I only need to add my configs onto otherwise stock install. The auto-installer + Ansible can get me the rapid recover, it gets me current packages and everything in little time, then put in custom networking, etc. The VM configs are basically just the backed up database. How often you want to be backing it up is up to you. This all of course assumes you are backing up your VM disk images (not what you were asking about in the OP, so leaving it out of scope here).

I've watched some Youtube videos introducing the Ansible priciples and altrough I'm thankful for the advice, I'll focus on more simple Proxmox tasks first. At the end I will dive into the Ansible magic but I feel I've to learn other basics first.

 

[esi_y]

I've watched some Youtube videos introducing the Ansible priciples and altrough I'm thankful for the advice, I'll focus on more simple Proxmox tasks first. At the end I will dive into the Ansible magic but I feel I've to learn other basics first.

I did not mean to make it sound complicated, also not sure how in depth you researched it, at the face of it, classic Ansible is just defining what SSH connection scripting should do for you. Ansible does not even need installing on the target nodes. If it feels somehow more natural, you may - with equal efficiency - achieve the same with writing yourself a post-install shell script that you remote launch via SSH. Ansible should not feel like magic, really. Just wanted to mention that.