How to recover cluster from backup after total loss?

[maxim.webster]

Dear all,

I am using Proxmox for a month now in a "homelab" context and so far everything is going well. I have two identical nodes in a cluster and a QDevice on an RPI. Backups of nodes (via proxmox-backup-client), VMs and containers are done utilizing PBS. A Hetzner Storagebox is used as a (remote) datastore. Backups are encrypted. However, due to the context (especially costs), PBS is not dedicated, instead, it is yet another HA VM running on the cluster.

Now, one thing that I didn't address so far is the recovery from backup in case of a failure or total loss of a node or the entire cluster. Let's call it "desaster recovery". How to do that properly, given the situation above? Is it anywhere documented?

Things that came into my mind, so far:

• before restoring any LXC or VM, I need to have a new datacenter up and running. Since PBS in my case is virtual, I cannot recover relevant datacenter configuration from it. So, I do need to store it (basically /etc/pve) aside, to recover is without being dependent on a running PBS.
• it looks like one could restore a VM using qmrestore from an archive on disk. But, looking at the Storagebox, I don't see no files, as PBS uses chunks to store data. So I stand no chance to get my hands on something to restore without a running PBS, with the datastore attached.
• so second task after recovering the datacenter is setting up a temporary PBS as a VM, reusing the existing datastore and the encryption key (that must be saved somewhere else). This temporary instance will then be used to restore the original PBS instance, to keep fingerprint, accounts, etc. consistent
• finally, even after having the PBS VM restored and access to the datastore, it looks like you still need a valid VM/LXC definition in PVE to restore it's content. PBS does not offer to restore anything, recovery can only be triggered from an existing VM/LXC machine in PVE. How to I get that definition from PBS?

The later for sure is documented somewhere, because the use case also adresses recovery of LXC and VMs from PBS that have been (accidentally) deleted in PVE.

Comments on my thoughts, corrections, add-ons and usefull links are highly appreciated.

Cheers

Maxim

 

[fabian]

finally, even after having the PBS VM restored and access to the datastore, it looks like you still need a valid VM/LXC definition in PVE to restore it's content. PBS does not offer to restore anything, recovery can only be triggered from an existing VM/LXC machine in PVE. How to I get that definition from PBS?

you can definitely restore into a new VMID for both containers and VMs, including recovering the backed-up guest config.. just setup PBS as storage in PVE, then browse that storage's contents and select restore..

 

[maxim.webster]

you can definitely restore into a new VMID for both containers and VMs, including recovering the backed-up guest config.. just setup PBS as storage in PVE, then browse that storage's contents and select restore..

You're absolutely right, I hadn't looked at it from that perspective. Thanks!