Restoring a VM with some disks excluded from the backup changes the hardware configuration

[massi-na]

Dear all,

Envirorment:
Proxmox 8.2.4
VM backed up using PBS

Description:
Recently I had to change the way to backup a VM by excluding some disks, currently the VM configuration is the following:

#```
agent: 1
balloon: 6144
boot: order=scsi0;ide2
cores: 6
cpu: x86-64-v2-AES
ide2: none,media=cdrom
memory: 8192
meta: creation-qemu=9.0.0,ctime=1720086143
name: web1
net0: virtio=BC:24:11:AC:AB:37,bridge=vmbr1,firewall=1
numa: 0
onboot: 1
ostype: l26
protection: 1
scsi0: local-lvm:vm-202-disk-0,iothread=1,size=32G
scsi1: local-lvm:vm-202-disk-1,iothread=1,size=120G
scsi2: local-lvm:vm-202-disk-2,backup=0,iothread=1,size=325G
scsi3: local-lvm:vm-202-disk-3,backup=0,cache=writeback,iothread=1,size=1G
scsihw: virtio-scsi-single
smbios1: uuid=958ece15-d09b-47b8-b0fa-d40ac7b98d52
sockets: 1
startup: order=6
vmgenid: f72fa436-21ef-42ce-9c6e-7b73658e4b5e
#qmdump#map:scsi0:drive-scsi0:local-lvm:raw:
#qmdump#map:scsi1:drive-scsi1:local-lvm:raw:

Doing a restore test on a new VM I expected that it would create the 4 disks and restore data only for scsi0 and scsi1, instead the backup restore created only scsi0 and scsi1 and completely ignored the missing disk even though it was present in the hardware configuration:

  Logical volume "vm-902-disk-0" created.
new volume ID is 'local-lvm:vm-902-disk-0'
  Logical volume "vm-902-disk-1" created.
new volume ID is 'local-lvm:vm-902-disk-1'
restore proxmox backup image: /usr/bin/pbs-restore --repository xxxx@pbs@xxxxx:pbs vm/202/2024-08-20T14:11:09Z drive-scsi0.img.fidx /dev/pve/vm-902-disk-0 --verbose --format raw --skip-zero
connecting to repository 'xxxx@pbs@xxxxx:pbs'
open block backend for target '/dev/pve/vm-902-disk-0'
starting to restore snapshot 'vm/202/2024-08-20T14:11:09Z'
download and verify backup index
progress 1% (read 343932928 bytes, zeroes = 3% (12582912 bytes), duration 13 sec)
[...]
progress 100% (read 34359738368 bytes, zeroes = 33% (11479810048 bytes), duration 601 sec)
restore image complete (bytes=34359738368, duration=601.29s, speed=54.50MB/s)
restore proxmox backup image: /usr/bin/pbs-restore --repository xxxx@pbs@xxxxxx:pbs vm/202/2024-08-20T14:11:09Z drive-scsi1.img.fidx /dev/pve/vm-902-disk-1 --verbose --format raw --skip-zero
connecting to repository 'xxx@pbs@xxxxx:pbs'
open block backend for target '/dev/pve/vm-902-disk-1'
starting to restore snapshot 'vm/202/2024-08-20T14:11:09Z'
download and verify backup index
progress 1% (read 1291845632 bytes, zeroes = 9% (125829120 bytes), duration 56 sec)
[...]
progress 100% (read 128849018880 bytes, zeroes = 7% (9982443520 bytes), duration 5808 sec)
restore image complete (bytes=128849018880, duration=5808.80s, speed=21.15MB/s)
rescan volumes...
TASK OK

I did not find anything on internet searching about it, I would expect that Proxmox will not silenty change the hardware configuration and just create the two empty disks.

Any suggestion ? Maybe I should open a bug?

Thanks,
Regards

 

[gfngfn256]

Any suggestion ? Maybe I should open a bug?

I would expect this behavior exactly. You are doing a restore of a VM & not a creation. A restoration will restore the backup as is within that saved backup. You can always add disk/s later.

Within PVE what you really are looking for is VM Templates.

 

[massi-na]

I would expect this behavior exactly. You are doing a restore of a VM & not a creation. A restoration will restore the backup as is within that saved backup. You can always add disk/s later.

Within PVE what you really are looking for is VM Templates.

But VM hardware configuration is backed up, for that reason my expected behavior is that it get restored as is.

Managing a missing disk, but already created with the right size and options as per hardware configuration, can be done in the VM itself without the need to interact with the hypervisor.

Additionally, with containers it behave exactly how I was wondering, by just creating empty disks with the right size.

I'm not looking to VM templates, but preparing myself for a disaster recovery scenario that I already had to face.

Regards.

 

[gfngfn256]

disaster recovery scenario that I already had to face.

I believe you miss the point.

Lets start with that "disaster recovery". Usually the main disaster would be disk-written data-loss. But obviously in your case, where you chose not to backup the disk/s, you either have the data backed up somewhere else - or (unusual) it is easily re-creatable. So in the most likely scenario where your disks are backed up elsewhere, when the VM gets restored from backup - if PVE behaved the way you want it to, these disks would be IMMEDIATELY created upon restore, and would take the exact naming convention that your "old" backed-up one's had. Then you'd have to manipulate that afterwards when restoring the disks from your external backup. Secondly, you'd have to anyway add these disks within the VM configuration later if you chose to import them into the VM. In such a scenario the easiest method would be to remove the newly created disks & import the "old" ones - so the exact behavior the VM restore does now!
I for one have VMs that I don't backup their disks - and I definitely do NOT want the method you seek.

I'm not looking to VM templates

If you don't like/need VM templates, just create & configure your VM within PVE, never start the VM, & backup. Then when "disaster" strikes - simply restore.

BTW, I believe in your scenario, all you really need is the /etc/pve/qemu-server/{VMID}.conf file. So just back that up on your own & all is good.

 

[massi-na]

I believe you miss the point.

Lets start with that "disaster recovery". Usually the main disaster would be disk-written data-loss. But obviously in your case, where you chose not to backup the disk/s, you either have the data backed up somewhere else - or (unusual) it is easily re-creatable. So in the most likely scenario where your disks are backed up elsewhere, when the VM gets restored from backup - if PVE behaved the way you want it to, these disks would be IMMEDIATELY created upon restore, and would take the exact naming convention that your "old" backed-up one's had.

Yes, data is backedup in a different way, with different policies and so on. But it requires the disks to exists to be restored and they should exists exactly in the same way as they are written in the QEMU configuration file.

Then you'd have to manipulate that afterwards when restoring the disks from your external backup. Secondly, you'd have to anyway add these disks within the VM configuration later if you chose to import them into the VM. In such a scenario the easiest method would be to remove the newly created disks & import the "old" ones - so the exact behavior the VM restore does now!

If the disks exists I've only to work inside the VM for import the data from the backup, but it's not something that have to and need to interact with the Hypervisor.

I for one have VMs that I don't backup their disks - and I definitely do NOT want the method you seek.

That's exactly your point of view, but not mine.
There are several scenario usage and it's good to show all of the me to the developer, but doesn't mean that your usage scenario fit to all

If you don't like/need VM templates, just create & configure your VM within PVE, never start the VM, & backup. Then when "disaster" strikes - simply restore.

BTW, I believe in your scenario, all you really need is the /etc/pve/qemu-server/{VMID}.conf file. So just back that up on your own & all is good.

It's already included in the PBS, I'm just asking why it doesn't restore is as expected or rather: "It's wanted by design or it's a bug ?"

Then both behavior can coexists with a checkbox.

Regards

 

[gfngfn256]

There are several scenario usage and it's good to show all of the me to the developer, but doesn't mean that your usage scenario fit to all

I agree. But I still maintain that it is not a bug rather a niche usage. Seeing as I pointed out, you can simply have a restorable backup of the VM with empty drives, and you subsequently told us that this is in fact what you do, I think its questionable if such an option should be offered at all. Do remember that for other (maybe most) scenarios of the user not including the said disks within the PVE VM backup, the desired effect of the restore creating empty disks - could be detrimental to the recovery procedure.

Anyway, I think its great that you posted, as this helps all in learning PVE operational usage.

 

[sw-omit]

A thought, what about the scenario where the disk that is set to not be backed up is stored on a storage-medium that does not have thin provisioning (enabled) and uses more then half the total capacity of the disk?
In that scenario, if PBS were to (try to) create an empty disk on the same storage as the original disk, it wouldn't work, as it would try to write more data then would fit.

Other scenario; Backup is restored to a different system for whatever reason, but on that system the storage-location is not there. Trying to create the disk would fail, but leaving the disk in the config would also fail when you try to start the VM, as it can't find one of its disks.

For those scenarios above alone, not creating a disk and removing it from the config seems to me like a reasonable and safe scenario.
What I WOULD think might be a possible addition though, would be to have a checkbox during the restore with "create empty disks for not backed up disks", giving the user the option to keep the config, as well as the disks at the ready (and basically all it does it automatically add some empty disks after the restore is done based on the config)