Restore VM encrypted doesn't work ?

[derf]

Hi,

I encountered a problem with the restoration of encrypted VM...
- HOST/CT/VM encrypted backup is OK.
- HOST and CT encrypted restore is OK.
- but VM encrypted restore doesn't work.

PVE restore task reply:

Error: wrong signature in manifest
TASK ERROR: command '/usr/bin/proxmox-backup-client restore '--crypt-mode=encrypt' '--keyfd=13' vm/901/2020-07-15T07:54:33Z index.json /var/tmp/vzdumptmp6695/index.json --repository backup@pbs@192.168.254.99:ds1' failed: exit code 255

In CLI, it's OK (without specify --key-file /etc/pve/priv/storage/pbs.enc):

~# proxmox-backup-client restore vm/901/2020-07-15T07:54:33Z index.json - --repository backup@pbs@192.168.254.99:ds1
Password for "backup@pbs": ******
fingerprint: <redacted>
Are you sure you want to continue connecting? (y/n): y
fingerprint: <redacted>
Are you sure you want to continue connecting? (y/n): y
{
"backup-id": "901",
"backup-time": 1594799673,
"backup-type": "vm",
"files": [
{
"crypt-mode": "encrypt",
"csum": "f231b88a6e289193e1066b2448213b57704071dbf850c36b843b0cbab04a8d37",
"filename": "qemu-server.conf.blob",
"size": 287
},
{
"crypt-mode": "encrypt",
"csum": "5be112c226676d0b69be6ec4c70a04893e790c800a47f946ff59f1ed63e932dd",
"filename": "drive-scsi0.img.fidx",
"size": 1073741824
}
],
"signature": "fa70ada139048fe76ee5c8986783c42d0c3640087f70a9043072b1f35edeb9c1",
"unprotected": {}
}

PBS and PVE are up to date.
PBS:
proxmox-backup-client/stable,now 0.8.6-1 amd64 [installed]
proxmox-backup-docs/stable,now 0.8.6-1 all [installed]
proxmox-backup-server/stable,now 0.8.6-1 amd64 [installed]
proxmox-backup/stable,now 1.0-3 all [installed]

PVE:
libproxmox-backup-qemu0/stable,now 0.6.1-1 amd64 [installed]
proxmox-backup-client/stable,now 0.8.7-2 amd64 [installed,automatic]
proxmox-ve: 6.2-1 (running kernel: 5.4.44-2-pve)
pve-manager: 6.2-10 (running version: 6.2-10/a20769ed)

 

[ntimo]

I can confirm this does not work, I get the same error when trying to restore a encrypted virtual machine.

 

[fabian]

can confirm as well - will be investigated!

 

[derf]

Added in Bugtracker : Bugzilla 2857

Regards

 

[wasteground]

I see a different error while trying to restore encrypted virtual machines, but I'm not clear if it's related to this or not (I seem to be running version of code which should have this specific bug resolved, hence I'm not sure if it's the same, related, or entirely unrelated)

new volume ID is 'local-zfs-ssd:vm-666-disk-0'
restore proxmox backup image: /usr/bin/pbs-restore --repository user@pbs@pbs.host.here:ds vm/113/2020-07-30T10:06:18Z drive-virtio0.img.fidx /dev/zvol/rpool-ssd/vm-666-disk-0 --verbose --format raw --skip-zero
connecting to repository 'user@pbs@pbs.host.here:ds'
open block backend for target '/dev/zvol/rpool-ssd/vm-666-disk-0'
starting to restore snapshot 'vm/113/2020-07-30T10:06:18Z'
download and verify backup index
restore failed: unable to decrypt blob - missing CryptConfig
temporary volume 'local-zfs-ssd:vm-666-disk-0' sucessfuly removed
TASK ERROR: command '/usr/bin/pbs-restore --repository user@pbs@pbs.host.here:ds vm/113/2020-07-30T10:06:18Z drive-virtio0.img.fidx /dev/zvol/rpool-ssd/vm-666-disk-0 --verbose --format raw --skip-zero' failed: exit code 255

 

[fabian]

@wasteground could you include the output of 'pveversion -v'? the fixed qemu-server package (6.2-11) is only available up to pve-enterprise..

 

[wasteground]

Ahh, that would be the issue - I am running pve-enterprise, 6.2-10 is the latest there I guess. I think I misunderstood which version has the fix. No problem, I can wait for 6.2-11 to be bumped to pve-enterprise

proxmox-ve: 6.2-1 (running kernel: 5.4.44-2-pve)
pve-manager: 6.2-10 (running version: 6.2-10/a20769ed)
pve-kernel-5.4: 6.2-4
pve-kernel-helper: 6.2-4
pve-kernel-5.3: 6.1-6
pve-kernel-5.0: 6.0-11
pve-kernel-5.4.44-2-pve: 5.4.44-2
pve-kernel-5.4.44-1-pve: 5.4.44-1
pve-kernel-5.4.41-1-pve: 5.4.41-1
pve-kernel-5.4.34-1-pve: 5.4.34-2
pve-kernel-5.3.18-3-pve: 5.3.18-3
pve-kernel-5.3.18-2-pve: 5.3.18-2
pve-kernel-5.3.13-3-pve: 5.3.13-3
pve-kernel-5.3.13-1-pve: 5.3.13-1
pve-kernel-5.3.10-1-pve: 5.3.10-1
pve-kernel-5.0.21-5-pve: 5.0.21-10
pve-kernel-5.0.21-3-pve: 5.0.21-7
pve-kernel-5.0.15-1-pve: 5.0.15-1
ceph-fuse: 12.2.11+dfsg1-2.1+b1
corosync: 3.0.4-pve1
criu: 3.11-3
glusterfs-client: 5.5-3
ifupdown: 0.8.35+pve1
ksm-control-daemon: 1.3-1
libjs-extjs: 6.0.1-10
libknet1: 1.16-pve1
libproxmox-acme-perl: 1.0.4
libpve-access-control: 6.1-2
libpve-apiclient-perl: 3.0-3
libpve-common-perl: 6.1-5
libpve-guest-common-perl: 3.1-1
libpve-http-server-perl: 3.0-6
libpve-storage-perl: 6.2-5
libqb0: 1.0.5-1
libspice-server1: 0.14.2-4~pve6+1
lvm2: 2.03.02-pve4
lxc-pve: 4.0.2-1
lxcfs: 4.0.3-pve3
novnc-pve: 1.1.0-1
proxmox-mini-journalreader: 1.1-1
proxmox-widget-toolkit: 2.2-9
pve-cluster: 6.1-8
pve-container: 3.1-11
pve-docs: 6.2-5
pve-edk2-firmware: 2.20200531-1
pve-firewall: 4.1-2
pve-firmware: 3.1-1
pve-ha-manager: 3.0-9
pve-i18n: 2.1-3
pve-qemu-kvm: 5.0.0-11
pve-xtermjs: 4.3.0-1
qemu-server: 6.2-10
smartmontools: 7.1-pve2
spiceterm: 3.1-1
vncterm: 1.6-1
zfsutils-linux: 0.8.4-pve1

 

[JamesT]

@fabian Just wondering if this is fixed yet? I'm getting the same error.
My scenario: PVE and PBS on host1 (TOULA). PVE and PBS on host2 (both hosts standalone).
I added host1 PBS datastore as "Proxmox Backup Server" storage in PVE on host1.
host1 PVE backs up vm 100 to this datastore.
I have setup host1 as a remote on host2, and synced the datastore from host to host2.
I've added the datastore from host2 PBS into host2 PVE and can see the backups.
I'm then attempting to do a vm restore of vm 100 in PVE on host2 (web gui).

TASK ERROR: command '/usr/bin/proxmox-backup-client restore '--crypt-mode=encrypt' '--keyfd=13' vm/100/2020-11-17T06:41:48Z index.json /var/tmp/vzdumptmp7930/index.json --repository root@pam@192.168.1.129:pbs-host2-store1' failed: exit code 255

Spoiler: pveversion -v

proxmox-ve: 6.2-2 (running kernel: 5.4.65-1-pve)
pve-manager: 6.2-15 (running version: 6.2-15/48bd51b6)
pve-kernel-5.4: 6.3-1
pve-kernel-helper: 6.3-1
pve-kernel-5.4.73-1-pve: 5.4.73-1
pve-kernel-5.4.65-1-pve: 5.4.65-1
pve-kernel-5.4.34-1-pve: 5.4.34-2
ceph-fuse: 12.2.11+dfsg1-2.1+b1
corosync: 3.0.4-pve1
criu: 3.11-3
glusterfs-client: 5.5-3
ifupdown: residual config
ifupdown2: 3.0.0-1+pve3
ksm-control-daemon: 1.3-1
libjs-extjs: 6.0.1-10
libknet1: 1.16-pve1
libproxmox-acme-perl: 1.0.5
libpve-access-control: 6.1-3
libpve-apiclient-perl: 3.0-3
libpve-common-perl: 6.2-4
libpve-guest-common-perl: 3.1-3
libpve-http-server-perl: 3.0-6
libpve-storage-perl: 6.2-10
libqb0: 1.0.5-1
libspice-server1: 0.14.2-4~pve6+1
lvm2: 2.03.02-pve4
lxc-pve: 4.0.3-1
lxcfs: 4.0.3-pve3
novnc-pve: 1.1.0-1
proxmox-backup-client: 1.0.1-1
proxmox-mini-journalreader: 1.1-1
proxmox-widget-toolkit: 2.3-10
pve-cluster: 6.2-1
pve-container: 3.2-3
pve-docs: 6.2-6
pve-edk2-firmware: 2.20200531-1
pve-firewall: 4.1-3
pve-firmware: 3.1-3
pve-ha-manager: 3.1-1
pve-i18n: 2.2-2
pve-qemu-kvm: 5.1.0-6
pve-xtermjs: 4.7.0-2
qemu-server: 6.2-20
smartmontools: 7.1-pve2
spiceterm: 3.1-1
vncterm: 1.6-2
zfsutils-linux: 0.8.5-pve1

 

[fabian]

please post the full task log..

 

[onlineapps]

same issuse with last updates! 6.2-15

 

[JamesT]

hi @fabian , apologies, that is the full log as far as I can see. if there's another way to get more infomation, please let me know.

UPDATE:
I found a bit more info in syslog

Nov 23 08:21:57 EEAWS19 pvedaemon[8101]: <root@pam> starting task UPID:EEAWS19:00002FC7:01E73214:5FBB00A5:qmrestore:102:root@pam:
Nov 23 08:21:57 EEAWS19 proxmox-backup-proxy[3603]: starting new backup reader datastore 'store1-eeaws19': "/rpool/"
Nov 23 08:21:57 EEAWS19 proxmox-backup-proxy[3603]: protocol upgrade done
Nov 23 08:21:57 EEAWS19 proxmox-backup-proxy[3603]: GET /download
Nov 23 08:21:57 EEAWS19 proxmox-backup-proxy[3603]: download "/rpool/vm/100/2020-11-17T06:41:48Z/index.json.blob"
Nov 23 08:21:57 EEAWS19 proxmox-backup-proxy[3603]: TASK ERROR: connection error: Transport endpoint is not connected (os error 107)
Nov 23 08:21:57 EEAWS19 proxmox-backup-proxy[3603]: Detected stopped task 'UPID:EEAWS19:00000E13:00001E3A:00000056:5FBB00A5:reader:store1\x2deeaws19\x3avm-100-5FB370AC:root@pam:'
Nov 23 08:21:58 EEAWS19 pvedaemon[12231]: command '/usr/bin/proxmox-backup-client restore '--crypt-mode=encrypt' '--keyfd=13' vm/100/2020-11-17T06:41:48Z index.json /var/tmp/vzdumptmp12231/index.json --repository root@pam@192.168.1.129:store1-eeaws19' failed: exit code 255
Nov 23 08:21:58 EEAWS19 pvedaemon[8101]: <root@pam> end task UPID:EEAWS19:00002FC7:01E73214:5FBB00A5:qmrestore:102:root@pam: command '/usr/bin/proxmox-backup-client restore '--crypt-mode=encrypt' '--keyfd=13' vm/100/2020-11-17T06:41:48Z index.json /var/tmp/vzdumptmp12231/index.json --repository root@pam@192.168.1.129:store1-eeaws19' failed: exit code 255
Nov 23 08:22:00 EEAWS19 systemd[1]: Starting Proxmox VE replication runner...
Nov 23 08:22:00 EEAWS19 systemd[1]: pvesr.service: Succeeded.
Nov 23 08:22:00 EEAWS19 systemd[1]: Started Proxmox VE replication runner.

###END OF SYSLOG####

 

[fabian]

wrong signature can only mean two things
- you use the wrong key to restore
- the manifest file is corrupt

I'd guess the former in this case..

 

[onlineapps]

i have same error, i can give access to my test machine to find problem, thanks, best regards

 

[onlineapps]

i fixed my problem, i want to restore my VM to another cluster, i found solution for me here

 

[JamesT]

i fixed my problem, i want to restore my VM to another cluster, i found solution for me here

Thank you, that was exactly what I've been looking for. I will check it out.