pbs change data disk and import datastore

abranca

Member
Mar 6, 2017
29
0
6
34
Hi guys,
i'm testing pbs and i must say i'm very happy so far.
one of the tests i'm running is the following:
- backup of 1 ct on pbs with nfs drive
- backup of 1 vm on pbs with nfs drive

For safety I bring on mega the folders of the backups with rclone.
Now I simulate the breaking of the data disk by creating a new nfs folder and go to download from mega my backups.
Browsing the contents of my datastore I get the error

Code:
Bad Request (400)
unable to detect file type

I thought it was an ownership issue because I noticed that another backup had .chunck vm ct and .lock owned by the backup user while the one restored from mega belongs to root.
With chown backup:backup to the folders and files just mentioned I can browse the contents of the datastore and restore the machine but I can't make new backups of either ct or vm.

Regardless of where I move a copy of the backups, not using mega but a second hard disk, how is it possible to import the datastore in case of failure of the main data disk?

I post the error here.

Code:
Proxmox
Virtual Environment 6.3-3
Search
Container 113 (test) on node 'pve'
local-bk
Filter VMID
Server View
Logs
()
INFO: starting new backup job: vzdump 113 --mode snapshot --node pve --storage testds --remove 0
INFO: Starting Backup of VM 113 (lxc)
INFO: Backup started at 2020-12-10 19:42:28
INFO: status = running
INFO: CT Name: test
INFO: including mount point rootfs ('/') in backup
INFO: backup mode: snapshot
INFO: ionice priority: 7
INFO: create storage snapshot 'vzdump'
  /dev/sdc: open failed: No medium found
  Logical volume "snap_vm-113-disk-0_vzdump" created.
  /dev/sdd: open failed: No medium found
  /dev/sdc: open failed: No medium found
  /dev/sdd: open failed: No medium found
INFO: creating Proxmox Backup Server archive 'ct/113/2020-12-10T18:42:28Z'
INFO: run: lxc-usernsexec -m u:0:100000:65536 -m g:0:100000:65536 -- /usr/bin/proxmox-backup-client backup --crypt-mode=none pct.conf:/var/tmp/vzdumptmp15911_113/etc/vzdump/pct.conf root.pxar:/mnt/vzsnap0 --include-dev /mnt/vzsnap0/./ --skip-lost-and-found --exclude=/tmp/?* --exclude=/var/tmp/?* --exclude=/var/run/?*.pid --backup-type ct --backup-id 113 --backup-time 1607625748 --repository root@pam@192.168.0.210:testds
INFO: Starting backup: ct/113/2020-12-10T18:42:28Z
INFO: Client name: pve
INFO: Starting backup protocol: Thu Dec 10 19:42:29 2020
INFO: Couldn't download previous manifest - no valid previous backup
INFO: Upload config file '/var/tmp/vzdumptmp15911_113/etc/vzdump/pct.conf' to 'root@pam@192.168.0.210:8007:testds' as pct.conf.blob
INFO: Upload directory '/mnt/vzsnap0' to 'root@pam@192.168.0.210:8007:testds' as root.pxar.didx
INFO: catalog upload error - channel closed
INFO: Error: No such file or directory (os error 2)
INFO: cleanup temporary 'vzdump' snapshot
  /dev/sdc: open failed: No medium found
  Logical volume "snap_vm-113-disk-0_vzdump" successfully removed
  /dev/sdd: open failed: No medium found
ERROR: Backup of VM 113 failed - command 'lxc-usernsexec -m u:0:100000:65536 -m g:0:100000:65536 -- /usr/bin/proxmox-backup-client backup '--crypt-mode=none' pct.conf:/var/tmp/vzdumptmp15911_113/etc/vzdump/pct.conf root.pxar:/mnt/vzsnap0 --include-dev /mnt/vzsnap0/./ --skip-lost-and-found '--exclude=/tmp/?*' '--exclude=/var/tmp/?*' '--exclude=/var/run/?*.pid' --backup-type ct --backup-id 113 --backup-time 1607625748 --repository root@pam@192.168.0.210:testds' failed: exit code 255
INFO: Failed at 2020-12-10 19:42:43
INFO: Backup job finished with errors
TASK ERROR: job errors

Proxmox VE version
Code:
root@pve:~# pveversion -v
proxmox-ve: 6.3-1 (running kernel: 5.4.78-1-pve)
pve-manager: 6.3-3 (running version: 6.3-3/eee5f901)
pve-kernel-5.4: 6.3-3
pve-kernel-helper: 6.3-3
pve-kernel-5.3: 6.1-6
pve-kernel-5.4.78-2-pve: 5.4.78-2
pve-kernel-5.4.78-1-pve: 5.4.78-1
pve-kernel-5.4.73-1-pve: 5.4.73-1
pve-kernel-5.3.18-3-pve: 5.3.18-3
pve-kernel-5.3.18-2-pve: 5.3.18-2
ceph-fuse: 12.2.11+dfsg1-2.1+b1
corosync: 3.0.4-pve1
criu: 3.11-3
glusterfs-client: 5.5-3
ifupdown: residual config
ifupdown2: 3.0.0-1+pve3
ksm-control-daemon: 1.3-1
libjs-extjs: 6.0.1-10
libknet1: 1.16-pve1
libproxmox-acme-perl: 1.0.5
libproxmox-backup-qemu0: 1.0.2-1
libpve-access-control: 6.1-3
libpve-apiclient-perl: 3.1-3
libpve-common-perl: 6.3-2
libpve-guest-common-perl: 3.1-3
libpve-http-server-perl: 3.0-6
libpve-storage-perl: 6.3-3
libqb0: 1.0.5-1
libspice-server1: 0.14.2-4~pve6+1
lvm2: 2.03.02-pve4
lxc-pve: 4.0.3-1
lxcfs: 4.0.3-pve3
novnc-pve: 1.1.0-1
proxmox-backup-client: 1.0.5-1
proxmox-mini-journalreader: 1.1-1
proxmox-widget-toolkit: 2.4-3
pve-cluster: 6.2-1
pve-container: 3.3-1
pve-docs: 6.3-1
pve-edk2-firmware: 2.20200531-1
pve-firewall: 4.1-3
pve-firmware: 3.1-3
pve-ha-manager: 3.1-1
pve-i18n: 2.2-2
pve-qemu-kvm: 5.1.0-7
pve-xtermjs: 4.7.0-3
qemu-server: 6.3-2
smartmontools: 7.1-pve2
spiceterm: 3.1-1
vncterm: 1.6-2
zfsutils-linux: 0.8.5-pve1

Proxmox backup server version
Code:
proxmox-backup: 1.0-4
proxmox-backup-server: 1.0.1-1
pve-kernel-5.4: 6.2-7
pve-kernel-helper: 6.2-7
pve-kernel-5.4.65-1-pve: 5.4.65-1
ifupdown2: 3.0.0-1+pve3
libjs-extjs: 6.0.1-10
proxmox-backup-docs: 1.0.1-1
proxmox-backup-client: 1.0.1-1
proxmox-mini-journalreader: 1.1-1
proxmox-widget-toolkit: 2.3-10
pve-xtermjs: 4.7.0-2
smartmontools: 7.1-pve2
zfsutils-linux: 0.8.4-pve2
 

oguz

Proxmox Staff Member
Staff member
Nov 19, 2018
2,944
334
88
hi,


I thought it was an ownership issue because I noticed that another backup had .chunck vm ct and .lock owned by the backup user while the one restored from mega belongs to root.
With chown backup:backup to the folders and files just mentioned I can browse the contents of the datastore and restore the machine but I can't make new backups of either ct or vm.
have you tried running chown recursively with the -R flag? like so: chown -R backup:backup /your/dir. if you haven't run it recursively then you might still be lacking write access to the directory
 

abranca

Member
Mar 6, 2017
29
0
6
34
Hello and thank you for your response.
My mount point is /mnt/testds.
I actually gave the chown -R command only to the contents of the mount point (.lock .chunk ct vm) but not to the mount point itself (/mnt/testds).

I'll redo the test and tell you more. Thanks for your reply.
 

abranca

Member
Mar 6, 2017
29
0
6
34
I tested again with a few gig machine but did not solve the problem.

The original folder from my first backup was copied to external/remote device.

I deleted the original folder (simulating a disk failure or replacement), restored the copy of the original folder by mounting it in /mnt/pb-firewall.
I edited the /etc/proxmox-backup/datastore.cfg file by adding the existing datastore with the following commands:

Code:
datastore: pb-firewall
        path /mnt/pb-firewall


I reconnected the disk in proxmox ve as proxmox backup server drive.
I gave chown -R backup:backup /mnt/pb-firewall but no other backups can be created.

I attach here the errors:

PVE Log
Code:
INFO: starting new backup job: vzdump 101 --mode snapshot --storage pb-firewall --remove 0 --node pve
INFO: Starting Backup of VM 101 (qemu)
INFO: Backup started at 2020-12-17 15:38:42
INFO: status = running
INFO: VM Name: OpnSense
  /dev/sdc: open failed: No medium found
  /dev/sdd: open failed: No medium found
INFO: backup mode: snapshot
INFO: ionice priority: 7
INFO: creating Proxmox Backup Server archive 'vm/101/2020-12-17T14:38:42Z'
INFO: started backup task 'e1cc45d1-0dad-4712-8a5f-f83020ba5cd6'
INFO: resuming VM again
INFO: scsi0: dirty-bitmap status: OK (352.0 MiB of 32.0 GiB dirty)
INFO: using fast incremental mode (dirty-bitmap), 352.0 MiB dirty of 32.0 GiB total
INFO:   1% (4.0 MiB of 352.0 MiB) in  1s, read: 4.0 MiB/s, write: 4.0 MiB/s
ERROR: backup write data failed: command error: write_data upload error: pipelined request failed: No such file or directory (os error 2)
INFO: aborting backup job
ERROR: Backup of VM 101 failed - backup write data failed: command error: write_data upload error: pipelined request failed: No such file or directory (os error 2)
INFO: Failed at 2020-12-17 15:38:44
INFO: Backup job finished with errors
TASK ERROR: job errors

PBS Log
Code:
2020-12-17T15:38:43+01:00: starting new backup on datastore 'pb-firewall': "vm/101/2020-12-17T14:38:42Z"
2020-12-17T15:38:43+01:00: download 'index.json.blob' from previous backup.
2020-12-17T15:38:43+01:00: register chunks in 'drive-scsi0.img.fidx' from previous backup.
2020-12-17T15:38:43+01:00: download 'drive-scsi0.img.fidx' from previous backup.
2020-12-17T15:38:43+01:00: created new fixed index 1 ("vm/101/2020-12-17T14:38:42Z/drive-scsi0.img.fidx")
2020-12-17T15:38:43+01:00: add blob "/mnt/pb-firewall/vm/101/2020-12-17T14:38:42Z/qemu-server.conf.blob" (389 bytes, comp: 389)
2020-12-17T15:38:43+01:00: POST /fixed_chunk: 400 Bad Request: No such file or directory (os error 2)
2020-12-17T15:38:43+01:00: POST /fixed_chunk: 400 Bad Request: error reading a body from connection: protocol error: stream no longer needed
2020-12-17T15:38:43+01:00: backup ended and finish failed: backup ended but finished flag is not set.
2020-12-17T15:38:43+01:00: removing unfinished backup
2020-12-17T15:38:43+01:00: TASK ERROR: removing backup snapshot "/mnt/pb-firewall/vm/101/2020-12-17T14:38:42Z" failed - Directory not empty (os error 39)
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE and Proxmox Mail Gateway. We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get your own in 60 seconds.

Buy now!