Cannot backup VM, Operation not permitted

L

lpallard

Renowned Member
Mar 22, 2014
99
5
73
Hello

Using proxmox 3.2-1/1933730b, I am trying to make a backup of a VM (KVM) using the local node's backup resource, and it fails every time with :

Code: 
ERROR: job failed with err -1 - Operation not permitted
INFO: aborting backup job
INFO: stopping kvm after backup task
ERROR: Backup of VM 101 failed - job failed with err -1 - Operation not permitted

The VM is stopped at the moment the backup is started (manually shutdown from within Guest Centos OS) and backup is started with no compression, in stopped mode.

I have already backed up other VM's to the same backup destination so I know it works. Also, there is plenty of space left on the backup destination (237GB) while the VM I am trying to backup is 54GB.

Complete task log:

Code: 
INFO: starting new backup job: vzdump 101 --remove 0 --mode stop --storage backups --node proxmox
INFO: Starting Backup of VM 101 (qemu)
INFO: status = stopped
INFO: update VM 101: -lock backup
INFO: backup mode: stop
INFO: ionice priority: 7
INFO: creating archive '/mnt/backups/dump/vzdump-qemu-101-2014_04_26-12_01_34.vma'
INFO: starting kvm to execute backup task
INFO: started backup task '76969dec-f690-4302-a69e-2b1782fe7a3a'
INFO: status: 0% (854065152/13806035533824), sparse 0% (591794176), duration 3, 284/87 MB/s
INFO: status: 1% (138063380480/13806035533824), sparse 0% (17557340160), duration 829, 166/145 MB/s
INFO: status: 2% (276173291520/13806035533824), sparse 0% (17557590016), duration 1690, 160/160 MB/s
INFO: status: 2% (289787478016/13806035533824), sparse 0% (17557626880), duration 1779, 152/152 MB/s
ERROR: job failed with err -1 - Operation not permitted
INFO: aborting backup job
INFO: stopping kvm after backup task
ERROR: Backup of VM 101 failed - job failed with err -1 - Operation not permitted
INFO: Backup job finished with errors
TASK ERROR: job errors

That would be great to be able to backup this VM since its been in production for about 2 months and no backups have been performed so far...

Any insight appreciated.

Best regards and thank you!
 
Hello lpallard

According to

lpallard said:
INFO: status: 0% (854065152/13806035533824), sparse 0% (591794176), duration 3, 284/87 MB/s
INFO: status: 1% (138063380480/13806035533824), sparse 0% (17557340160), duration 829, 166/145 MB/s
Click to expand...

etc. the HD size of the VM is 13 TB - and after 289 GB it cannot continue.

Are you sure? Verify your VM config (-files)!

Kind regards

Mr.Holmes
 
etc. the HD size of the VM is 13 TB - and after 289 GB it cannot continue.

Are you sure? Verify your VM config (-files)!
Click to expand...

According to ls -l the VM is 128GB (which rings a bell and seems familiar)

root@proxmox:/# ls -l /var/lib/vz/images/101
total 110633244
-rw-r--r-- 1 root root 128849018880 Apr 26 21:00 vm-101-disk-2.raw
Click to expand...

128849018880 = 128.8GB

root@proxmox:/# df -H
Filesystem Size Used Avail Use% Mounted on
udev 11M 0 11M 0% /dev
tmpfs 6.8G 463k 6.8G 1% /run
/dev/mapper/pve-root 32G 1.3G 29G 5% /
tmpfs 5.3M 4.1k 5.3M 1% /run/lock
tmpfs 14G 29M 14G 1% /run/shm
/dev/mapper/pve-data 209G 170G 39G 82% /var/lib/vz
/dev/sda2 518M 62M 430M 13% /boot
/dev/mapper/datastore-backups 318G 91G 211G 31% /mnt/backups
/dev/fuse 32M 17k 32M 1% /etc/pve
Click to expand...

Backup has 211GB left.. Should be enough to store a backup of 128GB

The VM that is failing to backup (other than its main VHD) has also two drives assignment. One is a software RAID passed to the VM via the vm.conf file (virtio) and the other is a LV from the node passed to the VM also via vm.conf.

I have setup a test VM using the exact same CentOS release and tried to backup. It worked.

I am thinking it has something to do with the MD raid or the LVM. Could it be?

EDIT: I just realized ALL drives assigned to the VM added together gives roughly 13TB

VHD Virtio0: 120GB
VHD Virtio1 (mdadm): 10TB
VHD Virtio2 (LV): 3TB

So I take that proxmox is trying to backup ALL associated drives, not only the VM image! Thats nonsense. How do I disable this!?
 
Last edited:
lpallard said:
According to ls -l the VM is 128GB (which rings a bell and seems familiar)



128849018880 = 128.8GB



Backup has 211GB left.. Should be enough to store a backup of 128GB

The VM that is failing to backup (other than its main VHD) has also two drives assignment. One is a software RAID passed to the VM via the vm.conf file (virtio) and the other is a LV from the node passed to the VM also via vm.conf.

I have setup a test VM using the exact same CentOS release and tried to backup. It worked.

I am thinking it has something to do with the MD raid or the LVM. Could it be?

EDIT: I just realized ALL drives assigned to the VM added together gives roughly 13TB

VHD Virtio0: 120GB
VHD Virtio1 (mdadm): 10TB
VHD Virtio2 (LV): 3TB

So I take that proxmox is trying to backup ALL associated drives, not only the VM image! Thats nonsense. How do I disable this!?
Click to expand...

Found the problem. The backup was really 13TB (!) because of what I explained in my previous post. I added ",backup=no" at the end of the lines of the VHD's in the VMID.conf and I could backup the VM.

Perhaps I missed that proxmox was backing up all HD's by default but this is what it does.

I hope this will help others with similar issue..
 
You must log in or register to reply here.
Top Bottom