Proxmox backups

[smcnallie]

I have set up backups through the Proxmox Backup screen on the web interface using snapshot mode (screenshot attached). I'm backing up to a 1.5TB external USB drive. For smaller VMs, the backups work just fine. For the two large VMs, over 100GB, the backups hang. I was able to get one of the larger vm's to backup using suspend mode, but I would rather use snapshot.

Below is lvs when the backup fails. The backup hangs when snap% 100. The LSize is 100MB in this example, but i've set it as high as 4GB. Same result every time. If I set it higher than 4GB, I get a this error: Insufficient free extents (1023) in volume group pve: 2500 required.

virt0:/etc# lvs
/dev/dm-2: read failed after 0 of 4096 at 0: Input/output error
LV VG Attr LSize Origin Snap% Move Log Copy% Convert
data pve owi-ao 1.65T
root pve -wi-ao 96.00G
swap pve -wi-ao 15.00G
vzsnap-virt0-0 pve Swi-Io 100.00M data 100.00


Here is lvdisplay when snap% gets to 100.


virt0:/etc# lvdisplay /dev/pve/vzsnap-virt0-0
/dev/dm-2: read failed after 0 of 4096 at 0: Input/output error
--- Logical volume ---
LV Name /dev/pve/vzsnap-virt0-0
VG Name pve
LV UUID qQdp5E-XdxW-8Wf8-FZcO-DajF-eWwf-Ccde2B
LV Write Access read/write
LV snapshot status INACTIVE destination for /dev/pve/data
LV Status available
# open 1
LV Size 1.65 TB
Current LE 431232
COW-table size 100.00 MB
COW-table LE 25
Snapshot chunk size 4.00 KB
Segments 1
Allocation inherit
Read ahead sectors auto
- currently set to 256
Block device 251:2



Any ideas how to get the backup to work in snapshot mode? What exactly is snap% and why is it getting to 100?

Thanks!

 

[tom]

post the output of /etc/vzdump.conf and pvdisplay.

I expect you have only 4 GB free for snapshots, so if you can´t use more than 4 GB.

 

[smcnallie]

post the output of /etc/vzdump.conf and pvdisplay.

I expect you have only 4 GB free for snapshots, so if you can´t use more than 4 GB.

vzdump.conf currently has:
size:100

Like I said in the prior post, I've changed this to other values. 1000, 2000, 4000. Snap% always hits 100 and the backup hangs.

pvdisplay:

virt0:/etc# pvdisplay
--- Physical volume ---
PV Name /dev/sda2
VG Name pve
PV Size 1.76 TB / not usable 3.86 MB
Allocatable yes
PE Size (KByte) 4096
Total PE 460671
Free PE 998
Allocated PE 459673
PV UUID QBqG5a-Rrgh-7L5o-JJfg-9VLu-ExPQ-Faj2oq


How do I make more space for snapshots?

 

[tom]

you have 4 GB Free in total.

(PE Size (KByte) 4096 x Free PE 998 = 4087808 = 4GB)

this means, you can set up to 4 GB in /etc/vzdump.conf (set 4096).

if you want more here you need to change your LVM setup - reduce /dev/pve/data. not that trivial, you need to know what you are doing here.

but I suggest you need to find out why you have so many writes during backup (more than 4 GB inside your VM). give detail about your VMID.conf and your OS and application inside.

also tell more about your backup target, maybe its too slow and therefore it take too just too long.

and good news on the end, the Proxmox VE 2.3 can do live backups without LVM so you problem will be solved anyways. maybe we release a beta of this before christmas, depends on the internal testing.

 

[smcnallie]

you have 4 GB Free in total.

(PE Size (KByte) 4096 x Free PE 998 = 4087808 = 4GB)

this means, you can set up to 4 GB in /etc/vzdump.conf (set 4096).

if you want more here you need to change your LVM setup - reduce /dev/pve/data. not that trivial, you need to know what you are doing here.

but I suggest you need to find out why you have so many writes during backup (more than 4 GB inside your VM). give detail about your VMID.conf and your OS and application inside.

also tell more about your backup target, maybe its too slow and therefore it take too just too long.

and good news on the end, the Proxmox VE 2.3 can do live backups without LVM so you problem will be solved anyways. maybe we release a beta of this before christmas, depends on the internal testing.

This VM OS is Small Business Server 2011 and it's main funtions are MS Sql Database server for our website, Exchange Server, and shared drives. Here is the vmid.conf output:
name: Small_Business_Server
ide2: cdrom,media=cdrom
vlan0: e1000=79:34:44:CA:46:6R
bootdisk: ide0
ostype: w2k8
memory: 12000
onboot: 1
sockets: 1
cores: 6
ide0: local:106/vm-110-disk-1.raw
ide1: local:106/vm-110-disk-2.raw
ide3: local:106/vm-110-disk-3.raw
boot: cad
freeze: 0
cpuunits: 1000
vga: std
acpi: 1
kvm: 1


I don't know much about the USB drive besides that it's a 1.5 TB seagate. It took about 10 hours to do a 181.0 GB Suspend backup. So it's pretty slow.

When you say to "find out why there are so many writes during backup", do you mean what is happening inside the OS at the time of backup?

Thanks for the quick responses!

 

[tom]

yes. as this is a quite a big VM I expect there are a lot of writes in 10 hours.

so 3 solutions:
- faster backup drive and hopefully the backup is finished before the snapshot is full.
- bigger snapshot (LVM resize, work)
- do nothing and wait till 2.3

maybe you choose the third solution.?

you can also think of adding a small virtio harddisk (e.g. 10 GB), using "cache writeback" and "backup no". format this drive and use it ONLY for the windows swap file (pagefile.sys). disable all swapspace on other drives.

by doing this, all writes to the swap are not in the backup. if you restore, the windows will just complain about a missing swap file and will create a new one.
test it.

 

[smcnallie]

I'd like try and back up to the server hard disk. Unfortunately, the server only has 90GB which is not enough for the VM backup. Is it a problem to create a new storage device on /var/lib/vz? It has plenty of space. Attached is a screenshot of what i'm talking about.

 

[smcnallie]

Any answer on my last comment. Good/Bad idea?

 

[tom]

if you want to use LVM snapshots, you cannot use dir under /var/lib/vz/ ...

this will create a loop. only the upcoming new KVM live snapshot feature (for 2.3) will be able to use this dir for live backups.

 

[smcnallie]

if you want to use LVM snapshots, you cannot use dir under /var/lib/vz/ ...

this will create a loop. only the upcoming new KVM live snapshot feature (for 2.3) will be able to use this dir for live backups.

Thanks Tom. I think i'll wait for 2.3 and just make sure my data within the vm is backed up for now.