How to backup on Proxmox?

E

eiger3970

Well-Known Member
Sep 9, 2012
276
3
58
Hello, I would like some help understanding backups on Proxmox. I have 2 backups no saying Disk usage is 90.6% and 23.2%. All I need for now is to backup a VM and then transfer it over to another computer.
I understand that if a VM breaks, I could then bring the backed up file from the other Desktop computer, into the Proxmox filepath and restore and the VM will work.
 
Hi,
putting it simple, backups in proxmox work in two ways:

manual or automatic. both ways need a storage enabled for backups (it may be the same for both ways)

you can take manual backups form the vm tab in the pve gui
you can schedule automatic backups from the datacenter tab

any storage enabled for backups has a configurable policy for rotation: you can define the maximum number of backups to keep for a vm, on that storage (reached that limit, old backups are deleted when new backups are taken).

[note: iirc, that rotation happens automatically only for automatic scheduled backups, not for manual ones. so, if you have a limit of, say, 2 and there are already 2 backup copies for a vm, you can't take another backup manually for that vm, there, unless you manually delete one of the 2 old backups (the oldest, I suppose)

instead, the scheduled ones will do this "rotation" themseles, and you will always end with that maximum number of backup copies for the vm. ]

if use a "local" (to the node) folder, your backups will stay on the node until you move them away, to other machines, and you will have to move them back when you need to restore them on that node

if you use a "network" folder (like nfs on a nas, but there are other options) your backup will be stored directly there, so no need to move them after completion, and they will always be available as long as the network storage is available to the pve node.

I hope this, as I understand it, clears the topic a bit more, to you

Marco
 
Last edited:
So I setup a Storage for BackupFiles.vma.lzo in /var/backups/dump Shared.
I try to remove the wrong Storage I setup to /var/lib/vz, but Remove doesn't remove it?
At the moment, I just Backup to a local Proxmox node directory, then SCP the file to a Desktop computer.
I would like this all to be automatic, so I think I'll try an NFS. Someone said NFS isn't worth it on a Desktop computer though?

So, I think local is the Proxmox default Storage location, so I am using this default local storage now and Removed my created Storage.

I tested a backup and
3 VMs worked but,
1. received no email,

2. VM2 had error Failed to allocate 1073741824 B: Cannot allocate memory and

3. VM3 had error: ERROR: vma_co_write write error - Broken pipe
<linuxmint> INFO: aborting backup job
<linuxmint> ERROR: Backup of VM 163 failed - vma_co_write write error - Broken pipe

Here are the full details:
VM 162 error:
INFO: Starting Backup of VM 162 (qemu)
INFO: status = stopped
INFO: update VM 162: -lock backup
INFO: backup mode: stop
INFO: ionice priority: 7
INFO: creating archive '/var/lib/vz/dump/vzdump-qemu-162-2014_11_03-12_49_42.vma.lzo'
INFO: starting kvm to execute backup task
Failed to allocate 1073741824 B: Cannot allocate memory
ERROR: Backup of VM 162 failed - start failed: command '/usr/bin/kvm -id 162 -chardev 'socket,id=qmp,path=/var/run/qemu-server/162.qmp,server,nowait' -mon 'chardev=qmp,mode=control' -vnc unix:/var/run/qemu-server/162.vnc,x509,password -pidfile /var/run/qemu-server/162.pid -daemonize -name Mining -smp 'sockets=1,cores=1' -nodefaults -boot 'menu=on' -vga cirrus -cpu kvm64,+x2apic,+sep -k en-us -m 1024 -device 'piix3-usb-uhci,id=uhci,bus=pci.0,addr=0x1.0x2' -device 'usb-tablet,id=tablet,bus=uhci.0,port=1' -device 'virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x3' -drive 'file=/var/lib/vz/template/iso/ubuntu-14.04-server-amd64.iso,if=none,id=drive-ide2,media=cdrom,aio=native' -device 'ide-cd,bus=ide.1,unit=0,drive=drive-ide2,id=ide2,bootindex=200' -drive 'file=/var/lib/vz/images/162/vm-162-disk-1.raw,if=none,id=drive-virtio0,format=raw,aio=native,cache=none' -device 'virtio-blk-pci,drive=drive-virtio0,id=virtio0,bus=pci.0,addr=0xa,bootindex=100' -netdev 'type=tap,id=net0,ifname=tap162i0,script=/var/lib/qemu-server/pve-bridge' -device 'e1000,mac=2E:92:98:F6:FB:CF,netdev=net0,bus=pci.0,addr=0x12,id=net0,bootindex=300' -S' failed: exit code 1

VM 163 error:
INFO: status: 15% (8085307392/53687091200), sparse 0% (447283200), duration 175, 44/44 MB/s
lzop: No space left on device: <stdout>
INFO: status: 15% (8210350080/53687091200), sparse 0% (454688768), duration 178, 41/39 MB/s
ERROR: vma_co_write write error - Broken pipe
INFO: aborting backup job
ERROR: Backup of VM 163 failed - vma_co_write write error - Broken pipe
 
Last edited:
eiger3970 said:
3 VMs worked but,
1. received no email,
2. VM2 had error Failed to allocate 1073741824 B: Cannot allocate memory and
3. VM3 had error: ERROR: vma_co_write write error - Broken pipe
Click to expand...

I'm not sure to understand what happens but

about 1)
- you should test your mail system, is postfix (or equivalent) installed?
- if postfix (or equivalent) is up and running you should test the mail address configured (during install I guess), see/edit the file: /etc/pve/vzdump.cron

about 2) this seems to show that the vm can't allocate its virtual ram (if you are you using dynamic ram for this vm) when it needs.

about 3) sometimes this is caused by lack of disk space
see http://forum.proxmox.com/threads/17266-Solved-Backup-Failed

try to check and report also your "pveversion -v" output (this will help others to help you)

Apart that, yes a desktop is not always the best choice as a network share (even a small server like qnaps or synology 2 units work better and have minimal power requirements)

Marco
 
Last edited:
Thank you for the reply.

# pveversion -v
proxmox-ve-2.6.32: 3.1-114 (running kernel: 2.6.32-20-pve)
pve-manager: 3.1-21 (running version: 3.1-21/93bf03d4)
pve-kernel-2.6.32-20-pve: 2.6.32-100
pve-kernel-2.6.32-26-pve: 2.6.32-114
lvm2: 2.02.98-pve4
clvm: 2.02.98-pve4
corosync-pve: 1.4.5-1
openais-pve: 1.1.4-3
libqb0: 0.11.1-2
redhat-cluster-pve: 3.2.0-2
resource-agents-pve: 3.9.2-4
fence-agents-pve: 4.0.0-2
pve-cluster: 3.0-8
qemu-server: 3.1-8
pve-firmware: 1.0-23
libpve-common-perl: 3.0-8
libpve-access-control: 3.0-7
libpve-storage-perl: 3.0-17
pve-libspice-server1: 0.12.4-2
vncterm: 1.1-4
vzctl: 4.0-1pve4
vzprocps: 2.0.11-2
vzquota: 3.1-2
pve-qemu-kvm: 1.4-17
ksm-control-daemon: 1.1-1
glusterfs-client: 3.4.1-1



1. Not quite sure how or what else to configure Proxmox's email settings.
I remember previous backup's did send an email though.

/etc/pve/vzdump.cron
# cluster wide vzdump cron schedule
# Atomatically generated file - do not edit


PATH="/usr/sbin:/usr/bin:/sbin:/bin"


0 3 * * 5 root vzdump 161 162 163 164 165 --quiet 1 --mode snapshot -$


2. Yes, it's possible the RAM is dynamic. I'll adjust that.


3. Disk space should be fine.
The proxmox Summary says:
HD space (root) Total: 27.31GB
Used: 1.25GB

The backup is only about 1GB.
 
eiger3970 said:
# pveversion -v
proxmox-ve-2.6.32: 3.1-114 (running kernel: 2.6.32-20-pve)
Click to expand...

atm I have a
pve-manager/3.1-24/060bd5a6 (running kernel: 2.6.32-26-pve)

eiger3970 said:
1. Not quite sure how or what else to configure Proxmox's email settings.
I remember previous backup's did send an email though.
/etc/pve/vzdump.cron
# cluster wide vzdump cron schedule
# Atomatically generated file - do not edit
PATH="/usr/sbin:/usr/bin:/sbin:/bin"
0 3 * * 5 root vzdump 161 162 163 164 165 --quiet 1 --mode snapshot -$
Click to expand...

my vzdump.cron is quite different (but it could well be that your is a newer pve version) and has explicit mail address, like:

0 1 * * 2,3,4,5,7 root vzdump 202 203 204 205 206 207 209 211 302 310 400 101 900 102 107 901 106 216 118 119 122 123 --quiet 1 --mode snapshot --mailto user@domain.it --compress lzo --storage pve_ts879

your line has "-$" which could be a substitution variable: check in syslog at backup time to see what real command is issued and if it fails for what reason

eiger3970 said:
2. Yes, it's possible the RAM is dynamic. I'll adjust that.
Click to expand...

There's nothing wrong having it, but somehow your vm seem unable to find the free ram it needs (dynamic vm ram could ask more memory at vm startup and then release it after, either by balooning or ksm, see http://pve.proxmox.com/wiki/Dynamic_Memory_Management). Just disable it to confirm if it is related or not.

eiger3970 said:
3. Disk space should be fine.
The proxmox Summary says:
HD space (root) Total: 27.31GB
Used: 1.25GB
The backup is only about 1GB.
Click to expand...

I don't know the real reason, but to be sure you should look at the vm.conf, disk real size, storage folder size, permissions etc.
Maybe others will help you better.


Marco
 
1. My Terminal was hiding details.
The full code is:
# cluster wide vzdump cron schedule
# Atomatically generated file - do not edit


PATH="/usr/sbin:/usr/bin:/sbin:/bin"


0 3 * * 5 root vzdump 161 162 163 164 165 --quiet 1 --mode snapshot -$shot --mailto myCorrectEmailAddress@domain.com --compress lzo --storage local

I thought the email failed twice because the Backup had errors, but I do remember the email working a few days ago.


2. Already removed and am rebuilding.


3. I logged in to Proxmox > /etc/qemuserver, but nothing there for vm.conf file?
 
eiger3970 said:
1. My Terminal was hiding details.
The full code is:
# cluster wide vzdump cron schedule
# Atomatically generated file - do not edit


PATH="/usr/sbin:/usr/bin:/sbin:/bin"


0 3 * * 5 root vzdump 161 162 163 164 165 --quiet 1 --mode snapshot -$shot --mailto myCorrectEmailAddress@domain.com --compress lzo --storage local

I thought the email failed twice because the Backup had errors, but I do remember the email working a few days ago.


2. Already removed and am rebuilding.


3. I logged in to Proxmox > /etc/qemuserver, but nothing there for vm.conf file?
Click to expand...

you should look syslog (even gz/archived) for vzdump and postfic (or whatever MTA you have instead of it)

eg: my syslog.2.gz today has
Code: 
Nov  2 01:00:01 pve2 /USR/SBIN/CRON[602123]: (root) CMD (vzdump 101... 400 --quiet 1 --mode snapshot --mailto [EMAIL="user@domain.it"]user@domain.it[/EMAIL] --compress lzo --storage pve_ts879)
Nov  2 01:00:02 pve2 vzdump[602124]: <root@pam> starting task ....
Nov  2 01:00:02 pve2 vzdump[602126]: INFO: starting new backup job: vzdump 101 ... 400 --quiet 1 --mailto [EMAIL="user@domain.it"]user@domain.it[/EMAIL] --mode snapshot --compress lzo --storage pve_ts879
Nov  2 01:00:02 pve2 vzdump[602126]: INFO: Starting Backup of VM 101 (qemu)
...
...
Nov  2 02:33:49 pve2 vzdump[602126]: INFO: Finished Backup of VM 400 (00:06:26)
Nov  2 02:33:49 pve2 vzdump[602126]: INFO: Backup job finished successfully
Nov  2 02:33:49 pve2 vzdump[602124]: <root@pam> end task ...: OK

Nov  2 02:33:49 pve2 postfix/smtp[610412]: 9E92982183: to=<[EMAIL="user@domain.it"]user@domain.it[/EMAIL]>, relay=192.168.3.6[192.168.3.6]:25, delay=0.17, delays=0.06/0.02/0.05/0.04, dsn=2.0.0, status=sent (250 Data received OK.)

so you see that right after backup finishes, the mail is sent

Marco
 
Thanks.
Yes, my Backup log didn't show those lines that sent the email.
Not quite sure on the path to syslog (even gz/archived) for vzdump and postfic.
 
well you only have 1 file (syslog)...

so something like
"cat /var/log/syslog | grep -e "vzdump" -e "your@emailaddress.com"

should pop out at least relevant lines

Marco
 
Last edited:
Actually, there's a few files in /var/log:
syslog
syslog.1
syslog.2.gz
syslog.3.gz
syslog.4.gz
syslog.5.gz
syslog.6.gz
syslog.7.gz

root@proxmox:/var/log# cat 7var7log/syslog | grep -e "vzdump" -e "myEmailAddress@domain.com"
cat: 7var7log/syslog: No such file or directory
 
sorry it was a typo :D

cat /var/log/syslog | grep -e "vzdump" -e "your@emailaddress.com"
cat /var/log/syslog.1 | grep -e "vzdump" -e "your@emailaddress.com"

to look in gz syslog entries try

zcat /var/log/syslog.*.gz | grep -e "vzdump" -e "your@emailaddress.com"

Marco
 
Thanks, I thought so, but wasn't sure what the correct command was.

Still not receiving any output?

root@proxmox:/var/log# cat /var/log/syslog | grep -e "vzdump" -e "myEmailAddress@domain.com"
root@proxmox:/var/log# zcat /var/log/syslog | grep -e "vzdump" -e "myEmailAddress@domain.com"


gzip: /var/log/syslog: unexpected end of file
 
yes zcat works only on gz files

try
root@proxmox:/var/log# zcat /var/log/syslog.2.gz | grep -e "vzdump" -e "myEmailAddress@domain.com"

or

root@proxmox:/var/log# zcat /var/log/syslog.*.gz | grep -e "vzdump" -e "myEmailAddress@domain.com"

Marco
 
Still no output?

root@proxmox:/var/log# zcat /var/log/syslog.2.gz | grep -e "vzdump" -e "myEmailAddress@domain.com"
root@proxmox:/var/log# zcat /var/log/syslog.*.gz | grep -e "vzdump" -e "myEmailAddress@domain.com"
 
well, this is strange...
I think you should see at least vzdump log entries...

You could try to launch backups manually on the command line like

vzdump 161 --quiet 1 --mode snapshot -$shot --mailto myCorrectEmailAddress@domain.com --compress lzo --storage local

and see the /var/log/syslog

(but I still don't know what this "-$shot" is for)

Marco
 
Thanks.

root@proxmox:/var/log# vzdump 161 --quiet 1--mode snapshot -$shot --mailto myCorrectEmailAddress@domain.com --compress lzo -storage local
400 unable to parse boolean option
vzdump {<vmid>} [OPTIONS]
root@proxmox:/var/log# cd /var/log/syslog
-bash: cd: /var/log/syslog: Not a directory
root@proxmox:/var/log# nano /var/log/syslog
Blank page appears.
 
eiger3970 said:
Thanks.

root@proxmox:/var/log# vzdump 161 --quiet 1--mode snapshot -$shot --mailto myCorrectEmailAddress@domain.com --compress lzo -storage local
400 unable to parse boolean option
vzdump {<vmid>} [OPTIONS]
root@proxmox:/var/log# cd /var/log/syslog
-bash: cd: /var/log/syslog: Not a directory
root@proxmox:/var/log# nano /var/log/syslog
Blank page appears.
Click to expand...

try (without "-$shot")
vzdump 161 --quiet 1--mode snapshot --mailto myCorrectEmailAddress@domain.com --compress lzo -storage local

and you can't cd /var/log/syslog
syslog is a file in cd /var/log/, not a folder

so
nano cd /var/log/syslog

should work

Marco
 
You must log in or register to reply here.
Top Bottom