Why did my vzdump fail to a broken pipe?

Alvks

New Member
Jun 1, 2019
17
0
1
25
Here is the log from my Proxmox node:

Code:
vzdump 100 102 --mailto root --mode snapshot --quiet 1 --compress lzo --mailnotification always --storage vm-backups


100: 2019-06-14 01:00:01 INFO: Starting Backup of VM 100 (qemu)
100: 2019-06-14 01:00:01 INFO: status = running
100: 2019-06-14 01:00:02 INFO: update VM 100: -lock backup
100: 2019-06-14 01:00:02 INFO: VM Name: entei
100: 2019-06-14 01:00:02 INFO: include disk 'sata0' 'hdd1:vm-100-disk-0' 3260G
100: 2019-06-14 01:00:02 INFO: include disk 'sata1' 'local-lvm:vm-100-disk-1' 50G
100: 2019-06-14 01:00:02 INFO: backup mode: snapshot
100: 2019-06-14 01:00:02 INFO: ionice priority: 7
100: 2019-06-14 01:00:02 INFO: pending configuration changes found (not included into backup)
100: 2019-06-14 01:00:02 INFO: creating archive '/srv/pve-vm-backups/dump/vzdump-qemu-100-2019_06_14-01_00_01.vma.lzo'
100: 2019-06-14 01:00:05 INFO: started backup task '7f4a5a6d-9988-42d5-b387-7ebfaf7d7f73'
100: 2019-06-14 01:00:08 INFO: status: 0% (490012672/3554085437440), sparse 0% (72097792), duration 3, read/write 163/139 MB/s
100: 2019-06-14 01:08:25 INFO: status: 1% (35559178240/3554085437440), sparse 0% (6890741760), duration 500, read/write 70/56 MB/s
100: 2019-06-14 01:17:19 INFO: status: 2% (71152828416/3554085437440), sparse 0% (13100220416), duration 1034, read/write 66/55 MB/s
100: 2019-06-14 01:24:26 INFO: status: 3% (106628120576/3554085437440), sparse 0% (15488667648), duration 1461, read/write 83/77 MB/s
100: 2019-06-14 01:29:26 INFO: status: 4% (142217707520/3554085437440), sparse 0% (15488667648), duration 1761, read/write 118/118 MB/s
100: 2019-06-14 01:34:25 INFO: status: 5% (177805459456/3554085437440), sparse 0% (15488667648), duration 2060, read/write 119/119 MB/s
100: 2019-06-14 01:39:25 INFO: status: 6% (213274329088/3554085437440), sparse 0% (18245079040), duration 2360, read/write 118/109 MB/s
100: 2019-06-14 01:47:27 INFO: status: 7% (248861032448/3554085437440), sparse 0% (18245079040), duration 2842, read/write 73/73 MB/s
100: 2019-06-14 01:52:43 INFO: status: 8% (284410642432/3554085437440), sparse 0% (18245079040), duration 3158, read/write 112/112 MB/s
100: 2019-06-14 01:57:55 INFO: status: 9% (319982731264/3554085437440), sparse 0% (18245079040), duration 3470, read/write 114/114 MB/s
100: 2019-06-14 02:03:08 INFO: status: 10% (355420733440/3554085437440), sparse 0% (18616782848), duration 3783, read/write 113/112 MB/s
100: 2019-06-14 02:08:52 INFO: status: 11% (390963331072/3554085437440), sparse 0% (18616782848), duration 4127, read/write 103/103 MB/s
100: 2019-06-14 02:14:40 INFO: status: 12% (426607509504/3554085437440), sparse 0% (18616782848), duration 4475, read/write 102/102 MB/s
100: 2019-06-14 02:20:00 INFO: status: 13% (462048854016/3554085437440), sparse 0% (18616782848), duration 4795, read/write 110/110 MB/s
100: 2019-06-14 02:25:43 INFO: status: 14% (497657446400/3554085437440), sparse 0% (18616782848), duration 5138, read/write 103/103 MB/s
100: 2019-06-14 02:31:21 INFO: status: 15% (533177434112/3554085437440), sparse 0% (18616782848), duration 5476, read/write 105/105 MB/s
100: 2019-06-14 02:37:01 INFO: status: 16% (568745132032/3554085437440), sparse 0% (18616782848), duration 5816, read/write 104/104 MB/s
100: 2019-06-14 02:42:35 INFO: status: 17% (604270362624/3554085437440), sparse 0% (18694754304), duration 6150, read/write 106/106 MB/s
100: 2019-06-14 02:48:08 INFO: status: 18% (639774425088/3554085437440), sparse 0% (18851127296), duration 6483, read/write 106/106 MB/s
100: 2019-06-14 02:53:45 INFO: status: 19% (675343106048/3554085437440), sparse 0% (19007983616), duration 6820, read/write 105/105 MB/s
100: 2019-06-14 02:59:22 INFO: status: 20% (710833733632/3554085437440), sparse 0% (19186581504), duration 7157, read/write 105/104 MB/s
100: 2019-06-14 03:05:04 INFO: status: 21% (746466574336/3554085437440), sparse 0% (19361800192), duration 7499, read/write 104/103 MB/s
100: 2019-06-14 03:10:49 INFO: status: 22% (781940817920/3554085437440), sparse 0% (19535155200), duration 7844, read/write 102/102 MB/s
100: 2019-06-14 03:16:41 INFO: status: 23% (817530404864/3554085437440), sparse 0% (19691941888), duration 8196, read/write 101/100 MB/s
100: 2019-06-14 03:22:32 INFO: status: 24% (852998094848/3554085437440), sparse 0% (19872370688), duration 8547, read/write 101/100 MB/s
100: 2019-06-14 03:28:34 INFO: status: 25% (888608587776/3554085437440), sparse 0% (20043321344), duration 8909, read/write 98/97 MB/s
100: 2019-06-14 03:34:40 INFO: status: 26% (924071100416/3554085437440), sparse 0% (20198240256), duration 9275, read/write 96/96 MB/s
100: 2019-06-14 03:40:53 INFO: status: 27% (959683493888/3554085437440), sparse 0% (20364861440), duration 9648, read/write 95/95 MB/s
100: 2019-06-14 03:47:07 INFO: status: 28% (995209576448/3554085437440), sparse 0% (20528205824), duration 10022, read/write 94/94 MB/s
100: 2019-06-14 03:53:26 INFO: status: 29% (1030693847040/3554085437440), sparse 0% (20682833920), duration 10401, read/write 93/93 MB/s
100: 2019-06-14 03:59:48 INFO: status: 30% (1066230808576/3554085437440), sparse 0% (20847558656), duration 10783, read/write 93/92 MB/s
100: 2019-06-14 04:06:16 INFO: status: 31% (1101847986176/3554085437440), sparse 0% (21001953280), duration 11171, read/write 91/91 MB/s
100: 2019-06-14 04:12:53 INFO: status: 32% (1137334943744/3554085437440), sparse 0% (21157699584), duration 11568, read/write 89/88 MB/s
100: 2019-06-14 04:19:40 INFO: status: 33% (1172890320896/3554085437440), sparse 0% (21320167424), duration 11975, read/write 87/86 MB/s
100: 2019-06-14 04:26:33 INFO: status: 34% (1208458215424/3554085437440), sparse 0% (21484761088), duration 12388, read/write 86/85 MB/s
100: 2019-06-14 04:33:40 INFO: status: 35% (1243994783744/3554085437440), sparse 0% (21640769536), duration 12815, read/write 83/82 MB/s
100: 2019-06-14 04:40:48 INFO: status: 36% (1279533645824/3554085437440), sparse 0% (21808959488), duration 13243, read/write 83/82 MB/s
100: 2019-06-14 04:48:55 INFO: status: 37% (1315028205568/3554085437440), sparse 0% (21970759680), duration 13730, read/write 72/72 MB/s
100: 2019-06-14 04:55:58 INFO: status: 38% (1350556975104/3554085437440), sparse 0% (22142111744), duration 14153, read/write 83/83 MB/s
100: 2019-06-14 05:01:08 INFO: status: 38% (1375579537408/3554085437440), sparse 0% (22251888640), duration 14463, read/write 80/80 MB/s
100: 2019-06-14 05:01:08 ERROR: vma_queue_write: write error - Broken pipe
100: 2019-06-14 05:01:08 INFO: aborting backup job
100: 2019-06-14 05:01:35 ERROR: Backup of VM 100 failed - vma_queue_write: write error - Broken pipe
I have a hunch though but just want confirmation. One of the disks used by the VM is twice as large as the external hard drive I am using for backups.

100: 2019-06-14 01:00:02 INFO: include disk 'sata0' 'hdd1:vm-100-disk-0' 3260G

That is a nearly 4TB disk that I was hoping could be compressed and backed up to a 2TB external hard drive. Looks like the backup job aborted after it ran out of space. Thoughts? I was hoping LZO or GZIP would do a better job compressing these snapshots but I guess I was wrong!
 

Faris Raouf

Member
Mar 19, 2018
97
7
8
50
I don't think you'll get 2:1 compression unless your disk image only contains plain text. There are too many things that are pre-compressed, at least to an extent. e.g. videos and images. Sure, plain text might be 2:1 or better. Having said that, if it fails at 38% I'm not sure what kind of compression you'r getting.
 

Alvks

New Member
Jun 1, 2019
17
0
1
25
I don't think you'll get 2:1 compression unless your disk image only contains plain text. There are too many things that are pre-compressed, at least to an extent. e.g. videos and images. Sure, plain text might be 2:1 or better. Having said that, if it fails at 38% I'm not sure what kind of compression you'r getting.
Honestly, that’s okay. The disk has a bunch of media on it but I have that all copied to an external that is passed through to the VM. How can I tell vzdump to ignore that 4TB disk and just backup the boot disk?
 

sb-jw

Active Member
Jan 23, 2018
551
49
28
28
How can I tell vzdump to ignore that 4TB disk and just backup the boot disk?
If you assigned this as an virtual disk, then you can enable the checkbox "no backup". But if it is pass-through, I think that's not possible, but you can backup your VM from inside with Veeam for free and Incrementel.
 

Alvks

New Member
Jun 1, 2019
17
0
1
25
If you assigned this as an virtual disk, then you can enable the checkbox "no backup". But if it is pass-through, I think that's not possible, but you can backup your VM from inside with Veeam for free and Incrementel.
Thanks for the suggestion! Another question. For snapshot backups through Proxmox, which compression method do you recommend? LZO or GZIP? GZIP seems to compress a tad bit more than LZO.
 

oguz

Proxmox Staff Member
Staff member
Nov 19, 2018
748
80
28
For snapshot backups through Proxmox, which compression method do you recommend? LZO or GZIP? GZIP seems to compress a tad bit more than LZO.
depends on what you want. lzo is a fast compressor/decompressor but achieves less compression, gzip is slower but you get more compression.
 
  • Like
Reactions: Alvks

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE and Proxmox Mail Gateway. We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get your own in 60 seconds.

Buy now!