NFS Performance / Cifs 3.1

[mgerste]

Hi! Since i've upgraded from 3.0 to 3.1 NFS Performance ist horrible.

Before:

102: Nov 06 02:35:59 INFO: Starting Backup of VM 102 (qemu)
102: Nov 06 02:35:59 INFO: status = running
102: Nov 06 02:35:59 INFO: exclude disk 'virtio1' (backup=no)
102: Nov 06 02:35:59 INFO: backup mode: snapshot
102: Nov 06 02:35:59 INFO: ionice priority: 7
102: Nov 06 02:35:59 INFO: creating archive '/mnt/pve/BACKUP/dump/vzdump-qemu-102-2013_11_06-02_35_59.vma.lzo'
102: Nov 06 02:36:00 INFO: started backup task '5f51049c-7c2d-4572-bceb-c609de5d2b32'
102: Nov 06 02:36:03 INFO: status: 0% (299565056/751619276800), sparse 0% (128274432), duration 3, 99/57 MB/s
102: Nov 06 02:38:09 INFO: status: 1% (7556562944/751619276800), sparse 0% (793174016), duration 129, 57/52 MB/s
102: Nov 06 02:41:49 INFO: status: 2% (15036710912/751619276800), sparse 0% (1268310016), duration 349, 34/31 MB/s
102: Nov 06 02:46:01 INFO: status: 3% (22593667072/751619276800), sparse 0% (2167083008), duration 601, 29/26 MB/s
102: Nov 06 02:48:59 INFO: status: 4% (30085611520/751619276800), sparse 0% (2934419456), duration 779, 42/37 MB/s
102: Nov 06 02:54:29 INFO: status: 5% (37588959232/751619276800), sparse 0% (3110170624), duration 1109, 22/22 MB/s
102: Nov 06 02:59:57 INFO: status: 6% (45099450368/751619276800), sparse 0% (3211063296), duration 1437, 22/22 MB/s
102: Nov 06 03:05:50 INFO: status: 7% (52626063360/751619276800), sparse 0% (3349630976), duration 1790, 21/20 MB/s
102: Nov 06 03:11:03 INFO: status: 8% (60137013248/751619276800), sparse 0% (3681865728), duration 2103, 23/22 MB/s
102: Nov 06 03:16:58 INFO: status: 9% (67663167488/751619276800), sparse 0% (3813318656), duration 2458, 21/20 MB/s
102: Nov 06 03:22:26 INFO: status: 10% (75166515200/751619276800), sparse 0% (3973238784), duration 2786, 22/22 MB/s
102: Nov 06 03:27:44 INFO: status: 11% (82702958592/751619276800), sparse 0% (4197785600), duration 3104, 23/22 MB/s
102: Nov 06 03:32:50 INFO: status: 12% (90199818240/751619276800), sparse 0% (4437299200), duration 3410, 24/23 MB/s
102: Nov 06 03:38:08 INFO: status: 13% (97750417408/751619276800), sparse 0% (4728258560), duration 3728, 23/22 MB/s
102: Nov 06 03:43:52 INFO: status: 14% (105244524544/751619276800), sparse 0% (5079506944), duration 4072, 21/20 MB/s
102: Nov 06 03:49:41 INFO: status: 15% (112774479872/751619276800), sparse 0% (5357940736), duration 4421, 21/20 MB/s
102: Nov 06 03:53:22 INFO: status: 16% (120327241728/751619276800), sparse 0% (5970874368), duration 4642, 34/31 MB/s
102: Nov 06 03:58:18 INFO: status: 17% (127788777472/751619276800), sparse 0% (6635188224), duration 4938, 25/22 MB/s
102: Nov 06 04:01:53 INFO: status: 18% (135382958080/751619276800), sparse 0% (7318396928), duration 5153, 35/32 MB/s
102: Nov 06 04:02:35 INFO: status: 19% (142895742976/751619276800), sparse 1% (7765274624), duration 5195, 178/168 MB/s
102: Nov 06 04:03:19 INFO: status: 20% (150473015296/751619276800), sparse 1% (8303472640), duration 5239, 172/159 MB/s
102: Nov 06 04:04:03 INFO: status: 21% (157946019840/751619276800), sparse 1% (8706113536), duration 5283, 169/160 MB/s
102: Nov 06 04:04:45 INFO: status: 22% (165514575872/751619276800), sparse 1% (9251196928), duration 5325, 180/167 MB/s
102: Nov 06 04:05:22 INFO: status: 23% (173071007744/751619276800), sparse 2% (15589883904), duration 5362, 204/32 MB/s
102: Nov 06 04:06:00 INFO: status: 24% (180589953024/751619276800), sparse 3% (23108812800), duration 5400, 197/0 MB/s
102: Nov 06 04:06:16 INFO: status: 25% (187954495488/751619276800), sparse 4% (30473342976), duration 5416, 460/0 MB/s
102: Nov 06 04:06:33 INFO: status: 26% (195781066752/751619276800), sparse 5% (38299897856), duration 5433, 460/0 MB/s
102: Nov 06 04:06:50 INFO: status: 27% (203112316928/751619276800), sparse 6% (45631135744), duration 5450, 431/0 MB/s
102: Nov 06 04:07:06 INFO: status: 28% (210465062912/751619276800), sparse 7% (52983865344), duration 5466, 459/0 MB/s
102: Nov 06 04:07:23 INFO: status: 29% (218271842304/751619276800), sparse 8% (60790632448), duration 5483, 459/0 MB/s
102: Nov 06 04:07:39 INFO: status: 30% (225609777152/751619276800), sparse 9% (68128550912), duration 5499, 458/0 MB/s
102: Nov 06 04:07:56 INFO: status: 31% (233411182592/751619276800), sparse 10% (75929944064), duration 5516, 458/0 MB/s
102: Nov 06 04:08:12 INFO: status: 32% (240773627904/751619276800), sparse 11% (83292372992), duration 5532, 460/0 MB/s
102: Nov 06 04:08:28 INFO: status: 33% (248158224384/751619276800), sparse 12% (90676957184), duration 5548, 461/0 MB/s
102: Nov 06 04:08:45 INFO: status: 34% (255987351552/751619276800), sparse 13% (98506067968), duration 5565, 460/0 MB/s
102: Nov 06 04:09:02 INFO: status: 35% (263328825344/751619276800), sparse 14% (105847529472), duration 5582, 431/0 MB/s
102: Nov 06 04:09:18 INFO: status: 36% (270682095616/751619276800), sparse 15% (113200783360), duration 5598, 459/0 MB/s
102: Nov 06 04:09:35 INFO: status: 37% (278526296064/751619276800), sparse 16% (121044971520), duration 5615, 461/0 MB/s
102: Nov 06 04:09:51 INFO: status: 38% (285859381248/751619276800), sparse 17% (128378040320), duration 5631, 458/0 MB/s
102: Nov 06 04:10:07 INFO: status: 39% (293216911360/751619276800), sparse 18% (135735554048), duration 5647, 459/0 MB/s
102: Nov 06 04:10:24 INFO: status: 40% (301054296064/751619276800), sparse 19% (143572738048), duration 5664, 461/0 MB/s
102: Nov 06 04:10:40 INFO: status: 41% (308475789312/751619276800), sparse 20% (150994219008), duration 5680, 463/0 MB/s
102: Nov 06 04:10:56 INFO: status: 42% (315838693376/751619276800), sparse 21% (158357106688), duration 5696, 460/0 MB/s
102: Nov 06 04:11:14 INFO: status: 43% (323645145088/751619276800), sparse 22% (166163357696), duration 5714, 433/0 MB/s
102: Nov 06 04:11:30 INFO: status: 44% (331021025280/751619276800), sparse 23% (173539221504), duration 5730, 460/0 MB/s
102: Nov 06 04:11:46 INFO: status: 45% (338368528384/751619276800), sparse 24% (180886712320), duration 5746, 459/0 MB/s
102: Nov 06 04:12:03 INFO: status: 46% (346177208320/751619276800), sparse 25% (188695375872), duration 5763, 459/0 MB/s
102: Nov 06 04:12:19 INFO: status: 47% (353518157824/751619276800), sparse 26% (196036313088), duration 5779, 458/0 MB/s
102: Nov 06 04:12:35 INFO: status: 48% (360875884544/751619276800), sparse 27% (203394023424), duration 5795, 459/0 MB/s
102: Nov 06 04:12:52 INFO: status: 49% (368697999360/751619276800), sparse 28% (211216125952), duration 5812, 460/0 MB/s
102: Nov 06 04:13:08 INFO: status: 50% (376064966656/751619276800), sparse 29% (218583076864), duration 5828, 460/0 MB/s
102: Nov 06 04:13:24 INFO: status: 51% (383419154432/751619276800), sparse 30% (225937252352), duration 5844, 459/0 MB/s
102: Nov 06 04:13:42 INFO: status: 52% (391193296896/751619276800), sparse 31% (233711378432), duration 5862, 431/0 MB/s
102: Nov 06 04:13:58 INFO: status: 53% (398526054400/751619276800), sparse 32% (241044123648), duration 5878, 458/0 MB/s
102: Nov 06 04:14:14 INFO: status: 54% (405875458048/751619276800), sparse 33% (248393515008), duration 5894, 459/0 MB/s
102: Nov 06 04:14:31 INFO: status: 55% (413696458752/751619276800), sparse 34% (256214499328), duration 5911, 460/0 MB/s
102: Nov 06 04:14:47 INFO: status: 56% (421030461440/751619276800), sparse 35% (263548297216), duration 5927, 458/0 MB/s
102: Nov 06 04:15:04 INFO: status: 57% (428846088192/751619276800), sparse 36% (271363911680), duration 5944, 459/0 MB/s
102: Nov 06 04:15:20 INFO: status: 58% (436190314496/751619276800), sparse 37% (278708121600), duration 5960, 459/0 MB/s
102: Nov 06 04:15:36 INFO: status: 59% (443546009600/751619276800), sparse 38% (286063804416), duration 5976, 459/0 MB/s
102: Nov 06 04:15:54 INFO: status: 60% (451343155200/751619276800), sparse 39% (293860933632), duration 5994, 433/0 MB/s
102: Nov 06 04:16:10 INFO: status: 61% (458689609728/751619276800), sparse 40% (301207375872), duration 6010, 459/0 MB/s
102: Nov 06 04:16:26 INFO: status: 62% (466064244736/751619276800), sparse 41% (308581994496), duration 6026, 460/0 MB/s
102: Nov 06 04:16:43 INFO: status: 63% (473911459840/751619276800), sparse 42% (316429197312), duration 6043, 461/0 MB/s
102: Nov 06 04:16:59 INFO: status: 64% (481269317632/751619276800), sparse 43% (323787038720), duration 6059, 459/0 MB/s
102: Nov 06 04:17:15 INFO: status: 65% (488641855488/751619276800), sparse 44% (331159564288), duration 6075, 460/0 MB/s
102: Nov 06 04:17:32 INFO: status: 66% (496452632576/751619276800), sparse 45% (338970324992), duration 6092, 459/0 MB/s
102: Nov 06 04:17:48 INFO: status: 67% (503797252096/751619276800), sparse 46% (346314932224), duration 6108, 459/0 MB/s
102: Nov 06 04:18:05 INFO: status: 68% (511152029696/751619276800), sparse 47% (353669693440), duration 6125, 432/0 MB/s
102: Nov 06 04:18:22 INFO: status: 69% (518953959424/751619276800), sparse 48% (361471610880), duration 6142, 458/0 MB/s
102: Nov 06 04:18:38 INFO: status: 70% (526311620608/751619276800), sparse 49% (368829255680), duration 6158, 459/0 MB/s
102: Nov 06 04:18:54 INFO: status: 71% (533652701184/751619276800), sparse 50% (376170323968), duration 6174, 458/0 MB/s
102: Nov 06 04:19:11 INFO: status: 72% (541475602432/751619276800), sparse 51% (383993208832), duration 6191, 460/0 MB/s
102: Nov 06 04:19:27 INFO: status: 73% (548821008384/751619276800), sparse 52% (391338602496), duration 6207, 459/0 MB/s
102: Nov 06 04:19:44 INFO: status: 74% (556621758464/751619276800), sparse 53% (399139336192), duration 6224, 458/0 MB/s
102: Nov 06 04:20:00 INFO: status: 75% (563972800512/751619276800), sparse 54% (406490365952), duration 6240, 459/0 MB/s
102: Nov 06 04:20:16 INFO: status: 76% (571324760064/751619276800), sparse 55% (413842309120), duration 6256, 459/0 MB/s
102: Nov 06 04:20:34 INFO: status: 77% (579142221824/751619276800), sparse 56% (421659758592), duration 6274, 434/0 MB/s
102: Nov 06 04:20:50 INFO: status: 78% (586478518272/751619276800), sparse 57% (428996038656), duration 6290, 458/0 MB/s
102: Nov 06 04:21:06 INFO: status: 79% (593841225728/751619276800), sparse 58% (436358733824), duration 6306, 460/0 MB/s
102: Nov 06 04:21:23 INFO: status: 80% (601653379072/751619276800), sparse 59% (444170870784), duration 6323, 459/0 MB/s
102: Nov 06 04:21:39 INFO: status: 81% (609011630080/751619276800), sparse 60% (451529109504), duration 6339, 459/0 MB/s
102: Nov 06 04:21:55 INFO: status: 82% (616372371456/751619276800), sparse 61% (458889834496), duration 6355, 460/0 MB/s
102: Nov 06 04:22:12 INFO: status: 83% (624207331328/751619276800), sparse 62% (466724782080), duration 6372, 460/0 MB/s
102: Nov 06 04:22:28 INFO: status: 84% (631564861440/751619276800), sparse 63% (474082295808), duration 6388, 459/0 MB/s
102: Nov 06 04:22:45 INFO: status: 85% (638922915840/751619276800), sparse 64% (481440337920), duration 6405, 432/0 MB/s
102: Nov 06 04:23:02 INFO: status: 86% (646714490880/751619276800), sparse 65% (489231896576), duration 6422, 458/0 MB/s
102: Nov 06 04:23:18 INFO: status: 87% (654063960064/751619276800), sparse 66% (496581353472), duration 6438, 459/0 MB/s
102: Nov 06 04:23:34 INFO: status: 88% (661426012160/751619276800), sparse 67% (503943393280), duration 6454, 460/0 MB/s
102: Nov 06 04:23:51 INFO: status: 89% (669250551808/751619276800), sparse 68% (511767916544), duration 6471, 460/0 MB/s
102: Nov 06 04:24:07 INFO: status: 90% (676624924672/751619276800), sparse 69% (519142273024), duration 6487, 460/0 MB/s
102: Nov 06 04:24:24 INFO: status: 91% (684411977728/751619276800), sparse 70% (526929313792), duration 6504, 458/0 MB/s
102: Nov 06 04:24:40 INFO: status: 92% (691756072960/751619276800), sparse 71% (534273392640), duration 6520, 459/0 MB/s
102: Nov 06 04:24:57 INFO: status: 93% (699124482048/751619276800), sparse 72% (541641789440), duration 6537, 433/0 MB/s
102: Nov 06 04:25:14 INFO: status: 94% (706924380160/751619276800), sparse 73% (549441671168), duration 6554, 458/0 MB/s
102: Nov 06 04:25:30 INFO: status: 95% (714294099968/751619276800), sparse 74% (556811378688), duration 6570, 460/0 MB/s
102: Nov 06 04:25:46 INFO: status: 96% (721639440384/751619276800), sparse 75% (564156702720), duration 6586, 459/0 MB/s
102: Nov 06 04:26:03 INFO: status: 97% (729459261440/751619276800), sparse 76% (571976511488), duration 6603, 459/0 MB/s
102: Nov 06 04:26:19 INFO: status: 98% (736799490048/751619276800), sparse 77% (579316723712), duration 6619, 458/0 MB/s
102: Nov 06 04:26:35 INFO: status: 99% (744174977024/751619276800), sparse 78% (586692198400), duration 6635, 460/0 MB/s
102: Nov 06 04:26:52 INFO: status: 100% (751619276800/751619276800), sparse 79% (594136481792), duration 6652, 437/0 MB/s
102: Nov 06 04:26:52 INFO: transferred 751619 MB in 6652 seconds (112 MB/s)
102: Nov 06 04:26:52 INFO: archive file size: 106.51GB
102: Nov 06 04:26:52 INFO: delete old backup '/mnt/pve/BACKUP/dump/vzdump-qemu-102-2013_10_30-02_39_39.vma.lzo'
102: Nov 06 04:27:19 INFO: Finished Backup of VM 102 (01:51:20)


After Upgrade:


102: Nov 20 00:09:24 INFO: Starting Backup of VM 102 (qemu)
102: Nov 20 00:09:24 INFO: status = running
102: Nov 20 00:09:24 INFO: update VM 102: -lock backup
102: Nov 20 00:09:24 INFO: backup mode: snapshot
102: Nov 20 00:09:24 INFO: ionice priority: 7
102: Nov 20 00:09:24 INFO: creating archive '/mnt/pve/BACKUP/dump/vzdump-qemu-102-2013_11_20-00_09_24.vma.lzo'
102: Nov 20 00:09:24 INFO: started backup task 'de8482aa-c045-44bc-a1eb-cb02a7353835'
102: Nov 20 00:09:27 INFO: status: 0% (776142848/751619276800), sparse 0% (436387840), duration 3, 258/113 MB/s
102: Nov 20 00:12:25 INFO: status: 1% (7742816256/751619276800), sparse 0% (2138759168), duration 181, 39/29 MB/s
102: Nov 20 00:15:38 INFO: status: 2% (15044706304/751619276800), sparse 0% (2943410176), duration 374, 37/33 MB/s
102: Nov 20 00:24:58 INFO: status: 3% (22608871424/751619276800), sparse 0% (3961868288), duration 934, 13/11 MB/s
102: Nov 20 00:31:38 INFO: status: 4% (30168055808/751619276800), sparse 0% (5005381632), duration 1334, 18/16 MB/s
102: Nov 20 00:44:03 INFO: status: 5% (37661179904/751619276800), sparse 0% (5138141184), duration 2079, 10/9 MB/s
102: Nov 20 00:50:42 INFO: status: 6% (45286162432/751619276800), sparse 0% (5272760320), duration 2478, 19/18 MB/s
102: Nov 20 01:07:55 INFO: status: 7% (52697432064/751619276800), sparse 0% (5389123584), duration 3511, 7/7 MB/s
102: Nov 20 01:18:54 INFO: status: 8% (60288073728/751619276800), sparse 0% (5727084544), duration 4170, 11/11 MB/s
102: Nov 20 01:32:41 INFO: status: 9% (67746791424/751619276800), sparse 0% (5828067328), duration 4997, 9/8 MB/s
102: Nov 20 01:42:08 INFO: status: 10% (75189321728/751619276800), sparse 0% (6003773440), duration 5564, 13/12 MB/s
102: Nov 20 01:53:54 INFO: status: 11% (82715475968/751619276800), sparse 0% (6213480448), duration 6270, 10/10 MB/s
102: Nov 20 02:00:23 INFO: status: 12% (90473496576/751619276800), sparse 0% (6482096128), duration 6659, 19/19 MB/s
102: Nov 20 02:14:49 INFO: status: 13% (97760378880/751619276800), sparse 0% (6755700736), duration 7525, 8/8 MB/s
102: Nov 20 02:23:04 INFO: status: 14% (105313861632/751619276800), sparse 0% (7146541056), duration 8020, 15/14 MB/s
102: Nov 20 02:35:28 INFO: status: 15% (112755539968/751619276800), sparse 0% (7420735488), duration 8764, 10/9 MB/s
102: Nov 20 02:45:49 INFO: status: 16% (120307449856/751619276800), sparse 1% (8068493312), duration 9385, 12/11 MB/s
102: Nov 20 02:54:50 INFO: status: 17% (127825149952/751619276800), sparse 1% (8780845056), duration 9926, 13/12 MB/s
102: Nov 20 03:01:21 INFO: status: 18% (135314931712/751619276800), sparse 1% (10747588608), duration 10317, 19/14 MB/s
102: Nov 20 03:03:23 INFO: status: 19% (142869790720/751619276800), sparse 2% (18294382592), duration 10439, 61/0 MB/s
102: Nov 20 03:03:31 INFO: status: 20% (150453485568/751619276800), sparse 3% (25868677120), duration 10447, 947/1 MB/s
102: Nov 20 03:03:39 INFO: status: 21% (158034690048/751619276800), sparse 4% (33441824768), duration 10455, 947/1 MB/s
102: Nov 20 03:03:47 INFO: status: 22% (165632016384/751619276800), sparse 5% (41029722112), duration 10463, 949/1 MB/s
102: Nov 20 03:04:32 INFO: status: 23% (173515014144/751619276800), sparse 6% (48359108608), duration 10508, 175/12 MB/s
102: Nov 20 03:04:40 INFO: status: 24% (181139668992/751619276800), sparse 7% (55972524032), duration 10516, 953/1 MB/s
102: Nov 20 03:04:48 INFO: status: 25% (188770156544/751619276800), sparse 8% (63595618304), duration 10524, 953/0 MB/s
102: Nov 20 03:04:55 INFO: status: 26% (195451813888/751619276800), sparse 9% (70267211776), duration 10531, 954/1 MB/s
102: Nov 20 03:05:03 INFO: status: 27% (203096588288/751619276800), sparse 10% (77903933440), duration 10539, 955/1 MB/s
102: Nov 20 03:05:11 INFO: status: 28% (210739396608/751619276800), sparse 11% (85536669696), duration 10547, 955/1 MB/s
102: Nov 20 03:05:19 INFO: status: 29% (218387709952/751619276800), sparse 12% (93176930304), duration 10555, 956/1 MB/s
102: Nov 20 03:05:27 INFO: status: 30% (226012758016/751619276800), sparse 13% (100791910400), duration 10563, 953/1 MB/s
102: Nov 20 03:05:35 INFO: status: 31% (233655042048/751619276800), sparse 14% (108426805248), duration 10571, 955/0 MB/s
102: Nov 20 03:05:43 INFO: status: 32% (241265541120/751619276800), sparse 15% (116027228160), duration 10579, 951/1 MB/s
102: Nov 20 03:05:51 INFO: status: 33% (248895045632/751619276800), sparse 16% (123649351680), duration 10587, 953/0 MB/s
102: Nov 20 03:05:58 INFO: status: 34% (255606063104/751619276800), sparse 17% (130359701504), duration 10594, 958/0 MB/s
102: Nov 20 03:06:06 INFO: status: 35% (263249395712/751619276800), sparse 18% (138003021824), duration 10602, 955/0 MB/s
102: Nov 20 03:06:15 INFO: status: 36% (270910095360/751619276800), sparse 19% (145663696896), duration 10611, 851/0 MB/s
102: Nov 20 03:06:23 INFO: status: 37% (278566404096/751619276800), sparse 20% (153319993344), duration 10619, 957/0 MB/s
102: Nov 20 03:06:31 INFO: status: 38% (286165630976/751619276800), sparse 21% (160919199744), duration 10627, 949/0 MB/s
102: Nov 20 03:06:39 INFO: status: 39% (293813092352/751619276800), sparse 22% (168566460416), duration 10635, 955/0 MB/s
102: Nov 20 03:06:47 INFO: status: 40% (301451968512/751619276800), sparse 23% (176205312000), duration 10643, 954/0 MB/s
102: Nov 20 03:06:55 INFO: status: 41% (309110308864/751619276800), sparse 24% (183863640064), duration 10651, 957/0 MB/s
102: Nov 20 03:07:02 INFO: status: 42% (315814903808/751619276800), sparse 25% (190568218624), duration 10658, 957/0 MB/s
102: Nov 20 03:07:10 INFO: status: 43% (323462234112/751619276800), sparse 26% (198215348224), duration 10666, 955/0 MB/s
102: Nov 20 03:07:25 INFO: status: 44% (331378851840/751619276800), sparse 27% (204650774528), duration 10681, 527/98 MB/s
102: Nov 20 03:09:23 INFO: status: 45% (338603016192/751619276800), sparse 28% (211870203904), duration 10799, 61/0 MB/s
102: Nov 20 03:09:31 INFO: status: 46% (346265026560/751619276800), sparse 29% (219532197888), duration 10807, 957/0 MB/s
102: Nov 20 03:09:39 INFO: status: 47% (353902002176/751619276800), sparse 30% (227164962816), duration 10815, 954/0 MB/s
102: Nov 20 03:09:47 INFO: status: 48% (361543761920/751619276800), sparse 31% (234806697984), duration 10823, 955/0 MB/s
102: Nov 20 03:09:55 INFO: status: 49% (369181327360/751619276800), sparse 32% (242444251136), duration 10831, 954/0 MB/s
102: Nov 20 03:10:02 INFO: status: 50% (375889002496/751619276800), sparse 33% (249151909888), duration 10838, 958/0 MB/s
102: Nov 20 03:10:10 INFO: status: 51% (383566741504/751619276800), sparse 34% (256829632512), duration 10846, 959/0 MB/s
102: Nov 20 03:10:18 INFO: status: 52% (391220428800/751619276800), sparse 35% (264483303424), duration 10854, 956/0 MB/s
102: Nov 20 03:10:26 INFO: status: 53% (398885191680/751619276800), sparse 36% (272148045824), duration 10862, 958/0 MB/s
102: Nov 20 03:10:34 INFO: status: 54% (406544121856/751619276800), sparse 37% (279806959616), duration 10870, 957/0 MB/s
102: Nov 20 03:10:44 INFO: status: 55% (414176378880/751619276800), sparse 38% (287335088128), duration 10880, 763/10 MB/s
102: Nov 20 03:10:53 INFO: status: 56% (421630640128/751619276800), sparse 39% (294789070848), duration 10889, 828/0 MB/s
102: Nov 20 03:11:01 INFO: status: 57% (429287473152/751619276800), sparse 40% (302445891584), duration 10897, 957/0 MB/s
102: Nov 20 03:11:08 INFO: status: 58% (435962576896/751619276800), sparse 41% (309120978944), duration 10904, 953/0 MB/s
102: Nov 20 03:11:16 INFO: status: 59% (443613904896/751619276800), sparse 42% (316772294656), duration 10912, 956/0 MB/s
102: Nov 20 03:11:24 INFO: status: 60% (451254353920/751619276800), sparse 43% (324412727296), duration 10920, 955/0 MB/s
102: Nov 20 03:11:32 INFO: status: 61% (458898472960/751619276800), sparse 44% (332056834048), duration 10928, 955/0 MB/s
102: Nov 20 03:11:40 INFO: status: 62% (466552487936/751619276800), sparse 45% (339710832640), duration 10936, 956/0 MB/s
102: Nov 20 03:11:48 INFO: status: 63% (474223869952/751619276800), sparse 46% (347382198272), duration 10944, 958/0 MB/s
102: Nov 20 03:11:56 INFO: status: 64% (481885093888/751619276800), sparse 47% (355043405824), duration 10952, 957/0 MB/s
102: Nov 20 03:12:04 INFO: status: 65% (488591982592/751619276800), sparse 48% (361750282240), duration 10960, 838/0 MB/s
102: Nov 20 03:12:12 INFO: status: 66% (496197042176/751619276800), sparse 49% (369355325440), duration 10968, 950/0 MB/s
102: Nov 20 03:12:20 INFO: status: 67% (503868293120/751619276800), sparse 50% (377026564096), duration 10976, 958/0 MB/s
102: Nov 20 03:12:28 INFO: status: 68% (511535546368/751619276800), sparse 51% (384693796864), duration 10984, 958/0 MB/s
102: Nov 20 03:12:36 INFO: status: 69% (519158628352/751619276800), sparse 52% (392316862464), duration 10992, 952/0 MB/s
102: Nov 20 03:12:44 INFO: status: 70% (526837088256/751619276800), sparse 53% (399995305984), duration 11000, 959/0 MB/s
102: Nov 20 03:12:52 INFO: status: 71% (534500474880/751619276800), sparse 54% (407658680320), duration 11008, 957/0 MB/s
102: Nov 20 03:12:59 INFO: status: 72% (541207691264/751619276800), sparse 55% (414365880320), duration 11015, 958/0 MB/s
102: Nov 20 03:13:07 INFO: status: 73% (548881235968/751619276800), sparse 56% (422039412736), duration 11023, 959/0 MB/s
102: Nov 20 03:13:15 INFO: status: 74% (556557860864/751619276800), sparse 57% (429716021248), duration 11031, 959/0 MB/s
102: Nov 20 03:13:23 INFO: status: 75% (564232454144/751619276800), sparse 58% (437390602240), duration 11039, 959/0 MB/s
102: Nov 20 03:13:31 INFO: status: 76% (571904032768/751619276800), sparse 59% (445062160384), duration 11047, 958/0 MB/s
102: Nov 20 03:13:39 INFO: status: 77% (579558113280/751619276800), sparse 60% (452716228608), duration 11055, 956/0 MB/s
102: Nov 20 03:13:46 INFO: status: 78% (586266574848/751619276800), sparse 61% (459424673792), duration 11062, 958/0 MB/s
102: Nov 20 03:13:54 INFO: status: 79% (593920589824/751619276800), sparse 62% (467078676480), duration 11070, 956/0 MB/s
102: Nov 20 03:14:02 INFO: status: 80% (601584107520/751619276800), sparse 63% (474742177792), duration 11078, 957/0 MB/s
102: Nov 20 03:14:10 INFO: status: 81% (609249722368/751619276800), sparse 64% (482407780352), duration 11086, 958/0 MB/s
102: Nov 20 03:14:18 INFO: status: 82% (616922021888/751619276800), sparse 65% (490080059392), duration 11094, 959/0 MB/s
102: Nov 20 03:14:26 INFO: status: 83% (624596484096/751619276800), sparse 66% (497754509312), duration 11102, 959/0 MB/s
102: Nov 20 03:14:34 INFO: status: 84% (632197742592/751619276800), sparse 67% (505355751424), duration 11110, 950/0 MB/s
102: Nov 20 03:14:42 INFO: status: 85% (638914658304/751619276800), sparse 68% (512072654848), duration 11118, 839/0 MB/s
102: Nov 20 03:14:50 INFO: status: 86% (646580600832/751619276800), sparse 69% (519738580992), duration 11126, 958/0 MB/s
102: Nov 20 03:14:58 INFO: status: 87% (654249230336/751619276800), sparse 70% (527407198208), duration 11134, 958/0 MB/s
102: Nov 20 03:15:06 INFO: status: 88% (661930901504/751619276800), sparse 71% (535088852992), duration 11142, 960/0 MB/s
102: Nov 20 03:15:14 INFO: status: 89% (669591666688/751619276800), sparse 72% (542749601792), duration 11150, 957/0 MB/s
102: Nov 20 03:15:22 INFO: status: 90% (677255380992/751619276800), sparse 73% (550413299712), duration 11158, 957/0 MB/s
102: Nov 20 03:15:30 INFO: status: 91% (684922961920/751619276800), sparse 74% (558080868352), duration 11166, 958/0 MB/s
102: Nov 20 03:15:37 INFO: status: 92% (691580960768/751619276800), sparse 75% (564738850816), duration 11173, 951/0 MB/s
102: Nov 20 03:15:45 INFO: status: 93% (699245330432/751619276800), sparse 76% (572403208192), duration 11181, 958/0 MB/s
102: Nov 20 03:15:53 INFO: status: 94% (706916515840/751619276800), sparse 77% (580074377216), duration 11189, 958/0 MB/s
102: Nov 20 03:16:01 INFO: status: 95% (714555916288/751619276800), sparse 78% (587713761280), duration 11197, 954/0 MB/s
102: Nov 20 03:16:09 INFO: status: 96% (722226446336/751619276800), sparse 79% (595384274944), duration 11205, 958/0 MB/s
102: Nov 20 03:16:17 INFO: status: 97% (729851756544/751619276800), sparse 80% (603009572864), duration 11213, 953/0 MB/s
102: Nov 20 03:16:25 INFO: status: 98% (737500659712/751619276800), sparse 81% (610658459648), duration 11221, 956/0 MB/s
102: Nov 20 03:16:32 INFO: status: 99% (744208465920/751619276800), sparse 82% (617366253568), duration 11228, 958/0 MB/s
102: Nov 20 03:16:40 INFO: status: 100% (751619276800/751619276800), sparse 83% (624777048064), duration 11236, 926/0 MB/s
102: Nov 20 03:16:40 INFO: transferred 751619 MB in 11236 seconds (66 MB/s)
102: Nov 20 03:16:40 INFO: archive file size: 106.85GB
102: Nov 20 03:16:42 INFO: Finished Backup of VM 102 (03:07:18)



So i tried to backup to an CIFS Share. After i started the Backup, speed was great. In the middle of the backup i received a Message "no space on the device left". There is a big difference between PVE-Gui and Console




A "df" at console shows:

Filesystem Size Used Avail Use% Mounted on

//192.168.10.202/proxbackup 5.4T 1.6T 3.8T 30% /media/backup

 

[dietmar]

here is a big difference between PVE-Gui and Console

That graph looks like the statistics of the root drive?

 

[mgerste]

That graph looks like the statistics of the root drive?

Hi Dietmar!

I don't think so. The share "proxbackup" is mounted as /media/proxbackup


In storage for testing it's labeld as sbackup


This should be the free space on the share /media/proxbackup

 

[dietmar]

I don't think so. The share "proxbackup" is mounted as /media/proxbackup

That is storage 'SBACKUP', and that is just a directory on root (nothing mounted).

 

[mgerste]

That is storage 'SBACKUP', and that is just a directory on root (nothing mounted).

Sorry! My fault. But why is NFS Performance so bad, since upgrading to 3.1?

 

[Nachtfalke]

Hi,

I could not follow you on the complete thread because I am no linux expert but it looks like I have a smiliar "problem" with newer ProxmoxVe 3.x versions.

I am using the "Directory" storage model vor Backups:
http://pve.proxmox.com/wiki/Storage_Model#Directory

I am running a small Buffalo LinkStation which should store the backups of my VM I am doing every weekend at night. No need for high performance and no problem if the VM is down for some hours.

I am not sure when it change but on early versions of ProxmoxVE3.0 I got a backup of my VM (350GB) within ~6h. At the moment I am running the latest ProxmoxVE 3.1 version and I cannot get BACKUPS to work. It starts and then copies the first 0-5% and then after 24h or more it still stuck at 5%. I then stopp the VM BACKUP process from GUI and unlock the VM from shell.

So why I am posting this here and to make some connection to the screenshots:
When I go to the PVE GUI and click on the mounted directory of my NAS I see strange and for me completely useless diskusage and size. Further I cannot see the content of this directory. After doing a

mount -a

on shell I got the correct size on PVE GUI and I can see the in the past backuped VMs.

But I still cannot do any BACKUPS.

The NAS has gigabit ethernet like the switches and the proxmox NIC. Of course I know that this small NAS ist not fast it should do the job over night but it shouldn't stuck at 5% and never finishes.

My two VMs are running Windows 7 x64 and Windows Server 2008R2 x64
and the VMs are running as KVM.
The backup mode is SNAPSHOT (I tried with SUSPEND and with STOP)
I added GPO on the machines to allow to shutdown the machine even if nobody is logged on.
I manually shutdown the VMs and tried to backup but it always stuck at 5%.

So there must be something that changed but I don't know what and where but it looks like something between PVE3.0 and PVE3.1 because I could do 5 weekly updates ob both VMs with same hardware, network and NAS with earlier versions.

So I would appreciate any kind of help and I would do some tests if needed.

Thank you very much!

 

[snowman66]

In my case backup to samba share (directory) begin to fail after 11th October (i'm using enterprise repo.).
So I bought new hardware, install nas4free and now backup runs with NFS mount without problems.

 

[Nachtfalke]

Hmm, 11th october could be the point.
I shedulded the backups and my last working weekly backup was on 11th october 6am.
The last backup I have is from 20th september 6am.

Of course I cannot say for sure when I exactly updated ProxmoxVE and what version it has but probably I updated sometime after 11th october and before 18th october.

Of course buying a ne storage solution should not be the goal independent if it is a fast or a slow method. If it worked before it shouldn't not work afterward, right?

 

[Nachtfalke]

I did some further "tests" and tried with backup method SNAPSHOT and tried with LZO and with GZIP and without any compression. The result is the same.
While PVE GUI tells me it is doing some backup and it stuck on xx% I went to SSH and checked the folder. There I could find a .dat, .log and .tmp which stuck at the same size.

Unfortunately the aborted backups from the past where still there and I deleted these files. I tried with two different ProxmoxVE machines with different hardware but both using the same storage destination and the same PVE 3.1 version.

 

[snowman66]

Backuping openvz containers should work.

I also noticed that i could not copy larger iso file (>1GB) to cifs mount/folder. Please test this.

for example cp /var/lib/vz/template/iso/someISOfile.sio /mnt/smb

You could try to change elevator to cfq, default is deadline...

echo cfq > /sys/block/sda/queue/scheduler

 

[dietmar]

Also, does it help if you mount the cifs share with 'wsize=57344'?

 

[Nachtfalke]

Hi,

I tried your suggestions but not everythings seems to work.

So first I modified the /etc/fstab with Dietmars suggestions and now it looks like this:

//172.17.240.150/VM300-BACKUP /VM300-BACKUP cifs rsize=16384,wsize=57344,username=admin,password=MyVerySecretPassword.,domain=hpa 0 0
//172.17.240.150/VM301-BACKUP /VM301-BACKUP cifs rsize=16384,wsize=57344,username=admin,password=MyVerySecretPassword.,domain=hpa 0 0

I rebooted the system.

Then I tried a backup of one VM with type SNAPSHOT, GZIP and it finished. The backup has a size of 850MB. Beofore the changes it did not finish. After that I did the same changes on another PVE installation and rebootet PVE.

Then I tried a backup of one VM with type SNAPSHOT, GZIP and it started but stuck at 4% for ~45min. Then I stopped the backup:

INFO: starting new backup job: vzdump 300 --remove 0 --mode snapshot --compress gzip --storage VM300-BACKUP --node ProxmoxVE3
INFO: Starting Backup of VM 300 (qemu)
INFO: status = running
INFO: update VM 300: -lock backup
INFO: backup mode: snapshot
INFO: ionice priority: 7
INFO: snapshots found (not included into backup)
INFO: creating archive '/VM300-BACKUP/dump/vzdump-qemu-300-2013_11_24-18_52_31.vma.gz'
INFO: started backup task '5b39825c-74a2-4b99-96f9-08d30aaa53c5'
INFO: status: 0% (121241600/107374182400), sparse 0% (89300992), duration 3, 40/10 MB/s
INFO: status: 1% (1078722560/107374182400), sparse 0% (99758080), duration 93, 10/10 MB/s
INFO: status: 2% (2156003328/107374182400), sparse 0% (109142016), duration 164, 15/15 MB/s
INFO: status: 3% (3231121408/107374182400), sparse 0% (128319488), duration 235, 15/14 MB/s
INFO: status: 4% (4295294976/107374182400), sparse 0% (136269824), duration 311, 14/13 MB/s

On both machines I did:

cat /sys/block/sda/queue/scheduler

and it showed me [deadline] enabled so I changed it accoriding to your suggestions to [cfq].
I did NO reboot and hopefully this change will take effect without reboot and started a backup. I will give it one hour more time to do any progress but at the moment it stucks.

Thank you for your help but I need further suggestions to do some more tests and get this working again


--- edit ---


Need to add what happend when copying a .RAW file with 1.8GB using the "cp" command to the cifs share. This worked. I did this before changing the sheduler to [cfq]


--- edit2 ---


Ok, after waiting for ~1h the status did not change. The bytes on the cifs folder do not increase. This is where it stops:

drwxr-xr-x 1 root root 0 Nov 24 19:58 vzdump-qemu-201-2013_11_24-19_58_55.tmp
-rwxr-xr-x 1 root root 2454605824 Nov 24 19:58 vzdump-qemu-201-2013_11_24-19_58_55.vma.dat

 

[mgerste]

Hi Dietmar!

Is there anything which has changed in NFS Settings since Proxmox 3.0?

 

[snowman66]

Are you using CIFS (windows share) or NFS (Linux)? This two things are not the same.

cat /proc/mounts

 

[mgerste]

Are you using CIFS (windows share) or NFS (Linux)? This two things are not the same.

cat /proc/mounts

Hi!

I know the difference between CIFS and NFS. With 3.0 my backup was written to an nfs share. Since 3.1 vzdump performance ist very bad so i tried to make backups to an cifs share which was not better.

 

[Nachtfalke]

Hi,

any news about the performance issue with CIFS? Weekend starts and I can do some more tests - if you have any suggestions ;-)

--- edit ---

Did another test and tried to do a backup to my windows 7 PC with cifs which has more performance than my small Buffalo NAS but the backup stop like all the tries before.

 

[Nachtfalke]

Hello everybody,

I did some more tests on one server. I want to backup a Windows Server 2008R2 running my WSUS. I tried the backup to my NAS first in SNAPSHOT mode and it started with ~20-30% CPU usage and 2-3% IO Delay. When reching 1% backup state the IO Delay dramatically increased up to ~30% and the backup modus does not go on any further.

Perhaps this could be the reason? Somewhere on the forum I read that this could happen because of a bug somewhere the underlying system of proxmox when there should be backuped a VM/folder which contains many many files. Perhaps WSUS stores that many files into one folder - don't know. The suggestion was to stop the machine and use STOP as backup mode. I tried that and got the same error.

Hopefully someone could help me on this. I really really need the BACKUP and RESTORE functionality from/to my external NAS which only can do cifs. I do not have any other possibility to do the backups

 

[blackpaw]

I found that sparse provisioning was killing my backups, converting to thin provisioned made a huge difference

Sent from my MT27i using Tapatalk

 

[Nachtfalke]

I found that sparse provisioning was killing my backups, converting to thin provisioned made a huge difference
Sent from my MT27i using Tapatalk

Sorry for that stupid question:
What do you mean iwth "sparse provisioning" and "thin provisioning"? Do you mean the compression when doing backups?

--- edit ---

So just to make sure you did not meant the compression I tried on another PVE node with other hardware and other VM with:
- no compression
- LZO
- GZIP
The both VMs tested (One Windows Server 2008R2 with WSUS - different than my VM from previous post) and one with Windows 7. Both VMs were shutdown and the backup mode was STOP.

No success. Reaching a point in the backup process and IO delay dramatically increases.

 

[blackpaw]

Sparse is the default on px 3.1, the vm's disks are preallocated but only take up as much space as they actually use. Which worked on my nas - 512gb hd only occupied 20gb but the backup seemed to process the entire 512gb. Converting to thin provisioned changed backups from 3 hours to 20 minutes.

Sent from my MT27i using Tapatalk