G
gorancv
Guest
There is a problem with snapshot backup with Windows 2008 Server guest on KVM.
Host system is Fujitsu Server with Intel Xeon 3220, 8 GB RAM, 4x500GB SATA drives with 3ware 9690SA controller.
Storage and network drivers for guest are virtio.
When backup is started the backup process is working without any registered problem, but guest system crash with unresponsive network or with blue screen in Windows.
Here are some info about system:
system performance:
vzdump log:
syslog during backup process:
Host system is Fujitsu Server with Intel Xeon 3220, 8 GB RAM, 4x500GB SATA drives with 3ware 9690SA controller.
Storage and network drivers for guest are virtio.
When backup is started the backup process is working without any registered problem, but guest system crash with unresponsive network or with blue screen in Windows.
Here are some info about system:
Code:
pve-manager: 1.5-10 (pve-manager/1.5/4822)
running kernel: 2.6.18-2-pve
proxmox-ve-2.6.18: 1.5-5
pve-kernel-2.6.18-2-pve: 2.6.18-5
qemu-server: 1.1-16
pve-firmware: 1.0-5
libpve-storage-perl: 1.0-13
vncterm: 0.9-2
vzctl: 3.0.23-1pve11
vzdump: 1.2-5
vzprocps: 2.0.11-1dso2
vzquota: 3.0.11-1
pve-qemu-kvm-2.6.18: 0.9.1-5
Code:
pve1:~# pveperf
CPU BOGOMIPS: 19151.98
REGEX/SECOND: 786344
HD SIZE: 61.27 GB (/dev/pve/root)
BUFFERED READS: 208.37 MB/sec
AVERAGE SEEK TIME: 9.69 ms
FSYNCS/SECOND: 2497.71
DNS EXT: 91.53 ms
DNS INT: 0.65 ms (localdns.local)
Code:
Jul 13 02:30:02 INFO: Starting Backup of VM 201 (qemu)
Jul 13 02:30:02 INFO: running
Jul 13 02:30:02 INFO: status = running
Jul 13 02:30:03 INFO: backup mode: snapshot
Jul 13 02:30:03 INFO: bandwidth limit: 10240 KB/s
Jul 13 02:30:03 INFO: suspend vm to make snapshot
Jul 13 02:30:03 INFO: Logical volume "vzsnap-pve1-0" created
Jul 13 02:30:04 INFO: Logical volume "vzsnap-pve1-0" created
Jul 13 02:30:04 INFO: resume vm
Jul 13 02:30:04 INFO: vm is online again after 1 seconds
Jul 13 02:30:04 INFO: creating archive '/var/lib/backup/vzdump-qemu-201-2010_07_13-02_30_02.tgz'
Jul 13 02:30:04 INFO: adding '/var/lib/backup/vzdump-qemu-201-2010_07_13-02_30_02.tmp/qemu-server.conf' to archive ('qemu-server.conf')
Jul 13 02:30:04 INFO: adding '/mnt/vzsnap0/images/201/vm-201-disk-1.raw' to archive ('vm-disk-virtio0.raw')
Jul 13 02:47:18 INFO: adding '/mnt/vzsnap1/images/201/vm-201-disk-1.raw' to archive ('vm-disk-virtio1.raw')
Jul 13 03:00:49 INFO: Total bytes written: 11155355136 (5.77 MiB/s)
Jul 13 03:00:49 INFO: archive file size: 4.19GB
Jul 13 03:00:49 INFO: delete old backup '/var/lib/backup/vzdump-qemu-201-2010_07_12-02_30_02.tgz'
Jul 13 03:00:58 INFO: Logical volume "vzsnap-pve1-0" successfully removed
Jul 13 03:00:59 INFO: Logical volume "vzsnap-pve1-0" successfully removed
Jul 13 03:00:59 INFO: Finished Backup of VM 201 (00:30:57)
Code:
Jul 13 02:30:02 pve1 vzdump[11095]: INFO: starting new backup job: vzdump --quiet --snapshot --compress --storage
backup 201
Jul 13 02:30:02 pve1 vzdump[11095]: INFO: Starting Backup of VM 201 (qemu)
Jul 13 02:30:03 pve1 qm[11143]: VM 201 suspend
Jul 13 02:30:04 pve1 kernel: kjournald starting. Commit interval 5 seconds
Jul 13 02:30:04 pve1 kernel: EXT3 FS on dm-5, internal journal
Jul 13 02:30:04 pve1 kernel: EXT3-fs: mounted filesystem with ordered data mode.
Jul 13 02:30:04 pve1 kernel: kjournald starting. Commit interval 5 seconds
Jul 13 02:30:04 pve1 kernel: EXT3 FS on dm-8, internal journal
Jul 13 02:30:04 pve1 kernel: EXT3-fs: mounted filesystem with ordered data mode.
Jul 13 02:30:04 pve1 qm[11219]: VM 201 resume
Jul 13 02:30:52 pve1 kernel: vmtar invoked oom-killer: gfp_mask=0x201d2, order=0, oomkilladj=0
Jul 13 02:30:52 pve1 kernel:
Jul 13 02:30:52 pve1 kernel: Call Trace:
Jul 13 02:30:52 pve1 kernel: [<ffffffff800cf96f>] out_of_memory+0x9f/0x25f
Jul 13 02:30:52 pve1 kernel: [<ffffffff8000eda5>] __alloc_pages+0x249/0x334
Jul 13 02:30:52 pve1 kernel: [<ffffffff8001283e>] __do_page_cache_readahead+0x95/0x178
Jul 13 02:30:52 pve1 kernel: [<ffffffff800330d2>] blockable_page_cache_readahead+0x53/0xb2
Jul 13 02:30:52 pve1 kernel: [<ffffffff8002fb94>] make_ahead_window+0x82/0x9e
Jul 13 02:30:52 pve1 kernel: [<ffffffff80013b2d>] page_cache_readahead+0x17f/0x1af
Jul 13 02:30:52 pve1 kernel: [<ffffffff8000b89b>] do_generic_mapping_read+0xc7/0x353
Jul 13 02:30:52 pve1 kernel: [<ffffffff800d4a92>] zone_statistics+0x3e/0x6d
Jul 13 02:30:52 pve1 kernel: [<ffffffff8000c860>] file_read_actor+0x0/0x141
Jul 13 02:30:52 pve1 kernel: [<ffffffff8000bc73>] __generic_file_aio_read+0x14c/0x198
Jul 13 02:30:52 pve1 kernel: [<ffffffff80016b78>] generic_file_aio_read+0x34/0x39
Jul 13 02:30:52 pve1 kernel: [<ffffffff8000c573>] do_sync_read+0xc7/0x104
Jul 13 02:30:52 pve1 kernel: [<ffffffff800a0cca>] autoremove_wake_function+0x0/0x2e
Jul 13 02:30:52 pve1 kernel: [<ffffffff800634b8>] __sched_text_start+0x118/0xfc4
Jul 13 02:30:52 pve1 kernel: [<ffffffff8000ae21>] vfs_read+0xaa/0x150
Jul 13 02:30:52 pve1 kernel: [<ffffffff8000b03e>] fget_light+0x20/0x7c
Jul 13 02:30:52 pve1 kernel: [<ffffffff80011521>] sys_read+0x49/0xc2
Jul 13 02:30:52 pve1 kernel: [<ffffffff80060166>] system_call+0x7e/0x83
Jul 13 02:30:52 pve1 kernel:
...
Jul 13 02:30:52 pve1 kernel: Node 0 HighMem: empty
Jul 13 02:30:52 pve1 kernel: 886738 pagecache pages
Jul 13 02:30:52 pve1 kernel: Swap cache: add 1269301, delete 1058626, find 188360/216221, race 0+313+1
Jul 13 02:30:52 pve1 kernel: Free swap = 0kB
Jul 13 02:30:52 pve1 kernel: Total swap = 4194296kB
Jul 13 02:30:52 pve1 kernel: Free swap: 0kB
Jul 13 02:30:52 pve1 kernel: 2621440 pages of RAM
Jul 13 02:30:52 pve1 kernel: 586731 reserved pages
Jul 13 02:30:52 pve1 kernel: 384516 pages shared
Jul 13 02:30:52 pve1 kernel: 210348 pages swap cached
Jul 13 02:30:52 pve1 kernel: Top 10 caches:
Jul 13 02:30:52 pve1 kernel: inode_cache : size 1212416 objsize 608
Jul 13 02:30:52 pve1 kernel: dentry_cache : size 2678784 objsize 248
Jul 13 02:30:52 pve1 kernel: radix_tree_node : size 12828672 objsize 536
Jul 13 02:30:52 pve1 kernel: buffer_head : size 5128192 objsize 96
Jul 13 02:30:52 pve1 kernel: size-2048 : size 1464320 objsize 2048
Jul 13 02:30:52 pve1 kernel: kvm_mmu_page_header : size 2945024 objsize 176
Jul 13 02:30:52 pve1 kernel: kvm_rmap_desc : size 2465792 objsize 40
Jul 13 02:30:52 pve1 kernel: shmem_inode_cache : size 4603904 objsize 824
Jul 13 02:30:52 pve1 kernel: ext3_inode_cache : size 1427712 objsize 808
Jul 13 02:30:52 pve1 kernel: page_beancounter : size 89280512 objsize 64
Jul 13 02:30:52 pve1 kernel: Out of memory: Killed process 11082 (kvm).
...
Jul 13 02:30:52 pve1 kernel: Node 0 HighMem per-cpu: empty
Jul 13 02:30:52 pve1 kernel: Free pages: 65084kB (0kB HighMem)
Jul 13 02:30:52 pve1 kernel: Active:1056628 inactive:875357 dirty:4 writeback:82 unstable:0 free:16271 slab:33683
mapped-file:2040 mapped-anon:1222437 pagetables:5122
Jul 13 02:30:52 pve1 kernel: Node 0 DMA free:10496kB min:12kB low:12kB high:16kB active:0kB inactive:0kB
present:10008kB pages_scanned:0 all_unreclaimable? yes
Jul 13 02:30:52 pve1 kernel: lowmem_reserve[]: 0 1967 8015 8015
Jul 13 02:30:52 pve1 kernel: Node 0 DMA32 free:41564kB min:2808kB low:3508kB high:4212kB active:1378144kB
inactive:511624kB present:2014208kB pages_scanned:53 all_unreclaimable? no
Jul 13 02:30:52 pve1 kernel: lowmem_reserve[]: 0 0 6048 6048
Jul 13 02:30:52 pve1 kernel: Node 0 Normal free:13024kB min:8640kB low:10800kB high:12960kB active:2848368kB
inactive:2989804kB present:6193152kB pages_scanned:174 all_unreclaimable? no
Jul 13 02:30:52 pve1 kernel: lowmem_reserve[]: 0 0 0 0
Jul 13 02:30:52 pve1 kernel: Node 0 HighMem free:0kB min:128kB low:128kB high:128kB active:0kB inactive:0kB
present:0kB pages_scanned:0 all_unreclaimable? no
Jul 13 02:30:52 pve1 kernel: lowmem_reserve[]: 0 0 0 0
Jul 13 02:30:52 pve1 kernel: Node 0 DMA: 4*4kB 2*8kB 2*16kB 4*32kB 3*64kB 3*128kB 0*256kB 1*512kB 1*1024kB 0*2048kB
2*4096kB = 10496kB
Jul 13 02:30:52 pve1 kernel: Node 0 DMA32: 1205*4kB 439*8kB 187*16kB 53*32kB 20*64kB 3*128kB 1*256kB 0*512kB
0*1024kB 1*2048kB 6*4096kB = 41564kB
Jul 13 02:30:52 pve1 kernel: Node 0 Normal: 766*4kB 125*8kB 24*16kB 0*32kB 0*64kB 1*128kB 1*256kB 2*512kB 1*1024kB
1*2048kB 1*4096kB = 13024kB
Jul 13 02:30:52 pve1 kernel: Node 0 HighMem: empty
Jul 13 02:30:52 pve1 kernel: 884122 pagecache pages
Jul 13 02:30:52 pve1 kernel: Swap cache: add 1269311, delete 1059051, find 188370/216232, race 0+313+1
Jul 13 02:30:52 pve1 kernel: Free swap = 8kB
Jul 13 02:30:52 pve1 kernel: Total swap = 4194296kB
Jul 13 02:30:52 pve1 kernel: Free swap: 8kB
Jul 13 02:30:52 pve1 kernel: 2621440 pages of RAM
Jul 13 02:30:52 pve1 kernel: 586731 reserved pages
Jul 13 02:30:52 pve1 kernel: 384089 pages shared
Jul 13 02:30:52 pve1 kernel: 210260 pages swap cached
Jul 13 02:30:53 pve1 kernel: OOM killed process kvm (pid=6383, ve=0) exited, free=346165 gen=1.
Jul 13 02:30:53 pve1 kernel: vmbr0: port 2(vmtab201i0) entering disabled state
Jul 13 02:30:53 pve1 kernel: vmbr0: port 2(vmtab201i0) entering disabled state
Jul 13 02:31:24 pve1 ntpd[6324]: Deleting interface #8 vmtab201i0, fe80::b0f0:11ff:fe84:fcb3#123, interface stats:
received=0, sent=0, dropped=0, active_time=18900 secs