When I try to restore a KVM machine with qmrestore on my Proxmox 2.1 cluster, I have lot of this messages in dmesg.
Code:
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
vgs D ffff880c7709c280 0 3981 1 0 0x00000007
ffff880c63fd1aa8 0000000000000086 0000000000000000 ffffffff8141c3bc
ffff881873814cb0 ffff88187955dc00 0000000000000001 000000000000000c
ffff880c63fd1a48 ffff880c7709c820 ffff880c63fd1fd8 ffff880c63fd1fd8
Call Trace:
[<ffffffff8141c3bc>] ? dm_table_unplug_all+0x5c/0x100
[<ffffffff81525ed3>] io_schedule+0x73/0xc0
[<ffffffff811d2bce>] __blockdev_direct_IO_newtrunc+0x6ee/0xb80
[<ffffffff811d30be>] __blockdev_direct_IO+0x5e/0xd0
[<ffffffff811cf940>] ? blkdev_get_blocks+0x0/0xc0
[<ffffffff811d0797>] blkdev_direct_IO+0x57/0x60
[<ffffffff811cf940>] ? blkdev_get_blocks+0x0/0xc0
[<ffffffff81123f38>] mapping_direct_IO+0x48/0x70
[<ffffffff8112713b>] generic_file_read_iter+0x60b/0x680
[<ffffffff8105bca3>] ? perf_event_task_sched_out+0x33/0x80
[<ffffffff81064b88>] ? dequeue_entity+0x108/0x290
[<ffffffff8112723b>] generic_file_aio_read+0x8b/0xa0
[<ffffffff811949da>] do_sync_read+0xfa/0x140
[<ffffffff810d7c45>] ? finish_callback+0xa5/0x140
[<ffffffff81095be0>] ? autoremove_wake_function+0x0/0x40
[<ffffffff810d83fc>] ? utrace_stop+0x12c/0x1e0
[<ffffffff811953c5>] vfs_read+0xb5/0x1a0
[<ffffffff81195501>] sys_read+0x51/0x90
[<ffffffff8100b398>] tracesys+0xd9/0xdeI also see with ps that there are vgs processes "hanging"
2286 ? Ss 0:00 pvedaemon worker
2293 ? S 0:00 \_ pvedaemon
4981 ? D 0:00 | \_ /sbin/vgs --separator : --noheadings --units b --unbuffered --nosuffix --options vg_name,vg_size,vg_free
2294 ? S 0:00 \_ pvedaemon worker
4994 ? D 0:00 | \_ /sbin/vgs --separator : --noheadings --units b --unbuffered --nosuffix --options vg_name,vg_size,vg_free
2295 ? S 0:00 \_ pvedaemon worker
4986 ? D 0:00 \_ /sbin/vgs --separator : --noheadings --units b --unbuffered --nosuffix --options vg_name,vg_size,vg_free
2306 ? Ss 0:00 /usr/sbin/apache2 -k start
4821 ? S 0:00 \_ /usr/sbin/apache2 -k start
4852 ? S 0:00 \_ /usr/sbin/apache2 -k start
2316 ? S 0:02 pvestatd
4938 ? D 0:00 \_ /sbin/vgs --separator : --noheadings --units b --unbuffered --nosuffix --options vg_name,vg_size,vg_free
After a few minutes the cluster icon of this cluster node turns red and the connection has a timeout. When the restore finishes everything seems to work normal again.
I wonder if qmrestore uses too much IO and therefore blocks vgs (it should not, this storage is a fast FC Array). Is there a possibility to ionice qmrestore (just like vzdump)?
I wonder if qmrestore uses too much IO and therefore blocks vgs (it should not, this storage is a fast FC Array). Is there a possibility to ionice qmrestore (just like vzdump)?
pveversion --verbose
pve-manager: 2.1-14 (pve-manager/2.1/f32f3f46)
running kernel: 2.6.32-14-pve
proxmox-ve-2.6.32: 2.1-74
pve-kernel-2.6.32-11-pve: 2.6.32-66
pve-kernel-2.6.32-14-pve: 2.6.32-74
lvm2: 2.02.95-1pve2
clvm: 2.02.95-1pve2
corosync-pve: 1.4.3-1
openais-pve: 1.1.4-2
libqb: 0.10.1-2
redhat-cluster-pve: 3.1.92-3
resource-agents-pve: 3.9.2-3
fence-agents-pve: 3.1.8-1
pve-cluster: 1.0-27
qemu-server: 2.0-49
pve-firmware: 1.0-18
libpve-common-perl: 1.0-30
libpve-access-control: 1.0-24
libpve-storage-perl: 2.0-31
vncterm: 1.0-3
vzctl: 3.0.30-2pve5
vzprocps: 2.0.11-2
vzquota: 3.0.12-3
pve-qemu-kvm: 1.1-8
ksm-control-daemon: 1.1-1
