High CPU when migrating windows 10 VM

[gboyce]

Hi, this is my first post here. Proxmox is so easy and intuitive that I haven't needed any assistance at all until now.
I want to migrate a windows 10 VM HDD from the local storage to an ISCSI shared storage. I have moved my other Linux VM HDDs with absolutely no issue. When I try to move the windows VM HDD the Host CPU sky rockets to over 50% from its average of about 7%. When this happens all VMs on that host become unresponsive and IO delay goes as high as 58%. The host is a Dell R720 with a dual 10G fiber network card. I haven't found anything related on a google search but maybe I just searched with the wrong questions. What are your thoughts?

 

[showiproute]

What disks are you using at your ISCSI share storage? Regular spinners (HDDs)?

How much space is Windows using within the virtual disk?


In general the IO delay goes up if you want to copy stuff to a "low speed" target like a regular disk.

 

[gboyce]

The ISCSI storage is on a dell R720 also, with 6 3TB drives in a RAID10 with a 3Gbps bond/port-channel. Yes they are spinners (7200RPM).
The VM is using 32.2 GB out of 120GB. Like I said, I didn't have this issue moving the VM HDD of any of my other VMs (6 Linux VMs).

 

[showiproute]

I can remember when migrating some W2k19 servers I had similar issues - either CPU load or IO delay was high.
What helped me (at least a bit) was to modify some sysctl values:

##
# Adjust vfs cache
# https://lonesysadmin.net/2013/12/22/better-linux-disk-caching-performance-vm-dirty_ratio/
# Decriase dirty cache to faster flush on disk
vm.dirty_background_ratio = 5
vm.dirty_ratio = 10

Source: https://gist.github.com/sergey-dryabzhinsky/bcc1a15cb7d06f3d4606823fcc834824

 

[gboyce]

I can remember when migrating some W2k19 servers I had similar issues - either CPU load or IO delay was high.
What helped me (at least a bit) was to modify some sysctl values:

##
# Adjust vfs cache
# https://lonesysadmin.net/2013/12/22/better-linux-disk-caching-performance-vm-dirty_ratio/
# Decriase dirty cache to faster flush on disk
vm.dirty_background_ratio = 5
vm.dirty_ratio = 10

Source: https://gist.github.com/sergey-dryabzhinsky/bcc1a15cb7d06f3d4606823fcc834824

I assume this will require me to reboot the PVE host?

 

[showiproute]

I assume this will require me to reboot the PVE host?

No, a simply

sysctl --system

will load all new/changed values directly to the system without a reboot.

 

[gboyce]

No, a simply

sysctl --system

will load all new/changed values directly to the system without a reboot.

Sorry about the delay. Do I use the entire conf file that you referenced or just make one with what you specified?

 

[showiproute]

Sorry about the delay. Do I use the entire conf file that you referenced or just make one with what you specified?

I have a dedicated .conf file just for the values I need which is within /etc/sysctl.d/ folder

 

[gboyce]

Thanks. That seems to have prevented the lockup. But the HDD clone is still not really doing anything...

create full clone of drive scsi0 (vmstore1:107/vm-107-disk-0.qcow2)
Logical volume "vm-107-disk-3" created.
drive mirror is starting for drive-scsi0 with bandwidth limit: 51200 KB/s

What do I look for to see why its not progressing?

 

[showiproute]

Hmmm have you tried to do the clone without bandwith limitation?

 

[gboyce]

Yes, I only turned on rate limits once I had a problem. Tried again without the rate limit and it still doesn't make progress and the CPU usage still takes the host from 5% to over 50% CPU. I have been trying to do this with the VM running. Could that be the issue. Do I need to shutdown the VM first?

 

[showiproute]

How long did you let the task run?

You might try that (shutdown VM) but in general there should not be any difference from copying speed.