Hello!
I've seen several threads on the forum regarding the "qm importdisk" command and its possible impact on VM performance.
The importdisk command is a little bit too fast, meaning it impacts the iowait performance of the VMs running on the host. I.e. we are seeing 10-12G/s writes happening during the convert and the VMs have 25-30% io-wait. It's not too bad for small disks but when importing 400-500G disks we would like the impact to be less.
I've been looking at several ways of limiting this and found that there is actually a fairly easy way to do this, however qm does not support passing the rate limit option to qemu-img convert.
My Perl skills are pretty much non-existing, but from what I can see this is where qm arranges for the disk import:
https://github.com/proxmox/qemu-server/blob/master/PVE/QemuServer/ImportDisk.pm#L73
Specifically the second line performs the qemu-img convert command that is very IO intesive.
There is actually a 6th argument that can be given to the qemu_img_convert option to limit the IO.
This can be seen here: https://github.com/proxmox/qemu-ser...fac1dc7264771a3b7a7ae/PVE/QemuServer.pm#L7640
And then the actual rate limit is being passed here. Note that it adds a hardcoded `K` suffix, meaning that the call to this function must pass the ratelimit in kilobytes/second:
https://github.com/proxmox/qemu-ser...fac1dc7264771a3b7a7ae/PVE/QemuServer.pm#L7680
I have manually patched `/usr/share/perl5/PVE/QemuServer/ImportDisk.pm` to test this with a += 1GB/sec speed limit and made the call as such:
This limits the speed correctly and solves the issue.
The feature I request is that we can add a ratelimit through the `qm importdisk` command natively in order to ratelimit the call to `qemu-img` without having to manually patch the perl files.
Thank you!
I've seen several threads on the forum regarding the "qm importdisk" command and its possible impact on VM performance.
The importdisk command is a little bit too fast, meaning it impacts the iowait performance of the VMs running on the host. I.e. we are seeing 10-12G/s writes happening during the convert and the VMs have 25-30% io-wait. It's not too bad for small disks but when importing 400-500G disks we would like the impact to be less.
I've been looking at several ways of limiting this and found that there is actually a fairly easy way to do this, however qm does not support passing the rate limit option to qemu-img convert.
My Perl skills are pretty much non-existing, but from what I can see this is where qm arranges for the disk import:
https://github.com/proxmox/qemu-server/blob/master/PVE/QemuServer/ImportDisk.pm#L73
Code:
PVE::Storage::activate_volumes($storecfg, [$dst_volid]);
PVE::QemuServer::qemu_img_convert($src_path, $dst_volid, $src_size, undef, $zeroinit);
PVE::Storage::deactivate_volumes($storecfg, [$dst_volid]);
PVE::QemuConfig->lock_config($vmid, $create_drive) if !$params->{'skip-config-update'};Specifically the second line performs the qemu-img convert command that is very IO intesive.
There is actually a 6th argument that can be given to the qemu_img_convert option to limit the IO.
This can be seen here: https://github.com/proxmox/qemu-ser...fac1dc7264771a3b7a7ae/PVE/QemuServer.pm#L7640
Code:
my ($src_volid, $dst_volid, $size, $snapname, $is_zero_initialized, $bwlimit) = @_;And then the actual rate limit is being passed here. Note that it adds a hardcoded `K` suffix, meaning that the call to this function must pass the ratelimit in kilobytes/second:
https://github.com/proxmox/qemu-ser...fac1dc7264771a3b7a7ae/PVE/QemuServer.pm#L7680
Code:
push @$cmd, '-r', "${bwlimit}K" if defined($bwlimit);I have manually patched `/usr/share/perl5/PVE/QemuServer/ImportDisk.pm` to test this with a += 1GB/sec speed limit and made the call as such:
Code:
PVE::QemuServer::qemu_img_convert($src_path, $dst_volid, $src_size, undef, $zeroinit, '1000000');This limits the speed correctly and solves the issue.
The feature I request is that we can add a ratelimit through the `qm importdisk` command natively in order to ratelimit the call to `qemu-img` without having to manually patch the perl files.
Thank you!
