KVM guest installation is a bit slow, how to improve?

[Magneto]

Hi,

I wonder if someone can shed some light on this matter.
The environment:

3x Supermicro server with following features per server:
128GB RAM
3x 8TB SATA HDD
2x SSD drives (intel_ssdsc2ba400g4 - 400GB DC S3710)
2x 12 core CPU (Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz
Quad port 10Gbe Inter NIC
2x 10GB Cisco switches (to isolate storage network from LAN)

Installing CentOS 7 with 1523 package (i.e. the Development option on the CentOS 7 ISO), take aprox 18 minutes to install on this cluster.

As a test, to install it on a standalone machine with a hardware RAID controller with RAID1 and 256Mb cache, took 7 minutes - for the same installation.

I am using the VirtIO disk for the Linux VM, with 4 cores and 4GB RAM.

Disk IO, on the host node, spiked at 6.39Mbps during installation.
Network traffic spiked at 12.66Kbps
RAM spiked at 4.13Gb
CPU Usage spiked at 32.73%


Running top on the hostnode I noticed that KVM used 100% CPU, but only on a single core.

root@virt1:~# pveversion -v
proxmox-ve: 5.1-25 (running kernel: 4.13.4-1-pve)
pve-manager: 5.1-36 (running version: 5.1-36/131401db)
pve-kernel-4.13.4-1-pve: 4.13.4-25
pve-kernel-4.10.17-4-pve: 4.10.17-24
pve-kernel-4.10.17-2-pve: 4.10.17-20
libpve-http-server-perl: 2.0-6
lvm2: 2.02.168-pve6
corosync: 2.4.2-pve3
libqb0: 1.0.1-1
pve-cluster: 5.0-15
qemu-server: 5.0-17
pve-firmware: 2.0-3
libpve-common-perl: 5.0-20
libpve-guest-common-perl: 2.0-13
libpve-access-control: 5.0-7
libpve-storage-perl: 5.0-16
pve-libspice-server1: 0.12.8-3
vncterm: 1.5-2
pve-docs: 5.1-12
pve-qemu-kvm: 2.9.1-2
pve-container: 2.0-17
pve-firewall: 3.0-3
pve-ha-manager: 2.0-3
ksm-control-daemon: 1.2-2
glusterfs-client: 3.8.8-1
lxc-pve: 2.1.0-2
lxcfs: 2.0.7-pve4
criu: 2.11.1-1~bpo90
novnc-pve: 0.6-4
smartmontools: 6.5+svn4324-1
zfsutils-linux: 0.7.2-pve1~bpo90
ceph: 12.2.1-pve3

root@virt1:~# pveperf
CPU BOGOMIPS: 211200.00
REGEX/SECOND: 2496022
HD SIZE: 28.67 GB (/dev/mapper/pve-root)
BUFFERED READS: 510.61 MB/sec
AVERAGE SEEK TIME: 0.07 ms
FSYNCS/SECOND: 331.83
DNS EXT: 152.90 ms
DNS INT: 2.68 ms (fermel.co.za)

 

[czechsys]

On what storage you install? From what source you install? Plus, any installation will be single core anyway, so it looks as problem with network or storage.

 

[Magneto]

Installing onto CEPH. You're saying 10Gbe is too slow??

 

[Magneto]

On what storage you install? From what source you install? Plus, any installation will be single core anyway, so it looks as problem with network or storage.

I am installing from an ISO hosted on the proxmox cluster, hosted on glusterfs, to a CEPH storage.
The network is 10Gb and utilization is low.

I find it hard to believe that installation is single threaded. The exact same installation, from a USB memory stick onto a Corei5 desktop PC was 2x faster.