Proxmox KVM disk Slow

[reloaded]

Good morning,

I'm with a Proxmox working with a hardware raid card, but apparently is giving me problems with the slowness of the disk. The disks are SATA II with 1TB of capacity. . 7200rpm

I have problems with any copies of files in Proxmox virtual machines or the I / O delay rises to levels of 50 and 60%

I got a pveperf but there are values which are not interpreted. Can you tell me if these settings are correct?


ded-it151:~# pveperf
REGEX/SECOND: 786200
HD SIZE: 94.49 GB (/dev/pve/root)
BUFFERED READS: 113.99 MB/sec
AVERAGE SEEK TIME: 9.12 ms
FSYNCS/SECOND: 111.56
DNS EXT: 2229.96 ms
DNS INT: 2131.51 ms


which may be the problem?

Thanks very much

 

[nick]

How much cache do you have on RAID card? Write cache it's enable on controller?

 

[Ninjix]

Your fsync/s numbers look really low for a hardware RAID. Please provide us more detailed information about your RAID card and Proxmox system configuration.

The output from the pveversion -v command would also be helpful.

 

[tom]

FSYNCS/SECOND: 111.56

shows that you do not have the write cache enabled on the raid controller (write back).

depending on the raid controller, it should be at least 1000.

I have a test machine with 4 x 1 TB sata drives (WD RE2), configured as raid10 (Adaptec 5805Z):

pveperf
CPU BOGOMIPS:      8533.63
REGEX/SECOND:      717886
HD SIZE:           94.49 GB (/dev/mapper/pve-root)
BUFFERED READS:    349.00 MB/sec
AVERAGE SEEK TIME: 7.24 ms
FSYNCS/SECOND:     3034.91
DNS EXT:           28.07 ms
DNS INT:           0.92 ms

 

[jroper]

Hi

I have a hosted server with a supposedly fast company in the UK and pveperf gives me the following results.

proxmox:~# pveperf
CPU BOGOMIPS: 11199.82
REGEX/SECOND: 693334
HD SIZE: 57.09 GB (/dev/mapper/pve-root)
BUFFERED READS: 0.49 MB/sec
AVERAGE SEEK TIME: 11426.83 ms
FSYNCS/SECOND: 76.53
DNS EXT: 9.85 ms
DNS INT: 9.14 ms

The disk IO seem a tad slow to me, and the Average seek times seem a little high - can anyone confirm?

pverversion gives:-

proxmox:~# pveversion -v
pve-manager: 1.5-9 (pve-manager/1.5/4728)
running kernel: 2.6.18-2-pve
proxmox-ve-2.6.18: 1.5-5
pve-kernel-2.6.18-2-pve: 2.6.18-5
pve-kernel-2.6.18-1-pve: 2.6.18-4
qemu-server: 1.1-14
pve-firmware: 1.0-5
libpve-storage-perl: 1.0-13
vncterm: 0.9-2
vzctl: 3.0.23-1pve11
vzdump: 1.2-5
vzprocps: 2.0.11-1dso2
vzquota: 3.0.11-1
pve-qemu-kvm-2.6.18: 0.9.1-5

It usually takes about 45 minutes to an hour to create a new openvz image from a 500Mb template, and about 10 minutes to start an openVZ container.

Joe

 

[nick]

you have a serious disk issue. Please tell us what hard-disk do you use and what main board.

maybe you need to make some settings into your BIOS...or maybe some update to latest BIOS release

 

[jroper]

Hi

hdparm -i gives:

 Model=ST3250310AS, FwRev=3.AAF, SerialNo=6RYPCH52
 Config={ HardSect NotMFM HdSw>15uSec Fixed DTR>10Mbs RotSpdTol>.5% }
 RawCHS=16383/16/63, TrkSize=0, SectSize=0, ECCbytes=4
 BuffType=unknown, BuffSize=8192kB, MaxMultSect=16, MultSect=16
 CurCHS=16383/16/63, CurSects=16514064, LBA=yes, LBAsects=488397168
 IORDY=on/off, tPIO={min:120,w/IORDY:120}, tDMA={min:120,rec:120}
 PIO modes:  pio0 pio1 pio2 pio3 pio4
 DMA modes:  mdma0 mdma1 mdma2
 UDMA modes: udma0 udma1 udma2 udma3 udma4 udma5 *udma6
 AdvancedPM=no WriteCache=enabled
 Drive conforms to: Unspecified:  ATA/ATAPI-1,2,3,4,5,6,7

Googling on the part number suggests it is a Seagate Barracuda 250 GB Internal hard drive - 300 MBps - 7200 rpm

The mainboard is unknown to me as this is a hosted dedicated server - but lspci suggests the chipset is Intel.

Results of pveperf look slightly better this AM, when there is no activity going on, but still not particularly quick.

proxmox:~# pveperf
CPU BOGOMIPS:      11199.82
REGEX/SECOND:      527719
HD SIZE:           57.09 GB (/dev/mapper/pve-root)
BUFFERED READS:    0.61 MB/sec
AVERAGE SEEK TIME: 40.32 ms
FSYNCS/SECOND:     98.76
DNS EXT:           10.55 ms
DNS INT:           10.23 ms (concordiax.com)

Joe

 

[sumsum]

Hello

I've had similar issues in the past, even using a 3ware controller with write cache eneabled. There are serveral ways to tune the I/O performance of your disks. most of the time slow disk performance are because of the I/O wait state. you can check this by using the the tool "dstat"
For myself I'm using this little shell script - starts on every boot. the value in this script may differ, depending on your raid controller and various iteration of testing.

EXAMPLE SCRIPT

#! /bin/sh
echo "deadline" > /sys/block/sda/queue/scheduler
echo "64" > /sys/block/sda/queue/max_sectors_kb
blockdev --setra 16384 /dev/sda
echo "512" > /sys/block/sda/queue/nr_requests

Google for IO performance tuning or use the following link as a first starting point.

http://hep.kbfi.ee/index.php/IT/KernelTuning

Regards
tom

 

[jroper]

Things are looking a little healthier now.

It was tested with a live cd, and the hardware gave good results, so it had to be a problem in the OS.

The key was in an error message " /dev/dm-5: read failed after 0 of 4096 at 0: Input/output error" and the first hit on Google took me to this forum - http://forum.proxmox.com/threads/635-backup-with-snapshot, running lvscan showed I had the same problem, and below are my new results of pveperf and hdparm -tT


/dev/sda:~# hdparm -tT /dev/sda

/dev/sda:
Timing cached reads: 3884 MB in 2.00 seconds = 1944.10 MB/sec
Timing buffered disk reads: 296 MB in 3.01 seconds = 98.42 MB/sec


CPU BOGOMIPS: 11200.31
REGEX/SECOND: 564200
HD SIZE: 57.09 GB (/dev/mapper/pve-root)
BUFFERED READS: 96.58 MB/sec
AVERAGE SEEK TIME: 10.95 ms
FSYNCS/SECOND: 1173.86
DNS EXT: 14.11 ms
DNS INT: 8.61 ms


Which look more reasonable.

Thanks for your time.

Joe