ProxmoxVE host very slow to cp file

[garrypb]

Have ProxmoxVE installed & running a couple of test vm's for past 2 weeks, with no observed problems. Today converted a vmdk to .raw 10G file on our vmware server, then spc it to proxmox in the / partition by mistake, no probs, just mv it to /var/lib/vz/images , But its taken over 90 minutes with big IO delays to move it.

Its a standard 1.5 install onto a single 500gb sata drive with VE's standard default partitioning, so / & /var/lib/vz/images/ are different partitions. No vm's running at the time, No CPU or Mem overheads, but could see big % IO delays on web interface graph. Its just a test machine so nothing fancy, CPU Intel(R) Core(TM)2 CPU 6600, 4G ram with a new 500G SATA2 7290 32m cache HD.

If I boot the same test PC up on other identical HD, debian 5 server & cp similar size file between partitions, file copies normally.

Any ideas on what may cause this proxmox.

Thanks
Garry

 

[giner]

It looks like sata-compatible mode is enabled in bios.

 

[garrypb]

It looks like sata-compatible mode is enabled in bios.

Thanks for the reply giner. I tried changing bios setting for SATA, made no difference.

 

[garrypb]

I'm guessing this is an LMV issue, ProxmoxVE 1.5 default install partitions & sets up an LMV - but if I'm the only one seeing this issue, it could mean know one else has performed the default install to a single SATA drive, or that the problem is specific to me alone. I guessing that you can not run an LVM effectively on a single drive install???

If anyone reading has done a similar install to mine (default to single SATA) please let me know if you do or don't have this problem???

If I boot up this test machine with my debian base vmware server install (no LVM) I can rsync (using rsync to get some stats) files from one dir to another with average of 32MB/s for a 5.1G file taking 2.38sec. Boot up with deian base Proxmox VE (LMV) and I give up after 10min, max speed 3.6MB/s down to .05MB/s.

 

[tom]

install on single sata works without the mentioned problems here, so there must something different on your side.

 

[garrypb]

install on single sata works without the mentioned problems here, so there must something different on your side.

Hi Tom, thanks for the reply. After reading your reply I did a new install on a different PC with a new SATA drive, but I have only produced the same result. Just tested by rsync 5.1G file from /root > /var/lib/vz and will only avg a little over 1MB/s.

One thing I noticed when shutdown system was an error similar to: 'failed to stop LVM pve group 1' this error is always there on both installs, no errors on boot.

 

[tom]

hi,

here is the result from one of our test machines (Intel(R) Xeon(R) CPU X3220 @ 2.40GHz, 8 GB Ram, quite old 250 GB Seagate SATA).

time cp linux.iso /var/lib/vz

real    2m9.100s
user    0m0.080s
sys     0m11.370s
proxmox:~# ls -alh linux.iso
-rw-r--r-- 1 root root 4.1G May 25 15:33 linux.iso

this is around 30 mb per second.

 

[garrypb]

see below

 

[garrypb]

Your result is much the same as I get with both of my machines booted into a standard Debian install, standard partitioning, non LVM. But when booted in Proxmox VE, both machines produce the same slow file cp or mv speeds, they drop to about 1MB/s

Appreciate any suggesting, things to look & or try.

Thanks
Garry

 

[dietmar]

What kind of network card do you have?

 

[garrypb]

Hi Dietmar, thanks for the reply, its a Realtek RTL8111B.

 

[garrypb]

If This is only happening to me, I'm guessing it has to be a hardware driver issue, as:
Both PC's run very similar hardware -
Both PC's do not have this problem running Debain 5 with 2.6.31 kernel

So will try moving from proxmox-ve-2.6.18 to proxmox-ve-2.6.24 kernel and see what happens. Still find it hard to believe that know one else has run into this issue, well that is bothering to read this thread anyway.

 

[tom]

what mainboard do you have?

 

[garrypb]

Finally resolved with the install of the 2.6.24 kernel.
I nearly went over to that kernel last week, before I noticed this issue, but decided to stick with the stable .18. I've confirmed this on three of my PC's, all have a different gigabyte PC type MB's. Perhaps something I may not of ran into if using my server machine instead of a few test PC's. I've waisted a lot of time, so I hope that if this happens to some else, they find this thread useful.

Now keeping my fingers crossed that both KVM & OpenVZ are stable for me on this kernel.

Cheers
Garry

PS Still can't believe I'm the first, to run into this, I guess someone has to be first

 

[tom]

thanks for the update. our 2.6.18 kernel is optimized for server hardware (similar to RHEL5.4), so I can imagine that maybe some of your desktop mainboards components causes the issue.

for reliable operation, always choose server hardware (which is certified for Linux, e.g. for RHEL5.4) or ask others via the forum which boards are known to work well.

 

[kevinthecomputerguy]

I was having the same problem, and i found this command very helpful, it eliminates the network from the trouble-shooting. It creates a 10GB file.

time dd if=/dev/zero of=/var/lib/vz/10gb.fake.file.4speed_test.dd bs=1k count=10000000

and or

time dd if=/dev/zero of=/mymoutns/disk2/10gb.fake.file.4speed_test.dd bs=1k count=10000000

My troubleshooting seemed to point to LVM as well. As my Debian ext2 setups were super fast.
I was also using western digital green drives, which was always in the back of my mind.

I lucky had a friend that wanted to trade his server for my desktop (Dell optiplex 780) and on this server i dont have these problems.

So i have to agree with everyone here.
-Fresh Debian \ ext2, no problems.
-Traded hardware for server, no problems.


Loving proxmox, thanks guys

 

[kevinthecomputerguy]

Forgot to mention, this helped my about 10MB per second (on the desktop I dont have anymore)

echo 1 > /sys/block/sda/device/queue_depth
echo 1 > /sys/block/sdb/device/queue_depth
echo 1 > /sys/block/sdc/device/queue_depth

*Again, server hardware did fix my problem