> 3000 mSec Ping and packet drops with VirtIO under load

Discussion in 'Proxmox VE: Installation and configuration' started by Andreas Piening, Sep 2, 2017.

  1. aderumier

    aderumier Member

    Joined:
    May 14, 2013
    Messages:
    203
    Likes Received:
    18
    yes
    only dependency is the old libscsi (which don't conflict with current libiscsi from proxmox5).
    See my previous post with links to .deb

    if you have time to test ,it could be great to known if it's work or not with qemu 2.7 from proxmox4, to be 100% sure that regression is coming from qemu 2.9.
     
  2. AlBundy

    AlBundy New Member

    Joined:
    Sep 26, 2017
    Messages:
    7
    Likes Received:
    0
    This also is going on with LXC for me (4 qemu server that are almost idle and 1 LXC container doing a lot of write actions)
    Screen Shot 2017-09-26 at 23.48.10.png Screen Shot 2017-09-26 at 23.47.41.png
     
  3. mac.linux.free

    Joined:
    Jan 29, 2017
    Messages:
    96
    Likes Received:
    5
    so it is not the kernel and not qemu-kvm?
     
  4. AlBundy

    AlBundy New Member

    Joined:
    Sep 26, 2017
    Messages:
    7
    Likes Received:
    0
    Im going to check kernel, because i upgraded from 4.4 (still installed) will report my results later today
     
  5. mac.linux.free

    Joined:
    Jan 29, 2017
    Messages:
    96
    Likes Received:
    5
    perhaps someone could push kernel 4.13 to pvetest as mentioned in roadmap to 5.1
     
  6. AlBundy

    AlBundy New Member

    Joined:
    Sep 26, 2017
    Messages:
    7
    Likes Received:
    0
    Its already feeling faster, will keep you guys up to date
     
  7. AlBundy

    AlBundy New Member

    Joined:
    Sep 26, 2017
    Messages:
    7
    Likes Received:
    0
    The last day it's running more stable and faster! Even during the same actions as I did Tuesday the IO delay didn't go sky-high and the vm didn't became unresponsive
     
  8. aderumier

    aderumier Member

    Joined:
    May 14, 2013
    Messages:
    203
    Likes Received:
    18
  9. mac.linux.free

    Joined:
    Jan 29, 2017
    Messages:
    96
    Likes Received:
    5
  10. Andreas Piening

    Joined:
    Mar 11, 2017
    Messages:
    58
    Likes Received:
    7
    Code:
    #uname -a
    Linux pve7 4.13.3-1-pve #1 SMP PVE 4.13.3-2 (Wed, 27 Sep 2017 14:01:40 +0200) x86_64 GNU/Linux
    I have installed pve-kernel-4.13.3-1-pve_4.13.3-2_amd64.deb on my test system and I can't see any difference. I have 18% packet loss during the boot process and I was pinging from the host. Response times fluctuating and often over 300ms.
    My test server has nothing running but this single windows server 2016 KVM VM so it is pretty good to compare the behavior between a single change on the configuration.

    What do you mean by that? The kernel was installed successfully and boots, or did it resolve the issue we're discussing here for you?
     
  11. Andreas Piening

    Joined:
    Mar 11, 2017
    Messages:
    58
    Likes Received:
    7
    Can you explain what you have changed, please?
     
  12. Andreas Piening

    Joined:
    Mar 11, 2017
    Messages:
    58
    Likes Received:
    7
    I have tried to install these two .deb packages:
    Code:
    #dpkg -i *.deb
    Selecting previously unselected package libiscsi4.
    (Reading database ... 66968 files and directories currently installed.)
    Preparing to unpack libiscsi4_1.15.0-1_amd64.deb ...
    Unpacking libiscsi4 (1.15.0-1) ...
    dpkg: warning: downgrading pve-qemu-kvm from 2.9.1-1 to 2.7.1-4
    Preparing to unpack pve-qemu-kvm_2.7.1-4_amd64.deb ...
    Unpacking pve-qemu-kvm (2.7.1-4) over (2.9.1-1) ...
    Setting up libiscsi4 (1.15.0-1) ...
    dpkg: dependency problems prevent configuration of pve-qemu-kvm:
     pve-qemu-kvm depends on libgnutls-deb0-28 (>= 3.3.0); however:
      Package libgnutls-deb0-28 is not installed.
     pve-qemu-kvm depends on libnettle4 (>= 2.6); however:
      Package libnettle4 is not installed.
     pve-qemu-kvm depends on libpng12-0 (>= 1.2.13-4); however:
      Package libpng12-0 is not installed.
    
    dpkg: error processing package pve-qemu-kvm (--install):
     dependency problems - leaving unconfigured
    Processing triggers for man-db (2.7.6.1-2) ...
    Errors were encountered while processing:
     pve-qemu-kvm
    I have searched on http://download.proxmox.com/debian/pve/dists/jessie/pve-no-subscription/binary-amd64/ but can't find the mentioned packages there.
    My guess is these are system packages from debian. Maybe I can find these in the debian repos.
    If you know where I can get these packages or if there is a better way, please let me know.
     
  13. aderumier

    aderumier Member

    Joined:
    May 14, 2013
    Messages:
    203
    Likes Received:
    18
    I have theses 3 packages installed, but It was on a proxmox5 after upgrade from proxmox4.
    they are coming from jessie package.
    You can install them safely, no conflict with newer packages

    http://ftp.us.debian.org/debian/pool/main/g/gnutls28/libgnutls-deb0-28_3.3.8-6+deb8u7_amd64.deb
    http://ftp.us.debian.org/debian/pool/main/n/nettle/libnettle4_2.7.1-5+deb8u2_amd64.deb
    http://ftp.us.debian.org/debian/pool/main/libp/libpng/libpng12-0_1.2.50-2+deb8u3_amd64.deb
     
  14. Andreas Piening

    Joined:
    Mar 11, 2017
    Messages:
    58
    Likes Received:
    7
    Thank you!

    I had to download one additional library "libhogweed2" but then it installed fine. I rebooted the host after the installation was done.

    But: Still the same issue!
     
  15. aderumier

    aderumier Member

    Joined:
    May 14, 2013
    Messages:
    203
    Likes Received:
    18
    if with kernel 4.4 + pve-qemu-kvm 2.7 both from proxmox 4 on proxmox 5, you still have the problem, I really don't understand what it can be ....

    Are you 100% sure that you don't have the same problem with proxmox 4 on this specific server ?
     
  16. felipe

    felipe Member

    Joined:
    Oct 28, 2013
    Messages:
    150
    Likes Received:
    1
    hi, yes we are also experiencing similar problems but with ide. the guest is a windows 2012 (not r2)
    we allways used ide drivers because virtio was slower in win2012 (NOT r2) then ide. (with windows 2012 r2 virtio is faster)
    after upgrading to proxmox 5 from 4.4 the io load from the hoste node changed from under 1% to about 5% .
    once we also experienced a total network hang - and had to reactivate the network card of the guest

    as far as i can read there is still no solution fot this?
     
  17. aderumier

    aderumier Member

    Joined:
    May 14, 2013
    Messages:
    203
    Likes Received:
    18
    maybe problem is different. (different storage,network configuration).
    maybe you can try to install pve-kernel 4.4 , test. and if it don't work, test pve-qemu-kvm 2.7 from proxmox4. (I have posted link in previous posts).
     
  18. AlBundy

    AlBundy New Member

    Joined:
    Sep 26, 2017
    Messages:
    7
    Likes Received:
    0
    Sure thing, nothing special though!

    I still had kernel version 4.4 installed, booted from that one and the whole system became more stable! I don't know if all issues are resolved but my I/O is back at 0.5% except for backup times then it goes to 4%
     
  19. Phinitris

    Phinitris Member

    Joined:
    Jun 1, 2014
    Messages:
    83
    Likes Received:
    11
    @AlBundy did you also had the network issues under load like lags and packet loss/high latency before?
     
  20. felipe

    felipe Member

    Joined:
    Oct 28, 2013
    Messages:
    150
    Likes Received:
    1
    hi,
    i can confirm now that with kernel 4.4.79 the load ist a lot better! (then the actual and newet kernel from proxmox 5.0)
    i hope that with proxmox 5.1 the new 5.12 kernel will resolve the load problems.
     
  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.
    Dismiss Notice