PVE uses swap while RAM is*enough, why?

Discussion in 'Proxmox VE: Installation and configuration' started by achekalin, Oct 17, 2014.

  1. achekalin

    achekalin Member

    Joined:
    Jun 17, 2011
    Messages:
    74
    Likes Received:
    0
    Recently I got a chance to install PVE 3.3 on quite a capable server (Dual 6-core Xeons, 128 RAM, SSD disks). Due to fact it has a lot of RAM I set up swap to only 8Gb thinking I'll newer need any (and I hate to spend SSDs for swap, too). I created and run some VMs, and as I finished I have 126 out of 128 GB assigned to machines. Guest OSes are Ubuntu server 12.04 and Windows 2008 R2.

    To my great surprise, after only two days of running I see that swap usage on hast server is 6 Gb, and I see no reason why it is so.

    The free -m command says this:

    Code:
                 total       used       free     shared    buffers     cached
    Mem:        128902     124397       4504          0        244        142
    -/+ buffers/cache:     124010       4892
    Swap:         8191       1334       6857
    So I see 6.8 Gb of swap out of 8 is used now, quite a amount!

    I can play with swappiness variable, but should I do that or I'd better decrease RAM assignment on some of VMs to have some spare free RAM? Then how much RAM should I preserve and keep 'just in case'?

    I found no recommendation on that subject, but I think the question is serious: there are a lot of PVE servers nowdays uses SSDs (or something SSD-backed) as a storage, so wasting it as swap will wear it out and anyway decreases system speed.

    Thank you in advance for any help on this matter, PVE is great system and I hope this turns out to be easily resolvable question!
     
  2. dietmar

    dietmar Proxmox Staff Member
    Staff Member

    Joined:
    Apr 28, 2005
    Messages:
    16,432
    Likes Received:
    299
    No, only 1.3GB are swapped.
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  3. achekalin

    achekalin Member

    Joined:
    Jun 17, 2011
    Messages:
    74
    Likes Received:
    0
    You're right. I saw 6 Gb some time ago, so it appears to fluctuate.

    But anyway: why even 1 Gb of swap used while I have much more RAM as 'free'? Any approaches to free swap completely? Lower overall RAM assignment, zRAM?
     
  4. mo_

    mo_ Member

    Joined:
    Oct 27, 2011
    Messages:
    399
    Likes Received:
    3
    I'm afraid this is a major design flaw in both linux and windows alike: if you give it swap space, it'll use it, regardless of the amount of free ram available. For linux this even tends to happen if you mess around with the swappiness tunable.
     
  5. achekalin

    achekalin Member

    Joined:
    Jun 17, 2011
    Messages:
    74
    Likes Received:
    0
    Sounds quite correct! :(

    Just out of curiosity: may this be a good idea to implement swap made of zRAM block device(es)? It's common practice when we come to notebook with large RAM (which is not uncommon today) and SSD disk - to prevent SSD wear out and seepd things up a bit it's a good idea to use zRAM-based swap.
     
  6. mijanek

    mijanek Member

    Joined:
    Sep 3, 2013
    Messages:
    31
    Likes Received:
    0
    no I wouldn't say it that way...:

    Code:
    root@proxmox:~# free -m
                 total       used       free     shared    buffers     cached
    Mem:         16032      15869        163          0        816       5780
    -/+ buffers/cache:       9272       6760
    Swap:        15231          0      15231
    
    root@proxmox:~# uptime
     18:38:09 up 8 days, 20:20,  1 user,  load average: 0.20, 0.23, 0.26
    
    
    This is likely, that you may temporarily used more RAM than it was avaiable, therefore part of it has been swapped out, and as long there is no need of it, it will remain swapped.
     
  7. achekalin

    achekalin Member

    Joined:
    Jun 17, 2011
    Messages:
    74
    Likes Received:
    0
    Yes it looks like that. But given the fact I use th same number of VMs, and all of them are configured once and never touched, and sum of all RAM assigned is less that total RAM amount that's installed in host machine, and I finally got 'out of RAM' which given me swap usage, then I really wanted to know how mocu RAM should I keep in mind to be reserved when configure VMs.

    Say 128 GB is plenty, so how much should I keep untouched: 4, 6, 8 GB (thus limit total for VMs to 124, 122, 120 Gb), or maybe even more? Any idea how much should this number be, and if it differs for host with 64 or 128 or 256 Gb of RAM?
     
  8. achekalin

    achekalin Member

    Joined:
    Jun 17, 2011
    Messages:
    74
    Likes Received:
    0
    Funny thing: I tried to set up 4 zRAM block devices each of 1,5 Gb, and added it as swap with high priority. I set vm.swappiness = 10. After some hours with this config, I see these numbers:
    Code:
    # free -m
                 total       used       free     shared    buffers     cached
    Mem:        128902     125280       3621          0        295        147
    -/+ buffers/cache:     124837       4064
    Swap:        14335       5681       8654
    and
    Code:
    # swapon -s
    Filename                                Type            Size    Used    Priority
    /dev/mapper/pve-swap                    partition       8388600 954672  -1
    /dev/zram0                              partition       1572856 1215592 10
    /dev/zram1                              partition       1572856 1215976 10
    /dev/zram2                              partition       1572856 1215764 10
    /dev/zram3                              partition       1572856 1216124 10
    So I just can't understang, I see 3Gb of free ram, and I use 1,5*4 Gb RAM as zRAM devices - that amount is quite a lot to hold all that swap space used in my original post. But anyway I see swap is used. What I missed?
     
  9. macday

    macday Member

    Joined:
    Mar 10, 2010
    Messages:
    408
    Likes Received:
    0
    Please don´t use zram on proxmox. This will slowdown the whole system on swap usage.

    Just put a vm.swappiness = 0 in your /etc/sysctl.conf and reboot the host.
     
  10. achekalin

    achekalin Member

    Joined:
    Jun 17, 2011
    Messages:
    74
    Likes Received:
    0
    But if I have a lot of free memory, won't it be wise to keep some spare swap space made of zram (thus fast one, not like ordinary hard drive, even in raid)? I understand that if I have free memory then I should not need swap at all but as I used host 128 GB of RAM and assign 126 Gb to VMs I got some used GB of swap, so now I'm in doubt how much ram keep free so no swap be used at all.

    Will try, thanks!
     
  11. gerdnl

    gerdnl Member

    Joined:
    Oct 21, 2013
    Messages:
    114
    Likes Received:
    0
    that wont work with a privilaged LXC container
     
  12. jason0

    jason0 New Member

    Joined:
    Jan 12, 2018
    Messages:
    2
    Likes Received:
    0
    I have this problem too. I set swappiness to 0 and it will still occasionally try to swap even though I have plenty of free RAM.
    And anytime it tries to swap, it brings my system to a halt. Can't do anything!
    VM's that normally take a few seconds to boot, takes minutes instead.

    I feel like there is a different underlying problem, but performing 'swapoff -a' is a good solution for me.
     
  13. gerdnl

    gerdnl Member

    Joined:
    Oct 21, 2013
    Messages:
    114
    Likes Received:
    0
    if u put swapiness to 0 ur disabling the swap and ur system will crash when its out of memory , put it on 1 or 10
     
  14. gerdnl

    gerdnl Member

    Joined:
    Oct 21, 2013
    Messages:
    114
    Likes Received:
    0
    recently swapiness 0 has become to disable swap and 1 is now the same as (only swap when needed)
     
  15. gerdnl

    gerdnl Member

    Joined:
    Oct 21, 2013
    Messages:
    114
    Likes Received:
    0
    anyone from proxmox able to clear this up?
     
  16. gerdnl

    gerdnl Member

    Joined:
    Oct 21, 2013
    Messages:
    114
    Likes Received:
    0
    no answer from proxmox ?
     
  17. Petr

    Petr Member

    Joined:
    Dec 13, 2013
    Messages:
    40
    Likes Received:
    0
    Occasionally have this issue. Happened before on PVE 5.0 and happens now on recent PVE version.
    Intensive swap IO correlates to backup (vzdump) process. Processes, that don't release swap memory, are kvm running Windows.

    # free -m
    total used free shared buff/cache available
    Mem: 290204 158568 1493 1113 130143 137079
    Swap: 8191 8033 158

    # pveversion
    pve-manager/5.1-46/ae8241d4 (running kernel: 4.13.13-6-pve)
     
  18. Rhinox

    Rhinox Active Member

    Joined:
    Sep 28, 2016
    Messages:
    272
    Likes Received:
    35
    I really do not understand this whining about swap over and over again...
    If you do not want to see swap being used, simply buy more ram and disable swap completely!
     
  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.
    Dismiss Notice