RAM Problem since Upgrade to 3.4

[trendco]

Is there Change in RAM Handling in 3.4?

Since i Upgraded, it is not possible to start the same number of KVM-VM's -> the RAM usage increases, the Host slow down and after a few Minutes the Host crash completly.

Can I test the previous Version of QEMU anyway?
Or what could even be the reason for this? I've tested the previous Kernel, but there is the same Problem.

pveversion -v
proxmox-ve-2.6.32: 3.3-147 (running kernel: 3.10.0-7-pve)
pve-manager: 3.4-1 (running version: 3.4-1/3f2d890e)
pve-kernel-3.10.0-7-pve: 3.10.0-27
pve-kernel-3.10.0-4-pve: 3.10.0-17
pve-kernel-2.6.32-32-pve: 2.6.32-136
pve-kernel-2.6.32-27-pve: 2.6.32-121
pve-kernel-3.10.0-5-pve: 3.10.0-19
pve-kernel-2.6.32-28-pve: 2.6.32-124
pve-kernel-2.6.32-25-pve: 2.6.32-113
pve-kernel-2.6.32-37-pve: 2.6.32-147
pve-kernel-2.6.32-34-pve: 2.6.32-140
pve-kernel-2.6.32-31-pve: 2.6.32-132
pve-kernel-2.6.32-26-pve: 2.6.32-114
pve-kernel-2.6.32-23-pve: 2.6.32-109
lvm2: 2.02.98-pve4
clvm: 2.02.98-pve4
corosync-pve: 1.4.7-1
openais-pve: 1.1.4-3
libqb0: 0.11.1-2
redhat-cluster-pve: 3.2.0-2
resource-agents-pve: 3.9.2-4
fence-agents-pve: 4.0.10-2
pve-cluster: 3.0-16
qemu-server: 3.3-20
pve-firmware: 1.1-3
libpve-common-perl: 3.0-24
libpve-access-control: 3.0-16
libpve-storage-perl: 3.0-31
pve-libspice-server1: 0.12.4-3
vncterm: 1.1-8
vzctl: 4.0-1pve6
vzprocps: 2.0.11-2
vzquota: 3.1-2
pve-qemu-kvm: 2.1-12
ksm-control-daemon: 1.1-1
glusterfs-client: 3.5.2-1

 

[udo]

Is there Change in RAM Handling in 3.4?

Since i Upgraded, it is not possible to start the same number of KVM-VM's -> the RAM usage increases, the Host slow down and after a few Minutes the Host crash completly.

Can I test the previous Version of QEMU anyway?
Or what could even be the reason for this? I've tested the previous Kernel, but there is the same Problem.

Hi,
sounds that you overcommit your server a lot and now there are different ksm-settings?

What is the output of following commands:

cat  /sys/kernel/mm/ksm/pages_shared 

cat  /sys/kernel/mm/ksm/pages_sharing

cat  /sys/kernel/mm/ksm/pages_unshared

free

Udo

 

[trendco]

Ok, i started 4 VM's:

- Nas4Free -> 10GB assigned
- Ubuntu -> 2GB assigned
- Linux/Debian -> 2GB assigned
- Linux/Debian -> 2GB assigned

Webinterface Output:

RAM-Usage    Total: 15,63GB
              Used: 14.26GB

cat  /sys/kernel/mm/ksm/pages_shared
0

cat  /sys/kernel/mm/ksm/pages_sharing
0

cat  /sys/kernel/mm/ksm/pages_unshared
0

free

             total       used       free     shared    buffers     cached
Mem:      16386068   15106580    1279488          0      34084     120804
-/+ buffers/cache:   14951692    1434376
Swap:      7733244          0    7733244

If i now start a Windows-7 VM (2GB assigned), it happens -> Host slow down (can't do anything in Webinterface or Console), a few Minutes Later -> Crash.

 

[trendco]

It looks like overcommit no longer works (for me).
Is this managed by Qemu?
How can i Test the previous Version?

 

[udo]

Ok, i started 4 VM's:

- Nas4Free -> 10GB assigned
- Ubuntu -> 2GB assigned
- Linux/Debian -> 2GB assigned
- Linux/Debian -> 2GB assigned

Webinterface Output:

RAM-Usage    Total: 15,63GB
              Used: 14.26GB

cat  /sys/kernel/mm/ksm/pages_shared
0

cat  /sys/kernel/mm/ksm/pages_sharing
0

cat  /sys/kernel/mm/ksm/pages_unshared
0

free

             total       used       free     shared    buffers     cached
Mem:      16386068   15106580    1279488          0      34084     120804
-/+ buffers/cache:   14951692    1434376
Swap:      7733244          0    7733244

If i now start a Windows-7 VM (2GB assigned), it happens -> Host slow down (can't do anything in Webinterface or Console), a few Minutes Later -> Crash.

Hi,
edit your /etc/ksmtuned.conf

KSM_ENABLED=1
KSM_THRES_COEF=40

and restart ksm

service ksmtuned restart

And wait some time...

Udo

 

[trendco]

This is my "ksmtuned.conf", file date 15.03.13, which means the last time there was no Changes. But it worked definitely before the Update.

# Configuration file for ksmtuned.

# How long ksmtuned should sleep between tuning adjustments
# KSM_MONITOR_INTERVAL=60

# Millisecond sleep between ksm scans for 16Gb server.
# Smaller servers sleep more, bigger sleep less.
# KSM_SLEEP_MSEC=100

# KSM_NPAGES_BOOST=300
# KSM_NPAGES_DECAY=-50
# KSM_NPAGES_MIN=64
# KSM_NPAGES_MAX=1250

# KSM_THRES_COEF=20
# KSM_THRES_CONST=2048

# uncomment the following if you want ksmtuned debug info

# LOGFILE=/var/log/ksmtuned
# DEBUG=1

 

[trendco]

I checked a old Backup for "ksmtuned.conf", it's exact the same as now.

Once more, is it possible that it is a QEMU Problem? And how can i Test the previous Version?

 

[udo]

I checked a old Backup for "ksmtuned.conf", it's exact the same as now.

Once more, is it possible that it is a QEMU Problem? And how can i Test the previous Version?

Once more, why you don't try "KSM_ENABLED=1 + KSM_THRES_COEF=40"???

 

[trendco]

KSM is enabled by default, see "/etc/default/ksmtuned"

# Defaults for ksmtuned initscript
# sourced by /etc/init.d/ksmtuned
# installed at /etc/default/ksmtuned by the maintainer scripts

#
# This is a POSIX shell fragment
#


# start ksmtuned at boot [yes|no]
START=yes

Once more, why you don't try "KSM_ENABLED=1 + KSM_THRES_COEF=40"???

I have tested your Suggest now, but the Same Problem.

 

[trendco]

My last trial would be to test the previous QEMU Version.

Please can someone explain me how exactly i do that?
Downgrade "pve-qemu-kvm" or "qemu-server" or both? And How exactly?

 

[martin]

you can install packages by using dpkg:

1. wget <package name>
2. dpkg -i <package name>

 

[trendco]

Thanks.

Do I Need "pve-qemu-kvm" or "qemu-server" or both?

 

[tom]

we do not test every combination in our test lab, so just start with downgrading pve-qemu-kvm first

 

[trendco]

I tested, but no Change.

I think overcommit the RAM is no good Idea, now i bought more RAM

Solved (for me)

 

[alatteri]

maybe your Ballon settings got changed in the upgrade?

 

[spirit]

maybe your Ballon settings got changed in the upgrade?

can you type in vm monitor :

#info balloon

and post the result ?