49 = Max VM machines?

JoBlo69 · May 27, 2009

I can create more than 49 vm's, but cannot have more than 49 running at any given time, is this a built in limitation of proxmox? or is there a conf file somewhere i can edit to allow more than 49 simultaneous vm's running at once?

"vzctl start 150" command, does not work in starting the 50th slice, 150 being the VMID for the 50th slice..

My PC i am running proxmox in, I have configured to where i can have 200 VM's running. The way I built this PC, and the way i have allocated the resources across each slice, this PC should easily handle 200 VM's running at once. I would really like the ability to run more than 49 slices on any given PC. Is there a licensed version of proxmox that will handle more than 49 VM's? if there is, that would be a valid option in solving my issue.

And just to add, this issue, if resolved, would be worth a sizable donation to your project, or if there was a pay version, that allows more than 49 slices.

Thanks.

dietmar · May 28, 2009

There is no limit. Any error messages, logs?

JoBlo69 · May 28, 2009

dietmar said:
There is no limit. Any error messages, logs?

Here is part of the log file...

Code:

2009-05-28T06:23:55+0000 vzctl : CT 149 : Stopping container ...
2009-05-28T06:23:55+0000 vzctl : CT 149 : Container was stopped
2009-05-28T06:23:55+0000 vzctl : CT 149 : Container is unmounted
2009-05-28T06:24:06+0000 vzctl : CT 149 : Starting container ...
2009-05-28T06:24:06+0000 vzctl : CT 149 : Container is mounted
2009-05-28T06:24:06+0000 vzctl : CT 149 : Adding IP address(es): 10.0.0.148
2009-05-28T06:24:08+0000 vzctl : CT 149 : Setting CPU units: 1000
2009-05-28T06:24:08+0000 vzctl : CT 149 : Setting CPUs: 1
2009-05-28T06:24:08+0000 vzctl : CT 149 : Configure meminfo: 262144
2009-05-28T06:24:08+0000 vzctl : CT 149 : Set hostname: 1149.ilovemywimax.com
2009-05-28T06:24:08+0000 vzctl : CT 149 : File resolv.conf was modified
2009-05-28T06:24:08+0000 vzctl : CT 149 : Setting quota ugidlimit: 0
2009-05-28T06:24:08+0000 vzctl : CT 149 : Container start in progress...
2009-05-28T06:24:24+0000 vzctl : CT 150 : Starting container ...
2009-05-28T06:24:24+0000 vzctl : CT 150 : Container is mounted
2009-05-28T06:24:24+0000 vzctl : CT 150 : Container start failed
2009-05-28T06:24:24+0000 vzctl : CT 150 : Container is unmounted

Not sure where to look for an error message to locate exactly what the issue is...

JoBlo69 · May 28, 2009

This is the log file that's available from the web interface...

Note that is says that permission is denied when it tries to mount..

And to add, that i have more than 75 VM's on my proxmox, and i can stop and start any combination of them, and can only have a total of 49 running at any given time. It doesnt seem to matter what 49 are running, it's once you try and start a 50th slice, it fails..

Code:

starting init logger
 * Setting preliminary keymap...         [ OK ] 
 * Starting basic networking...         [ OK ] 
 * Loading kernel modules...         * Loading manual drivers...         [ OK ] 
 * Setting kernel variables...         [ OK ] 
 * Activating swap...         [ OK ] 
mount: permission denied 
 * Mounting local filesystems...         [ OK ] 
 * Activating swapfile swap...         [ OK ] 
 * Checking minimum space in /tmp...         [ OK ] 
 * Skipping firewall: ufw (not enabled)...         [ OK ] 
 * Configuring network interfaces...         [ OK ] 
 * Starting system log daemon...         [ OK ] 
generating ssh host keys 
 * Starting OpenBSD Secure Shell server sshd         [ OK ] 
 * Starting Postfix Mail Transport Agent postfix        postfix: warning: valid_hostname: numeric hostname: 1150 
postfix: fatal: unable to use my own hostname 
 [fail] 
 * Starting deferred execution scheduler atd         [ OK ] 
 * Starting periodic command scheduler crond         [ OK ] 
 * Running local boot scripts (/etc/rc.local)         [ OK ] 
 * Stopping Postfix Mail Transport Agent postfix         [ OK ] 
 * Stopping firewall: ufw...         [ OK ] 
 * Unmounting any overflow tmpfs from /tmp...         [ OK ] 
 * Terminating all remaining processes...        
init logger finished

dietmar · May 28, 2009

That 'mount' error is expected.

Please try to start manually with:

# vzctl --verbose start CTID

any errors?

JoBlo69 · May 28, 2009

I just now tried that command, and i still get a similar error message..

I stopped the 49th slice through the web interface, and ran this command on 149 and then on 150.

Code:

proxmox3:~# vzctl --verbose start 149
Starting container ...
Container is mounted
Adding IP address(es): 10.0.0.148
Running container script: /etc/vz/dists/scripts/debian-add_ip.sh
Setting CPU units: 1000
Setting CPUs: 1
Configure meminfo: 262144
Set hostname: 1149.ilovemywimax.com
Running container script: /etc/vz/dists/scripts/debian-set_hostname.sh
Running container script: /etc/vz/dists/scripts/set_dns.sh
File resolv.conf was modified
Setting quota ugidlimit: 0
Running container script: /etc/vz/dists/scripts/set_ugid_quota.sh
Container start in progress...
proxmox3:~# vzctl --verbose start 150
Starting container ...
Container is mounted
Container start failed
Container is unmounted
proxmox3:~#

and to add that i have installed Proxmox onto several different PC's and this is 100% consistent along all my servers, not just isolated to this specific PC.

i have tried this on both 1.1 and 1.2 and still get the same error message after trying to turn on the 50th slice..

/usr/sbin/vzctl start 150
Starting container ...
Container is mounted
Container start failed
Container is unmounted
VM 150 start failed -

JoBlo69 · May 28, 2009

I just now tried that command, and i still get a similar error message..

I stopped the 49th slice through the web interface, and ran this command on 149 and then on 150.

Code:

proxmox3:~# vzctl --verbose start 149
Starting container ...
Container is mounted
Adding IP address(es): 10.0.0.148
Running container script: /etc/vz/dists/scripts/debian-add_ip.sh
Setting CPU units: 1000
Setting CPUs: 1
Configure meminfo: 262144
Set hostname: 1149.ilovemywimax.com
Running container script: /etc/vz/dists/scripts/debian-set_hostname.sh
Running container script: /etc/vz/dists/scripts/set_dns.sh
File resolv.conf was modified
Setting quota ugidlimit: 0
Running container script: /etc/vz/dists/scripts/set_ugid_quota.sh
Container start in progress...
proxmox3:~# vzctl --verbose start 150
Starting container ...
Container is mounted
Container start failed
Container is unmounted
proxmox3:~#

And to add that i have installed Proxmox onto several different PC's and this is 100% consistent along all my servers, not just isolated to this specific PC.

I have been trying to locate the file that contains the 'create', and 'start' functions, to see if i can track down what the situation might me. If you could point me to anything you think is relevant to this problem, i could take a stab ad figuring it out. I did find the 'create' function in the /usr/bin/pvectl file, but an having issues on locating the 'start' function, to see if there is any code that i could tweak on to get this to work.

If anyone else wants to try and create this scenario i have running to verify that it is a real issue, or if its just me, that would be great.

Thanks again.

jens · May 28, 2009

Can you post a cat /proc/meminfo of the host and - assuming all your VMs are configured identical - a copy of /etc/vz/conf/101.conf

JoBlo69 · May 29, 2009

Here is the cat /proc/meminfo

Code:

MemTotal:      2057976 kB
MemFree:         26164 kB
Buffers:        352076 kB
Cached:        1244164 kB
SwapCached:          4 kB
Active:         597636 kB
Inactive:      1150868 kB
SwapTotal:     4194296 kB
SwapFree:      4194240 kB
Dirty:               4 kB
Writeback:           0 kB
AnonPages:      152284 kB
Mapped:          85412 kB
Slab:           230600 kB
SReclaimable:   177836 kB
SUnreclaim:      52764 kB
PageTables:       8472 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:   5223284 kB
Committed_AS:  1487284 kB
VmallocTotal: 34359738364 kB
VmallocUsed:     20872 kB
VmallocChunk: 34359717468 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
HugePages_Surp:      0
Hugepagesize:     2048 kB

and here is the /etc/vz/conf/101.conf

Code:

# PVE default config for 256MB RAM

ONBOOT="yes"

# Primary parameters
NUMPROC="1024:1024"
NUMTCPSOCK="9223372036854775807:9223372036854775807"
NUMOTHERSOCK="9223372036854775807:9223372036854775807"
VMGUARPAGES="262144:9223372036854775807"

# Secondary parameters
KMEMSIZE="9223372036854775807:9223372036854775807"
OOMGUARPAGES="262144:9223372036854775807"
PRIVVMPAGES="262144:274644"
TCPSNDBUF="9223372036854775807:9223372036854775807"
TCPRCVBUF="9223372036854775807:9223372036854775807"
OTHERSOCKBUF="9223372036854775807:9223372036854775807"
DGRAMRCVBUF="9223372036854775807:9223372036854775807"

# Auxiliary parameters
NUMFILE="9223372036854775807:9223372036854775807"
NUMFLOCK="9223372036854775807:9223372036854775807"
NUMPTY="255:255"
NUMSIGINFO="1024:1024"
DCACHESIZE="9223372036854775807:9223372036854775807"
LOCKEDPAGES="131072:131072"
SHMPAGES="9223372036854775807:9223372036854775807"
NUMIPTENT="9223372036854775807:9223372036854775807"
PHYSPAGES="0:9223372036854775807"

# Disk quota parameters
DISKSPACE="524288:576716"
DISKINODES="100000:110000"
QUOTATIME="0"
QUOTAUGIDLIMIT="0"

# CPU fair sheduler parameter
CPUUNITS="1000"
CPUS="1"
VE_ROOT="/var/lib/vz/root/$VEID"
VE_PRIVATE="/var/lib/vz/private/$VEID"
OSTEMPLATE="debian-5.0-standard_5.0-1_i386"
ORIGIN_SAMPLE="pve.auto"
IP_ADDRESS="10.0.0.101"
HOSTNAME="101.com"
DESCRIPTION="Debian 5.0 (standard)"
NAMESERVER="10.0.0.1"
SEARCHDOMAIN="com"

And to note, each slice is configured with exactly the same resources. Also all these logs are from a test server i put together, to show that every PC i have installed proxmox on, i always hit that 49 active VM limit.. I have installed proxmox on over 20 different hardware configurations, everything from quad core intel rigs with 16gm of memory and a 2TB+ raid and down to a single core amd with 2gb or memory with a 500gb hard drive, they all hit this 49 VM limit... both with proxmox 1.1 and 1.2

dietmar · May 29, 2009

JoBlo69 said:
Code:

MemTotal: 2057976 kB

So this Host has 2GB RAM?

And you are trying to run 50VM each using 512MB+512Swap ?

Note: 512MB*50 = 25GB

- Dietmar

JoBlo69 · May 29, 2009

dietmar said:
So this Host has 2GB RAM?

And you are trying to run 50VM each using 512MB+512Swap ?

Note: 512MB*50 = 25GB

- Dietmar

? Thats what it says right??

Are you saying that it will not work because i have to many resources allocated? I thought proxmox doesn't directly allocate resources, hence why i am able to run 49 VM's just fine. Once i get to that 50th one, it doesn't work...

Again, the logs i am posting here are from a test bed server. What would you like to see exactly in a log file to finally belive that this is an issue? What would satisfy you in a hardware, and VM configuration, before you believe what i am saying?

I don't understand what more you want from me. There is an obvious problem, and i can cut and paste all sorts of logs from all my servers, with various hardware and software configurations, and every single one stops at 49 slices. Every one.

And also, what i am trying to say here is no matter what the configuration is, 49 is the max. I understand that you say there is no limit. Clearly there is. Not that i want to argue, just that its there. I see it every time i build a server. Could you configure a PC with proxmox, and try it out yourself? I am 100% sure you will have exactly the same problem i am having.

Please prove me wrong. That's why i am here. What am i doing wrong? This problem happens on every single proxmox server i build, no matter the resources. its hard to keep thinking is on my side. I can re allocate each slice i have running on this slice to 1gb of ram and 49 will still work fine. Again, showing that proxmox does not directly allocate the hardware across its VM's.

It seems that you feel there isn't this limitation. If you could post some logs showing me that you can run more than 49 slices on a PC, that would be great.

How about this. I challenge anyone on this forum to build a proxmox server and run more than 50 active slices. I bet no one will be able to. I feel, from all my testing, I have depleted my ability to figure this out.

I do hate that I feel I may be upsetting, but the issue is there. What do u want before your satisfied that its an issue? That's why i ask that you build a server with 50 active running VM's and prove me wrong. I really do want to be wrong. I want to run more than 49 VM's on a few of my servers.

And again, anyone else offering to pay to have there issues looked at in more depth? I have...

I dont know what else to say here. Help me or not. I'll just move over to openvz, if i cant get any help here. It would be nice to have the web interface from proxmox, but if the support isnt there, i'll just move on to the next thing...

dietmar · May 29, 2009

Please test on a mashine with enough RAM (>50GB for what you want to do). Or use a VM setup which uses less RAM. I will try to reproduce if the error is still there).

dietmar · May 29, 2009

OK, we are able to reproduce the bug here - I will take a look at it next week.

JoBlo69 · May 29, 2009

Thanks.

I'll keep an eye out for any new posts in this thread over the next week.

If you want to post any relevant info you may think is the issue here, I will not have any problem spending some time helping on getting this resolved.

JoBlo69 · May 29, 2009

Also, I started a thread over in openvz, with the thought that this might be a openvz issue, and not promox related...

http://forum.openvz.org/index.php?t=tree&goto=36213&#msg_36213

If anyone is interested. I have no idea, so this might be irrelevant..

dietmar · May 29, 2009

The real question is why you can allocate 49 VMs - Overcommits should not be allowed at all with our system.

foxtroop11 · May 30, 2009

It appears to be a kernel issue. Hopefully the guys at Proxmox could recompile it after looking at this link.

http://forum.openvz.org/index.php?t=msg&goto=35871&&srch=ves#msg_35871

Your kernel has CONFIG_KEYS turned on. Recompile with this option unset and you'll be fine.

See
http://bugzilla.openvz.org/show_bug.cgi?id=945
http://bugzilla.openvz.org/show_bug.cgi?id=1242
for more details

dietmar · May 31, 2009

Thank for finding that bug. I will disable that in the next version.

- Dietmar

foxtroop11 · May 31, 2009

Sounds great. I was beating my head against the wall last night trying to recompile and do it myself. Finally called it quits and then saw this message so I'll just wait.

smilie · Jun 2, 2009

@Dietmar:

"The real question is why you can allocate 49 VMs - Overcommits should not be allowed at all with our system."

Could you please provide some more info on this?

- how would overcommit manifest itself?
- why have you chosen not to allow this (ok, answer to this can be probably debated for a few days, so I promise not to argue your cause)

Thanx!

Cheers,
Aleks

49 = Max VM machines?

JoBlo69

Guest

Proxmox Staff Member

JoBlo69

Guest

JoBlo69

Guest

Proxmox Staff Member

JoBlo69

Guest

JoBlo69

Guest

New Member

JoBlo69

Guest

Proxmox Staff Member

JoBlo69

Guest

Proxmox Staff Member

Proxmox Staff Member

JoBlo69

Guest

JoBlo69

Guest

Proxmox Staff Member

New Member

Proxmox Staff Member

New Member

New Member