Capacity testing with Proxmox 4.4

[wwPhil]

Hi!

I am using Proxmox 4.4 on a IBM X3550 M2 with 8 cores, 128GB ram, and a pair of 15K rpm 146GB drives (raid0).

This server is used to run as many WinXP vms as possible to simulate a high number of clients running our software and swarming the server side of the soft.

The server is part of a cluster, but all vms run from the local lvm storage and are linked clones to a local template.

I have a few issues so far and couldn't find my answers with research on this forum and google.

1. During the cloning process, we use a script that calls pvesh to clone, when the host is fairly busy, we frequently get "500 internal server error", the script launches a clone every 30sec, it doesn't do it in one shot. I have seen others mention this issue but no solution.

2. When the Proxmox host runs 120 vms, the host will be consuming around 40% of its total cpu and around 60GB of ram, plenty of resources left, but for some reason, some of the last vms to boot will have no network connectivity. (using a virtio nic, with configured vlan). Doing a release/renew or disable/enable doesn't help, rebooting the vm will most of the time fix it. We are trying to see if this is a behavior of the virtio drivers or not. Currently attempting to clone 120vms with vmxnet3 instead to verify.

3. When using spice, we couldn't connect to the console of vms beyond the 100th because of "unable to find free port 61000-61099" ? (we only had 1 spice session open)

4. When hosting 120vms (still with plenty of resources left), for example: hotplugging a nic in a vm will usually give error "got timeout", but if you connect to the vm's console, it is very responsive...

5. Any suggestions of tuning I could apply that could help (increase some memory/buffers, change scheduler) ?


Any help is very appreciated,
thank you,

Phil,

 

[zima]

1. Creating 120 clones takes on average 2.7s per clone without pause between them without any problem
i'm using in script:

    qm clone $TEMPLATE $VM_ID
    qm set $VM_ID -net0 virtio=$MAC,bridge=vmbr0,tag=$VLAN

2. Never had problem with network using virtio.
3. I'm not using spice. For me it looks like spice is binding to ports at every instance and you exhaust the range.
in tools.pm

sub next_spice_port {

    my ($family) = @_;
    return next_unused_port(61000, 61099, $family);

 

[wwPhil]

1. Creating 120 clones takes on average 2.7s per clone without pause between them without any problem
i'm using in script:

    qm clone $TEMPLATE $VM_ID
    qm set $VM_ID -net0 virtio=$MAC,bridge=vmbr0,tag=$VLAN

2. Never had problem with network using virtio.
3. I'm not using spice. For me it looks like spice is binding to ports at every instance and you exhaust the range.
in tools.pm

sub next_spice_port {

    my ($family) = @_;
    return next_unused_port(61000, 61099, $family);

1. Thanks, will try that one and report back
2. I was very doubtful that it would be that, and it isn't. I get the same problem using any of the other nic choices. I am now suspecting something along the lines of too many mac addresses registered to the same linux bridge or ethernet port or physical switch port/vlan (each vm has 4 nics). Will continue to work on that and report back.
3. No longer using spice either, but I would love to know if there was a workaround or proper fix for that.

Thanks!