Hi,
We are looking into Proxmox and are testing at the moment.
We need a redundant/failover system for around 100 VMs to start with. These are currently running on an old Xen cluster and we think about moving to Proxmox/KVM.
Our test configuration: 3 Hosts, each 24 cores and 128GB RAM, redundant connection to a dual-controller HA-SAS-storage system.
I set up Proxmox on the three hosts, configured multipathing for the storage and can create and migrate around VMs now - with some glitches and questions popping up:
some questions:
1. ISOs
When I install a new VM from an iso file, I can add it to the HA group, but it won't migrate/start on another node. I found the reason is the iso file missing on the other nodes.
Is this supposed to be like that or a bug? Looks like I can't put isos on a shared storage and the local iso folder is not being synced inbetween the nodes. It's just a matter of time until this would cause problems because somebody will forget to unlink the iso after installation and some important VM will not start in the case of a failover...
2. VNC
remote control is not working from neither a Ubuntu 13.10 client nor a Mac OS X client. This makes the console ununsable. Is there a fix? It is no option to have the need of special Java versions or such on the clients... to many systems, home offices and such.
3. failover domains
From what I understand, we need to set these up manually and also maintain them by hand in the cluster.conf? This is needes for any professional use and I would have expected to be able to do this via the GUI?
Our current idea to solve these problems is a simple script/daemon that monitors and modifies the following things:
(please give me some feedback if this sounds like nonsense or is actually a good approach)
1. monitor the template- and iso-folders and sync them between the hosts (inotify, rsync and such)
2. monitor /etc/pve/nodes/*/qemu-server/*.conf and add "args: -vnc 1.2.3.4:MACHINENUMBER" if missing. These changes should then automatically be synced to all other nodes, right? (at least it looks like that). Or will I have to use some API for these changes? This is not an optimal solution, but everyday-access to the VMs will be SSH (linux VMs) or rdesktop (windows VMs) anyway.
3. We don't need Pools for user rights or such, so my idea is the following: I would divert Pools to group my VMs to failover domains. So a Pool would be a failover domain and I could do the grouping of VMs via the GUI. I would then monitor changes in /etc/pve/cluster.conf and user.cfg: if a ne pvevm-entry pops up/changes, look it up in the /etc/pve/user.cfg and add domain="MyPoolName" to the pvevm-tag
ipmi-based fencing seems to work, but I haven't tested the following yet:
- bonding & vlans, but I assume once I set up vlans/bridges the "debian way", the interfaces will pop up in proxmox and I can assign them to the VMs NICs?
Thanks
We are looking into Proxmox and are testing at the moment.
We need a redundant/failover system for around 100 VMs to start with. These are currently running on an old Xen cluster and we think about moving to Proxmox/KVM.
Our test configuration: 3 Hosts, each 24 cores and 128GB RAM, redundant connection to a dual-controller HA-SAS-storage system.
I set up Proxmox on the three hosts, configured multipathing for the storage and can create and migrate around VMs now - with some glitches and questions popping up:
some questions:
1. ISOs
When I install a new VM from an iso file, I can add it to the HA group, but it won't migrate/start on another node. I found the reason is the iso file missing on the other nodes.
Is this supposed to be like that or a bug? Looks like I can't put isos on a shared storage and the local iso folder is not being synced inbetween the nodes. It's just a matter of time until this would cause problems because somebody will forget to unlink the iso after installation and some important VM will not start in the case of a failover...
2. VNC
remote control is not working from neither a Ubuntu 13.10 client nor a Mac OS X client. This makes the console ununsable. Is there a fix? It is no option to have the need of special Java versions or such on the clients... to many systems, home offices and such.
3. failover domains
From what I understand, we need to set these up manually and also maintain them by hand in the cluster.conf? This is needes for any professional use and I would have expected to be able to do this via the GUI?
Our current idea to solve these problems is a simple script/daemon that monitors and modifies the following things:
(please give me some feedback if this sounds like nonsense or is actually a good approach)
1. monitor the template- and iso-folders and sync them between the hosts (inotify, rsync and such)
2. monitor /etc/pve/nodes/*/qemu-server/*.conf and add "args: -vnc 1.2.3.4:MACHINENUMBER" if missing. These changes should then automatically be synced to all other nodes, right? (at least it looks like that). Or will I have to use some API for these changes? This is not an optimal solution, but everyday-access to the VMs will be SSH (linux VMs) or rdesktop (windows VMs) anyway.
3. We don't need Pools for user rights or such, so my idea is the following: I would divert Pools to group my VMs to failover domains. So a Pool would be a failover domain and I could do the grouping of VMs via the GUI. I would then monitor changes in /etc/pve/cluster.conf and user.cfg: if a ne pvevm-entry pops up/changes, look it up in the /etc/pve/user.cfg and add domain="MyPoolName" to the pvevm-tag
ipmi-based fencing seems to work, but I haven't tested the following yet:
- bonding & vlans, but I assume once I set up vlans/bridges the "debian way", the interfaces will pop up in proxmox and I can assign them to the VMs NICs?
Thanks
