Proxmox 3.2 with 3-nodes cluster and ceph server: running openvz/containers possible?

[Ruben Waitz]

Hi all,

I'm a noob on this subject, but is it possible to run openvz containers (not KVM!) stored on proxmox "ceph" powered nodes?
Maybe with Ceph File System (http://ceph.com/ceph-storage/file-system/) implementation?

Now I'm running the containers on the cluster from a separate NFS server (in DRBD setup), but that has low disk/IO performance.
Also this setup uses vzdump in "suspend" mode to make a container backup. The downtime for containers with many files is quite high (minutes).

Thanks!

Ruben

 

[m.ardito]

Re: Proxmox 3.2 with 3-nodes cluster and ceph server: running openvz/containers possi

I' m not sure, as I'm still on 3.1, but
http://pve.proxmox.com/wiki/Storage_Model
says <<RBD/Ceph: this storage type is only usable to store RAW VM disk images, and needs a working "Monitor host">>

Marco

 

[mo_]

Re: Proxmox 3.2 with 3-nodes cluster and ceph server: running openvz/containers possi

ceph fs is not production stable yet.

You can use Ceph RBD like any other SAN storage: mount one RBD for each node and store its containers in it. This setup won't work with HA though. If you want HA for openvz and Ceph, you need an additional NFS server (or more - for failover) that exports a single RBD volume via NFS. This also makes the NFS server / ceph proxy the bottle neck or potential point of failure of your setup.

Addition for clarification: 1 RBD per Proxmox node for containers is NOT shared storage. an RBD can only ever be mounted to one machine at a time.

 

[Marius Matei]

Re: Proxmox 3.2 with 3-nodes cluster and ceph server: running openvz/containers possi

@Ruben
I ran across the same issue. My solution is simple:
4 client nodes with kernel 3.x (only supports KVM)
3 ceph nodes.
4 KVM instances that have RBD storage which run proxmox.

I've clustered the 4 KVM instances just for migration purposes (If I'll ever need to move a container from one node to another).
Fail-over happens on the top layer, so if a physical node goes down, the corresponding KVM instance will just restart on one of the other 3 nodes. Boot-up is blazing fast.
So now I'm just running KVM products on the physical nodes, and OVZ products on the virtual nodes.
It works perfectly!