Choosing a storage type for LXC containers in an HA cluster

Thane_DE

New Member
Oct 19, 2017
6
0
1
26
Hello there,

I've been experimenting with the Proxmox HA stack in my lab for the last few months, but I'm now at a point where I need some advice on how to move forward. For reference, I'm running a small homelab with mostly consumer-grade hardware/used server stuff from a few years back. My nodes are all running Sandy-Bridge 4C8T CPUs and 16GB of RAM, my NAS/"SAN" is a low-power FreeNAS Box with a dual-core and 8GB of ECC RAM. The nodes are connected to a central switch via Gigabit-Ethernet, the FreeNAS box has 2-port LAGG bond to the same switch.

As for software, I'm mostly using LXC containers (~20 total) on top of Proxmox 5.2, with a few VMs thrown in. The root filesystems are currently on a NFS share provided by the FreeNAS box, using 2 SSDs in a mirror ZFS array for storage. Not exactly terrific, I know. In fact, the storage solution has given me a few headaches recently and I'm looking for a more elegant and more consistent solution. Essentially, I need a shared FS that can hold both VMs and LXC containers (and can preferably do snapshots)

Looking at this handy storage table, it looks like there are a few options that fit my requirements:
  • NFS/CIFS - What I'm using right now. It works, except when it doesn't
  • iSCSI - Doesn't seem to support LXC containers directly, same for ZFS over iSCSI. Guess that makes sense, given that iSCSI devices are pure block devices
  • LVM on iSCSI - Seems to fit my criteria, but I'm not sure about the stability
  • ZFS over iSCSI - Not sure if this is possible at all
  • Ceph - Would need to rebase my VM storage away from FreeNAS, but it seems to be a decent solution otherwise, if somewhat overkill
I'm honestly not sure what's the best way forward here. My experience with NFS as a pseudo-SAN hasn't exactly been great - no surprise given that it's really not intended for this use case.

Something iSCSI-based sounds interesting, but I'm not sure about the stability: How well does LVM over iSCSI work within a cluster of nodes? Any issues with nodes updating or anything like that? If anyone has any experiences, I'd be very happy if they shared them.

Ceph seems like a good solution, but perhaps a bit "overkill" for my setup. Since FreeNAS doesn't support it and I'm not planning on adding another box for storage, I'd probably use the built-in PVE Ceph server.

I'm curious to hear what you think. Any suggestions would be appreciated. Cheers!
 
LXC with Ceph, basically mount images with using krbd module, and container using the /dev/rbdX devices. IF you don't have anyway ceph cluster, there is no advantages on this setup.
 
When you have single point of failures in the system on hardware level, you can not do HA.
And there are quite a few of these in your system: single switch, single storage server, single connections from compute nodes, etc..

In my personal opinion your current setup will never be HA, and will be even less reliable than single nodes with local storage.
I would simplify and drop external storage and use ZFS sync (GUI option nowdays). If really needed, you can still enable HA on synced VMs, but i would rather make sure that the source node is dead manually and then manually power up the VMs on the new node.

Good luck.
 
When you have single point of failures in the system on hardware level, you can not do HA.
And there are quite a few of these in your system: single switch, single storage server, single connections from compute nodes, etc..

In my personal opinion your current setup will never be HA, and will be even less reliable than single nodes with local storage.
I would simplify and drop external storage and use ZFS sync (GUI option nowdays). If really needed, you can still enable HA on synced VMs, but i would rather make sure that the source node is dead manually and then manually power up the VMs on the new node.

Good luck.
(Not OP) you still cant zfs sync live VMs, right? Atleast I still couldnt get it to work
 
You can sync running VMs.

Qemu agent is told to freeze FS operations, then snapshot is made, then fs unfreeze, then syncing from snapshot to another node occurs.
Smallest interval is one minute. While this is not true HA, it is a really fast RTPO.
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!