Storage woes

mir

Famous Member
Apr 14, 2012
3,570
132
133
Copenhagen, Denmark
Since I hear more and more stories of admins having problems with Ceph I begin to be more and more glad for my ZFS storage. And yes, it is not clustered and high available but it simply keeps on running doing what it is supposed to do with only minimal interaction from my side, it never bothers me with strange errors or behaviors, it never complains about resources or suddenly shows an unaccountable drop in performance. All in all it is simply rock solid and never loose any data:D
 
Of course, a replicated high available distributed storage is more complex to manage than local storage (a SAN is also not that simple) and its still quite new for a lot of admins. And users without issues, do not ask for help here.

But I just want to say that I run a small three node ceph cluster since a year without any major problems. And yes, zfs is also great and you should watch out the upcoming 3.4 release.
 
I totally agree with mir about ZFS. My backup servers are all on ZFS. No issues for months. They "just work" and very reliably.

At the same time, the advantage of distributed storage such as Ceph cannot be ignored specially in an environment where redundancy over multi node is fundamental requirement. What I have noticed over the year is that some level competency "must" be followed when managing a robust distributed storage. From purely my experience, human error remains the #1 cause of all mess up with advanced storage. The recent issue i am facing is nothing but ignoring some basic rules about Ceph and its OSDs. The same cluster has been running for over a year with multiple HDD failures while maintaining data retention and Zero downtime.