Nutanix vs Proxmox with Ceph

offerlam

Renowned Member
Dec 30, 2012
218
0
81
Denmark
Hi all,

I'm toying with the idea of starting a hosting branch in my company. Maybe within the next 5 to 6 years.
Many hosting companies in Denmark are switching to Nutanix.

I was therefore considering building a business case for Proxmox with Ceph in stead. Competing on price since i won't have the support licens cost and forced to buy pricey hardware. I would buy a proxmox support licens but since this is only in working hours i would also need the 5 to 6 years to develop my technical skills with Proxmox and Ceph along with traning the staff we already have in how it works.

But can you make a case that proxmox with Ceph is "just as good" as running with a business product like Nutanix?
 
I don't know Nutanix too well to make a real statement. But Nutanix will fundamentally be no different than NetApp or EMC. So you have dedicated storage that uses manufacturer-specific components and possibly even protocols. If the performance is no longer enough for you, something breaks or support has expired, then you're usually just left with a big pile of electronic scrap lying around, which usually only brings you 2 - 3% of the purchase price on eBay .I know from EMC that features like Dedup etc. were directly integrated and worked well. The tiering issue was also problem-free. So I'm not saying that such storage systems are bad or junk - but you are simply locked in the world of this manufacturer and no longer free to use the product.

I am a very big friend of CEPH, so you will mostly hear positive things from me. First and foremost, you have to get an idea of the solutions yourself, because you will be responsible for them later. So you should look for a solution that you feel comfortable with and can handle.

At CEPH you have the great advantage that you can: For example, you can set up an HCI setup so that you can use the same nodes for computing, network and storage and you don't need any additional infrastructure to provide the storage. Especially when it comes to hosting, price is important, so that's definitely a plus point for CEPH.

With CEPH you can basically use any hardware you want, it will actually always work - whether it's good or bad depends on the components you choose. I would always recommend using Enterprise SSDs / NVMe and switches that can do MLAG (e.g. Arista or Juniper QFX5100), but never Juniper VC or other proprietary nonsense. With manufacturers like EMC or Nutanix, you will always have to purchase the SSDs from the manufacturer so that your support remains intact.

You can configure freely in CEPH, you can save storage space with erasure coding or design the replication of the pools the way you want. CEPH also scales across multiple data centers or server rooms. For example, you can instruct CEPH to have one replication per rack.So you can start with 6 nodes, divide them into 3 racks and adjust the crush map accordingly. You usually can't do that with such solutions from Nutanix, but you'll always need your own controllers in the respective rack.

There was also a function with tiering (primary affinity) in CEPH, but that has actually been eliminated with Flash Only. I would also advise you to do that, at least start with SSDs or directly NVMe. However, if your nodes have enough bays, you can also install SSDs and HDDs in the same node and divide the data between these two classes. CEPH doesn't do any tiering in this sense, but you can decide which data is where and simply move the virtual disk.

But CEPH is also very extensive and complex, you just have to be clear about certain factors. If you have a problem, you usually cannot access a support hotline but are responsible for it yourself. Of course, there are service providers who can help you, but usually no one will come to your location, as could be the case with EMC, for example.You also have to monitor the CEPH storage yourself very extensively in order to be able to react to problems at an early stage. There are tools here and there, but overall you have to set it up for yourself and define threshold values.

But what I've never had in my life with CEPH was data loss. As long as you do not permanently damage the hard drives with physical impact, your data is actually absolutely safe.
 
Hi all,

I'm toying with the idea of starting a hosting branch in my company. Maybe within the next 5 to 6 years.
Many hosting companies in Denmark are switching to Nutanix.

I was therefore considering building a business case for Proxmox with Ceph in stead. Competing on price since i won't have the support licens cost and forced to buy pricey hardware. I would buy a proxmox support licens but since this is only in working hours i would also need the 5 to 6 years to develop my technical skills with Proxmox and Ceph along with traning the staff we already have in how it works.

But can you make a case that proxmox with Ceph is "just as good" as running with a business product like Nutanix?
Hi,
I'm working for an hosting company.
We are running 5000vms with proxmox and around 400TB nvme storage with ceph (promxox since 2010, ceph since 2015).

I never had a single failure with my ceph clusters. So yes, it's just as good than nutanix ^_^

Maybe the only missing feature is vm/workload auto balancing when a node is overloaded.
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!