PVE Cluster Share Store

[majorchen]

During the test, it was found that there were 3 OSDs running in CEPH, one OSD network was interrupted or the host was powered off and restarted. All VMs using CEPH need to wait for the OSD heartbeat to be used normally.
In the production environment is not very friendly, in addition to modifying the heartbeat time, is there a better solution?

 

[Romsch]

Hi!
What is your configuration? The informations are a little bit rare.
How many nodes do you have, how many OSDs per host, ceph cluster is in a physically separated network?
ceph.conf? OSD heartbeat?

And please change the topic "Tutorial", i think thats wrong.

regards,
roman

 

[majorchen]

Thank you for pointing out the mistake.
Just a minimal cluster, 3host: 1host only 1osd, CEPH config except pool mini_size=1, othe default config, found in mysql imprort data for single point of failure test can not be written.

 

[Romsch]

with three nodes and ceph min size 1 will not correctly work i guess. Make standard, size = 3 and min size = 2. If you change e. g. the size to 2 and migrate a vm, the ceph cluster is in read only mode, because of the size 2 in a three node cluster.

 

[majorchen]

After modifying min_site, the test results are the same as before. It is necessary to wait for the osd heartbeat. It is considered that the problematic OSD is DOWN to resume writing.

 

[Romsch]

What you mean with osd heartbeat? OSDs for Ceph should have no raid! this works not so good.. for ceph you must have all disk connectet via sata port. Raid is a bottleneck for ceph

 

[majorchen]

Well, as you can see from this picture, do you have a shared storage solution that can be used in a production environment?

 

[majorchen]

Would like to ask, can you set the Pool to make all Primary dg in an osd?

 

[Romsch]

Well, as you can see from this picture, do you have a shared storage solution that can be used in a production environment?

I dont know what you wanna say with this picture, but yes of course, ceph is a shared storage solution that can be used in a production environment. We do that since more than five years!

Read this about ceph about RAID..

Avoid RAID
As Ceph handles data object redundancy and multiple parallel writes to disks (OSDs) on its own, using a RAID controller normally doesn’t improve performance or availability. On the contrary, Ceph is designed to handle whole disks on it’s own, without any abstraction in between. RAID controller are not designed for the Ceph use case and may complicate things and sometimes even reduce performance, as their write and caching algorithms may interfere with the ones from Ceph.


From: https://pve.proxmox.com/pve-docs/chapter-pveceph.html


Best regars, roman