ceph storage

[Hari Praveen]

Hi team

Please explain how to access proxmox ve storage to another proxmox ve server. Please once guide me. thank you. Please find the attached screenshot.

 

[gurubert]

I would build one cluster with 6 nodes and make sure that VMs are only running on the first three nodes (e.g. with HA groups).

 

[Daniel Keller]

As @gurubert said, build a large proxmoxcluster across all 6 nodes.

the only factor that determines which nodes are storage nodes is where you install the osd and mon services for ceph

 

[Maximiliano]

Are the two sets of nodes physically separated (different rooms, buildings, cities, etc)? If they have a direct connection (say, if they are in the same room or building) you could do a 6-node cluster and only have OSDs in three nodes and VMs in the remaining three nodes. You can set HA rules so that HA guests only migrate to a specific set of nodes, see [2].

If they are separated, but the network latency between them is very low, you can still do it, but I would advice to have a QDevice to ensure at least one half of the cluster keeps operational (you can configure the QDevice to prefer the partition which hosts the VMs) in case you lose communication between both halves of the cluster.

If you want to keep them as two different clusters, you can! You can add a RBD type storage from the web UI, this allows you to use an external Ceph cluster for storage. See [1] for more details.

Do note that for the best performance, the fewer layers you have in between your guest and their storage the better. If performance is critical, you can consider to have a 6-node Ceph cluster and distribute VMs and OSDs among the entire cluster. This is also the most redundant configuration.

[1] https://pve.proxmox.com/pve-docs/pve-admin-guide.html#ceph_rados_block_devices
[2] https://pve.proxmox.com/pve-docs/pve-admin-guide.html#chapter_ha_manager

 

[Hari Praveen]

Thank you guys, thank you so much for your responses.

 

[Hari Praveen]

I successfully communicated with ceph storage. thank you Maximiliano