3x proxmox node with ceph: how to allow single node mode? Or 2 node needed for ceph at least?

MrBruce

New Member
Mar 7, 2020
25
3
3
Hi!

I installed for 3 node: Proxmox host: 2x1 HDD RAID 1 for host and 1x 1 TB HDD for Ceph.
First node is the strongest, others are slower CPU / less RAM.
node 2 and 3 only for ceph backup, tested with live migration, worked with all of 3 node.
But I want use node 1 as nas, database server, monitoring.
I installing for 2 vm all i need. All in ceph.

All settings by default ceph and proxmox settings.
What I need to do to allow ceph working alone with node 1? I know, its risky and not nice.
Or possible to be node 2 will be same on 24/7. It's better? Will work fine the ceph?

If I turn on node 2 and node 3, it sync back and it's ok, but if node 2 and node 3 disabled, no writes for ceph in node 1.

config parts:

[global]
auth_client_required = cephx
auth_cluster_required = cephx
auth_service_required = cephx
mon_allow_pool_delete = true
...
osd_pool_default_min_size = 2
osd_pool_default_size = 3
...



...




# begin crush map
tunable choose_local_tries 0
tunable choose_local_fallback_tries 0
tunable choose_total_tries 50
tunable chooseleaf_descend_once 1
tunable chooseleaf_vary_r 1
tunable chooseleaf_stable 1
tunable straw_calc_version 1
tunable allowed_bucket_algs 54

# devices
device 0 osd.0 class hdd
device 1 osd.1 class hdd
device 2 osd.2 class hdd

...
...

# rules
rule replicated_rule {
id 0
type replicated
min_size 1
max_size 10
step take default
step chooseleaf firstn 0 type host
step emit
}

# end crush map

...

Thank you.
 
So, if i run in first node more 1 or 2 more Proxmox in vm with dedicated 1tb hdd for ceph, its ok?

So, 1 node config: Proxmox
2x hdd zfs raid 1 for Proxmox host and vm guest storage for node 2 and node 3.
1x 1tb hdd for node 1 ceph.
1x 1 tb hdd for vm node 2 ceph (Proxmox, base vm in zfs raid 1)
1x 1 tb hdd for vm node 3 ceph (Proxmox, base vm in zfs raid 1)

Possible?

Or simply set Size = 3, min size = 1, pg 128 and done? (node1: 1x1TB only)
Or Size = 4, min size = 2, pg 128? (node 1: 2x1TB?)
 
Last edited:

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!