Hi,
for Ceph, you need at least 3 nodes: https://pve.proxmox.com/pve-docs/chapter-pveceph.html#_recommendations_for_a_healthy_ceph_cluster
for Ceph, you need at least 3 nodes: https://pve.proxmox.com/pve-docs/chapter-pveceph.html#_recommendations_for_a_healthy_ceph_cluster
I came across this searching for the error I saw and realized the issue for me. I know this is a bit of a necro but.... I have a single node setup (using ceph for domain failure of osd) and in migrating from an old ubuntu OS I backed up everything to a single disk and bought a new os disk. This gives the ability to go back if my installation and configuration on debian fails. I did not want to invest in extra disks just to migrate.
DISCLAIMER: The solution below will put your data at risk of a single point of failure.
Proxmox cannot acquire a lock on the storage if it cannot write to the storage in the first place. If you have 1 OSD and a replica size of 2 you cannot write to the pool. The solution to 'healthy' PGs for your ceph osd pool is to set the min_size and size of the pool to "1". This gives you 0 redundancy and high risk of losing data but will allow you to use the pool. Just don't expect this configuration to perform well, come up after a power outage, survive a mobo failure, survive a disk failure, or really any failure at all.
ON YOUR SINGLE NODE SETUP:
1. Edit your /etc/ceph/ceph.conf with the below settings to change your minimum replica size:
2. Restart your ceph-mon for the config to take affect
3. Modify the pool you created for use with proxmox to set the replica size with the command line ceph commands:
You should be able to use the pool now that the min_size for the pool is set to 1. If you still get the error or you have just a single osd (like I did in migration testing) you can set the pool size to 1 as well. There is an additional argument for the below command but I leave it out so you have to read it and pay attention to what it says.
4.
With that you can get your pool to show as "healthy" with pgs active and ready. This will allow you to write data to the pool even though it is not resilient at all and proxmox does not allow you to set "unsafe" settings for your pool through the UI.
After all that be sure to add nodes to your ceph cluster or at the very least more disks and change your size settings after doing so to avoid data loss.
DISCLAIMER: The solution below will put your data at risk of a single point of failure.
Proxmox cannot acquire a lock on the storage if it cannot write to the storage in the first place. If you have 1 OSD and a replica size of 2 you cannot write to the pool. The solution to 'healthy' PGs for your ceph osd pool is to set the min_size and size of the pool to "1". This gives you 0 redundancy and high risk of losing data but will allow you to use the pool. Just don't expect this configuration to perform well, come up after a power outage, survive a mobo failure, survive a disk failure, or really any failure at all.
ON YOUR SINGLE NODE SETUP:
1. Edit your /etc/ceph/ceph.conf with the below settings to change your minimum replica size:
Code:
[global]
mon_allow_pool_size_one = true
osd_pool_default_min_size = 1
osd_pool_default_size = 2systemctl restart ceph-mon@<YOURNODE>3. Modify the pool you created for use with proxmox to set the replica size with the command line ceph commands:
ceph osd pool set YOUR_PROXMOX_POOL min_size 1You should be able to use the pool now that the min_size for the pool is set to 1. If you still get the error or you have just a single osd (like I did in migration testing) you can set the pool size to 1 as well. There is an additional argument for the below command but I leave it out so you have to read it and pay attention to what it says.
4.
ceph osd pool set YOUR_PROXMOX_POOL size 1With that you can get your pool to show as "healthy" with pgs active and ready. This will allow you to write data to the pool even though it is not resilient at all and proxmox does not allow you to set "unsafe" settings for your pool through the UI.
After all that be sure to add nodes to your ceph cluster or at the very least more disks and change your size settings after doing so to avoid data loss.