Delaying VM's until ceph/cephFS is mounted

[dlasher]

In larger clusters, it can be quite a few seconds until all the OSD's are happy and cephfs is able to mount. Just restarted one cluster today (power loss) and noticed that while all the KVM's started fine, any LXC that used a CEPHFS bind-mount wouldn't start until cephfs was ready. (got unable to mount messages until it was alive)

Thinking that adding:

ConditionPathIsReadWrite=/mnt/pve/cephfs
ConditionDirectoryNotEmpty=/mnt/pve/cephfs

to

lrwxrwxrwx 1 root root 38 Jul 25  2021 pve-guests.service -> /lib/systemd/system/pve-guests.service

should take care of that dependency. Posting in case it helps others, or in case pmx guys have a better suggestion

 

[gurubert]

We found that adding an override file to the pvestatd.service is sufficient as it will add the correct dependencies.
pve-guests.service is already waiting for pvestatd.service and with this pvestatd.service will wait for pve-storage.target which includes the Ceph services and remote-fs.target.

Put these two lines into /etc/systemd/system/pvestatd.service.d/override.conf or edit the unit file with systemctl edit pvestatd.service:

[Unit]
After=pve-storage.target

Your conditions could also be added to pve-guests.service but I really recommend it doing with systemctl edit.

 

[dlasher]

We found that adding an override file to the pvestatd.service is sufficient as it will add the correct dependencies.
pve-guests.service is already waiting for pvestatd.service and with this pvestatd.service will wait for pve-storage.target which includes the Ceph services and remote-fs.target.

Put these two lines into /etc/systemd/system/pvestatd.service.d/override.conf or edit the unit file with systemctl edit pvestatd.service:

[Unit]
After=pve-storage.target

Your conditions could also be added to pve-guests.service but I really recommend it doing with systemctl edit.

Fantastic, thank you - will give it a shot - The option I suggested above seems to work 80% of the time, but leaves at least one node with no started services.