Ceph stopped before VMs on host shutdown/reboot

[Davyd]

When I reboot host node, VMs cannot shutdown properly. In log I saw this problem:

Jul 26 12:59:08 kspb-vvv3 pvedaemon[5502]: <root@pam> successful auth for user 'root@pam'
Jul 26 12:59:22 kspb-vvv3 pvestatd[1704]: status update time (6.074 seconds)
Jul 26 12:59:31 kspb-vvv3 pvestatd[1704]: status update time (6.077 seconds)
Jul 26 13:00:41 kspb-vvv3 pvestatd[1704]: status update time (6.073 seconds)
Jul 26 13:01:11 kspb-vvv3 pvestatd[1704]: status update time (6.076 seconds)
Jul 26 13:02:11 kspb-vvv3 pvestatd[1704]: status update time (6.076 seconds)
Jul 26 13:02:21 kspb-vvv3 pvestatd[1704]: status update time (6.078 seconds)
Jul 26 13:02:31 kspb-vvv3 pvestatd[1704]: status update time (6.081 seconds)
Jul 26 13:02:41 kspb-vvv3 pvestatd[1704]: status update time (6.074 seconds)
Jul 26 13:03:42 kspb-vvv3 pvestatd[1704]: status update time (6.075 seconds)
Jul 26 13:04:11 kspb-vvv3 pvestatd[1704]: status update time (6.081 seconds)
<Davyd: REBOOT COMMAND>
Jul 26 13:04:28 kspb-vvv3 bash[2596]: 2016-07-26 13:04:28.715866 7fc57eb74700 -1 osd.2 307 *** Got signal Terminated ***
Jul 26 13:04:28 kspb-vvv3 bash[2356]: 2016-07-26 13:04:28.715990 7feee4481700 -1 osd.1 307 *** Got signal Terminated ***
Jul 26 13:04:28 kspb-vvv3 bash[2112]: 2016-07-26 13:04:28.716113 7f22c5f54700 -1 osd.0 307 *** Got signal Terminated ***
Jul 26 13:04:28 kspb-vvv3 bash[1894]: 2016-07-26 13:04:28.716203 7f4dcbfe5700 -1 osd.3 307 *** Got signal Terminated ***
Jul 26 13:04:28 kspb-vvv3 bash[1658]: 2016-07-26 13:04:28.716348 7fb263207700 -1 mon.0@0(leader) e1 *** Got Signal Terminated ***
Jul 26 13:04:28 kspb-vvv3 bash[2596]: 2016-07-26 13:04:28.716568 7fc57eb74700 -1 osd.2 307 shutdown
Jul 26 13:04:28 kspb-vvv3 bash[2112]: 2016-07-26 13:04:28.717829 7f22c5f54700 -1 osd.0 307 shutdown
Jul 26 13:04:28 kspb-vvv3 bash[2356]: 2016-07-26 13:04:28.717962 7feee4481700 -1 osd.1 307 shutdown
Jul 26 13:04:28 kspb-vvv3 ntpd[5369]: ntpd exiting on signal 15
Jul 26 13:04:28 kspb-vvv3 ntp[7759]: Stopping NTP server: ntpd.
Jul 26 13:04:28 kspb-vvv3 pure-ftpd[7762]: Stopping ftp server: pure-ftpd.
Jul 26 13:04:28 kspb-vvv3 postfix/master[1541]: terminating on signal 15
Jul 26 13:04:28 kspb-vvv3 postfix[7765]: Stopping Postfix Mail Transport Agent: postfix.
Jul 26 13:04:29 kspb-vvv3 acpid: exiting
Jul 26 13:04:29 kspb-vvv3 pve-manager[7839]: <root@pam> starting task UPID:kspb-vvv3:00001EA3:00015BBB:579735AD:stopall::root@pam:
Jul 26 13:04:29 kspb-vvv3 pve-manager[7843]: <root@pam> starting task UPID:kspb-vvv3:00001EA4:00015BBC:579735AD:qmshutdown:103:root@pam:
Jul 26 13:04:29 kspb-vvv3 pve-manager[7844]: shutdown VM 103: UPID:kspb-vvv3:00001EA4:00015BBC:579735AD:qmshutdown:103:root@pam:
Jul 26 13:05:30 kspb-vvv3 pveproxy[5283]: worker exit
...

As you can see, osds and mon were stopped before pve-manager started stopall task.
Is there any solution to fix it?

 

[Corey Boyle]

This sounds like a bug. You might want to file a bug report.