CEPH is stopped before pve-manager during shutdown of node


Renowned Member
Feb 24, 2015
Hi all,

When I shutdown a pve node (using the GUI, select the node and then select the shutdown button in the upper right) I noticed that one of the first services to actually stop is CEPH. CEPH is stopped before pve-manager for instance. When pve-manager is stopped, it tries to cleanly shutdown any VM/CT still running on the node.

However when these still running VM/CT are using CEPH storage and CEPH is running locally (My test system is only a 1 node CEPH with 1 mon and 3 osds) this results in VM's/CT's unable to shutdown anymore because the storage got pulled.

I know there are supposed to be more nodes in a CEPH cluster so in all other cases this would probably work because CEPH could still reach the other remote nodes for its storage but still the LOCAL CEPH daemons are killed before pve-manager, it would be a lot nicer if those CEPH daemons would be stopped after all services depending on it (like pve-manager in the case VM's are still running on CEPH storage).

It probably has something to do with the CEPH daemons not using systemd (but SysV compatibility mode).

I tried adding these to the pve-manager.service systemd file to no avail.


Is there any other way not to shutdown CEPH so early in the shutdown process so that running VM's can shutdown cleanly?

Thanks for any advice in advance and kind regards,
Sorry forgot the version info:

# pveversion -v

proxmox-ve: 4.1-39 (running kernel: 4.2.8-1-pve)
pve-manager: 4.1-15 (running version: 4.1-15/8cd55b52)
pve-kernel-4.2.8-1-pve: 4.2.8-39
lvm2: 2.02.116-pve2
corosync-pve: 2.3.5-2
libqb0: 1.0-1
pve-cluster: 4.0-33
qemu-server: 4.0-62
pve-firmware: 1.1-7
libpve-common-perl: 4.0-49
libpve-access-control: 4.0-11
libpve-storage-perl: 4.0-42
pve-libspice-server1: 0.12.5-2
vncterm: 1.2-1
pve-qemu-kvm: 2.5-8
pve-container: 1.0-46
pve-firewall: 2.0-18
pve-ha-manager: 1.0-23
ksm-control-daemon: 1.2-1
glusterfs-client: 3.5.2-2+deb8u1
lxc-pve: 1.1.5-7
lxcfs: 2.0.0-pve1
cgmanager: 0.39-pve1
criu: 1.6.0-1
zfsutils: 0.6.5-pve7~jessie

# ceph -v

ceph version 0.80.11 (8424145d49264624a3b0a204aedb127835161070)
Hi casparsmit,

I just ran into the same problem as you. Did you ever find a solution to this?

Kind regards


The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!