CEPH is stopped before pve-manager during shutdown of node

casparsmit

Active Member
Feb 24, 2015
38
1
28
Hi all,

When I shutdown a pve node (using the GUI, select the node and then select the shutdown button in the upper right) I noticed that one of the first services to actually stop is CEPH. CEPH is stopped before pve-manager for instance. When pve-manager is stopped, it tries to cleanly shutdown any VM/CT still running on the node.

However when these still running VM/CT are using CEPH storage and CEPH is running locally (My test system is only a 1 node CEPH with 1 mon and 3 osds) this results in VM's/CT's unable to shutdown anymore because the storage got pulled.

I know there are supposed to be more nodes in a CEPH cluster so in all other cases this would probably work because CEPH could still reach the other remote nodes for its storage but still the LOCAL CEPH daemons are killed before pve-manager, it would be a lot nicer if those CEPH daemons would be stopped after all services depending on it (like pve-manager in the case VM's are still running on CEPH storage).

It probably has something to do with the CEPH daemons not using systemd (but SysV compatibility mode).

I tried adding these to the pve-manager.service systemd file to no avail.

Requires=ceph.service
After=ceph.service

Is there any other way not to shutdown CEPH so early in the shutdown process so that running VM's can shutdown cleanly?

Thanks for any advice in advance and kind regards,
Caspar
 

casparsmit

Active Member
Feb 24, 2015
38
1
28
Sorry forgot the version info:

# pveversion -v

proxmox-ve: 4.1-39 (running kernel: 4.2.8-1-pve)
pve-manager: 4.1-15 (running version: 4.1-15/8cd55b52)
pve-kernel-4.2.8-1-pve: 4.2.8-39
lvm2: 2.02.116-pve2
corosync-pve: 2.3.5-2
libqb0: 1.0-1
pve-cluster: 4.0-33
qemu-server: 4.0-62
pve-firmware: 1.1-7
libpve-common-perl: 4.0-49
libpve-access-control: 4.0-11
libpve-storage-perl: 4.0-42
pve-libspice-server1: 0.12.5-2
vncterm: 1.2-1
pve-qemu-kvm: 2.5-8
pve-container: 1.0-46
pve-firewall: 2.0-18
pve-ha-manager: 1.0-23
ksm-control-daemon: 1.2-1
glusterfs-client: 3.5.2-2+deb8u1
lxc-pve: 1.1.5-7
lxcfs: 2.0.0-pve1
cgmanager: 0.39-pve1
criu: 1.6.0-1
zfsutils: 0.6.5-pve7~jessie

# ceph -v

ceph version 0.80.11 (8424145d49264624a3b0a204aedb127835161070)
 

bear

New Member
Apr 13, 2017
1
0
1
Sweden
Hi casparsmit,

I just ran into the same problem as you. Did you ever find a solution to this?

Kind regards
Björn
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get your own in 60 seconds.

Buy now!