shutdown of node shuts down VM

A part from /etc/init.d/rgmanager

### BEGIN INIT INFO
# Provides: rgmanager
# Required-Start: $remote_fs cman
# Required-Stop: $remote_fs cman
# Default-Start: 2 3 4 5
# Default-Stop: 0 1 6
# Short-Description: Starts and stops Red Hat Service (resource group) Manager
# Description: Starts and stops Red Hat Service (resource group) Manager
### END INIT INFO

Shouldn't umountiscsi.sh be listed under Required-Stop?
 
[patch provided with fix] Re: shutdown of node shuts down VM

Hi Deitmar,

Attached patch seems to have fixed this issue:

--- /tmp/rgmanager 2012-11-14 01:01:10.000000000 +0100
+++ /tmp/rgmanager.new 2012-11-14 01:03:50.000000000 +0100
@@ -8,6 +8,7 @@
# Provides: rgmanager
# Required-Start: $remote_fs cman
# Required-Stop: $remote_fs cman
+# Should-Stop: umountiscsi.sh
# Default-Start: 2 3 4 5
# Default-Stop: 0 1 6
# Short-Description: Starts and stops Red Hat Service (resource group) Manager


Nov 14 00:51:56 esx2 task UPID:esx2:00001144:00007264:50A2DD1C:qmshutdown:115:root@
pam:: shutdown VM 115: UPID:esx2:00001144:00007264:50A2DD1C:qmshutdown:115:root@pam
:
Nov 14 00:51:56 esx2 pvestatd[3477]: server closing
Nov 14 00:51:57 esx2 pmxcfs[1834]: [status] notice: RRDC update error /var/lib/rrdc
ached/db/pve2-node/esx1: -3
Nov 14 00:51:58 esx2 rgmanager[4509]: [pvevm] Task still active, waiting
Nov 14 00:51:58 esx2 pvedaemon[2691]: received terminate request
Nov 14 00:51:58 esx2 pvedaemon[2691]: worker 2741 finished
Nov 14 00:51:58 esx2 pvedaemon[2691]: worker 2745 finished
Nov 14 00:51:58 esx2 pvedaemon[2691]: worker 2746 finished
Nov 14 00:51:58 esx2 pvedaemon[2691]: server closing
Nov 14 00:51:59 esx2 rgmanager[4551]: [pvevm] Task still active, waiting
Nov 14 00:52:00 esx2 rgmanager[4574]: [pvevm] Task still active, waiting
Nov 14 00:52:01 esx2 rgmanager[4603]: [pvevm] Task still active, waiting
Nov 14 00:52:02 esx2 rgmanager[4626]: [pvevm] Task still active, waiting
Nov 14 00:52:03 esx2 rgmanager[4649]: [pvevm] Task still active, waiting
Nov 14 00:52:04 esx2 rgmanager[4678]: [pvevm] Task still active, waiting
Nov 14 00:52:05 esx2 rgmanager[4701]: [pvevm] Task still active, waiting
Nov 14 00:52:06 esx2 rgmanager[4725]: [pvevm] Task still active, waiting
Nov 14 00:52:07 esx2 rgmanager[4757]: [pvevm] Task still active, waiting
Nov 14 00:52:07 esx2 kernel: vmbr0: port 2(tap115i0) entering disabled state
Nov 14 00:52:07 esx2 kernel: vmbr0: port 2(tap115i0) entering disabled state
Nov 14 00:52:08 esx2 pvevm: <root@pam> end task UPID:esx2:00001144:00007264:50A2DD1C:qmshutdown:115:root@pam: OK
Nov 14 00:52:08 esx2 rgmanager[2423]: Service pvevm:115 is stopped
Nov 14 00:52:08 esx2 rgmanager[2423]: Disconnecting from CMAN
Nov 14 00:52:09 esx2 pmxcfs[1834]: [status] notice: received log
Nov 14 00:52:10 esx2 pmxcfs[1834]: [status] notice: received log
Nov 14 00:52:23 esx2 rgmanager[2423]: Exiting


The prior bug seems to have broken something:
Nov 14 00:52:24 esx2 kernel: connection1:0: detected conn error (1020)
Nov 14 00:52:24 esx2 kernel: connection2:0: detected conn error (1020)
Nov 14 00:52:24 esx2 qdiskd[2024]: Error reading node ID block 1
Nov 14 00:52:24 esx2 qdiskd[2024]: Error reading node ID block 2
Nov 14 00:52:24 esx2 qdiskd[2024]: Error reading node ID block 3
Nov 14 00:52:24 esx2 qdiskd[2024]: Error reading node ID block 4
Nov 14 00:52:24 esx2 qdiskd[2024]: Error reading node ID block 5
Nov 14 00:52:24 esx2 qdiskd[2024]: Error reading node ID block 6
Nov 14 00:52:24 esx2 qdiskd[2024]: Error reading node ID block 7
Nov 14 00:52:24 esx2 qdiskd[2024]: Error reading node ID block 8
Nov 14 00:52:24 esx2 qdiskd[2024]: Error reading node ID block 9
Nov 14 00:52:24 esx2 qdiskd[2024]: Error reading node ID block 10
Nov 14 00:52:24 esx2 qdiskd[2024]: Error reading node ID block 11
Nov 14 00:52:24 esx2 qdiskd[2024]: Error reading node ID block 12
Nov 14 00:52:24 esx2 qdiskd[2024]: Error reading node ID block 13
Nov 14 00:52:24 esx2 qdiskd[2024]: Error reading node ID block 14
Nov 14 00:52:24 esx2 qdiskd[2024]: Error reading node ID block 15
Nov 14 00:52:24 esx2 qdiskd[2024]: Error reading node ID block 16
Nov 14 00:52:24 esx2 qdiskd[2024]: diskRawWriteShadow: Input/output error
Nov 14 00:52:24 esx2 qdiskd[2024]: diskRawWriteShadow: aligned write returned -1, not 512
Nov 14 00:52:24 esx2 qdiskd[2024]: diskRawWriteShadow
Nov 14 00:52:24 esx2 qdiskd[2024]: Error writing node ID block 2
Nov 14 00:52:24 esx2 qdiskd[2024]: Error writing to quorum disk
......
Nov 14 00:53:34 esx2 pmxcfs[1830]: [quorum] crit: quorum_initialize failed: 6
Nov 14 00:53:34 esx2 pmxcfs[1830]: [quorum] crit: can't initialize service
Nov 14 00:53:34 esx2 pmxcfs[1830]: [confdb] crit: confdb_initialize failed: 6
Nov 14 00:53:34 esx2 pmxcfs[1830]: [quorum] crit: can't initialize service
Nov 14 00:53:34 esx2 pmxcfs[1830]: [dcdb] crit: cpg_initialize failed: 6
Nov 14 00:53:34 esx2 pmxcfs[1830]: [quorum] crit: can't initialize service
Nov 14 00:53:34 esx2 pmxcfs[1830]: [dcdb] crit: cpg_initialize failed: 6
Nov 14 00:53:34 esx2 pmxcfs[1830]: [quorum] crit: can't initialize service

Anyway I can fix this?
 
Re: [patch provided with fix] Re: shutdown of node shuts down VM

Hi Deitmar,
Attached patch seems to have fixed this issue:

I do not understand that fix: umountiscsis.sh umnounts file systems, and your VM does not use such file system.
 
Re: [patch provided with fix] Re: shutdown of node shuts down VM

I do not understand that fix: umountiscsis.sh umnounts file systems, and your VM does not use such file system.

wait, just found some code which deactivates iSCSI LVM Volumes. Did you set any 'LVMGROUPS' in /etc/default/open-iscsi?

wow, and found another bug - that script simply deactivates NFS volumes by default.

I guess I will add the dependency as you suggested.
 
Re: [patch provided with fix] Re: shutdown of node shuts down VM

Anyway I can fix this?

Seems we stop open-iscsi before cman, but qdisk needs open-iscsis. Maybe you can try to add an iscsi dependency to the cman init script?
 
Re: [patch provided with fix] Re: shutdown of node shuts down VM

Hi Dietmar,

It seems to have fixed it. I still see lines in syslog like the following on start and stop:
Nov 14 17:38:15 esx2 pmxcfs[1809]: [quorum] crit: quorum_initialize failed: 6
Nov 14 17:38:15 esx2 pmxcfs[1809]: [quorum] crit: can't initialize service
Nov 14 17:38:15 esx2 pmxcfs[1809]: [confdb] crit: confdb_initialize failed: 6
Nov 14 17:38:15 esx2 pmxcfs[1809]: [quorum] crit: can't initialize service
Nov 14 17:38:15 esx2 pmxcfs[1809]: [dcdb] crit: cpg_initialize failed: 6
Nov 14 17:38:15 esx2 pmxcfs[1809]: [quorum] crit: can't initialize service
Nov 14 17:38:15 esx2 pmxcfs[1809]: [dcdb] crit: cpg_initialize failed: 6
Nov 14 17:38:15 esx2 pmxcfs[1809]: [quorum] crit: can't initialize service

Is this caused by the prior bug, and how can it be fixed?
 
Re: [patch provided with fix] Re: shutdown of node shuts down VM

Is this caused by the prior bug, and how can it be fixed?

We stop cman before pmxcfs, so those messages are expected (It is non-trivial to avoid them, sorry).
 
Re: [patch provided with fix] Re: shutdown of node shuts down VM

Ok. Just to be curtain. Quorum is establish perfect and messages flow between nodes when status change. I conclude this issue has been fixed:)
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!