Proxmox VE Cluster 4.0 watchdog fencing

[Blinkiz]

Proxmox 4.0 comes with watchdog fencing built in.

I have a Supermicro moderboard with watchdog function. Activating it in BIOS just make my node restart every 5 minutes.

If your hardware has a hardware watchdog, this one will be automatically detected and used.

Am seeing my moderboard as a hardware watchdog. How should this work?

 

[wolfgang]

Hi,
AFIK this set the hw watchdog to 5 min.
this mean if no one reset the timer you get fenced.
We set the hw watchdog if you have a cluster and if you have a HA resource on it.

 

[t.lamprecht]

Yes it uses it automatically if the watchdog is configured for your system, i.e. a /dev/watchdog[0] device is there.
Have you loaded it's driver?

what does:
# dmesg | grep watchdog
outputs, further look at your syslog for the "watchdog-mux" entry, something like

watchdog-mux[1152]: Watchdog driver 'iTCO_wdt', version 0

should be visible, the driver differs naturally.

If you do not activate it in BIOS, do you still see it loading?

If it's an IPMI watchdog, loads but doesn't triggers this may be helpful for other (not you in that case)
http://pve.proxmox.com/wiki/High_Availability_Cluster_4.x#IPMI_Watchdog

 

[Blinkiz]

Oct  8 14:55:30 vc11 watchdog-mux[1690]: Watchdog driver 'iTCO_wdt', version 0
Oct  8 14:55:30 vc11 kernel: [    0.097016] NMI watchdog: enabled on all CPUs, permanently consumes one hw-PMU counter.

Then after ~5 minutes, the node is reset by BIOS.

We set the hw watchdog if you have a cluster and if you have a HA resource on it.

I have cluster and now one virtual machine handled by HA running on another machine in the cluster.

Quorum information
------------------
Date:             Thu Oct  8 15:03:42 2015
Quorum provider:  corosync_votequorum
Nodes:            3
Node ID:          0x00000001
Ring ID:          132
Quorate:          Yes


Votequorum information
----------------------
Expected votes:   3
Highest expected: 3
Total votes:      3
Quorum:           2  
Flags:            Quorate 


Membership information
----------------------
    Nodeid      Votes Name
0x00000001          1 172.16.26.33 (local)
0x00000003          1 172.16.26.34
0x00000002          1 172.16.26.35

Why is not Proxmox handling the watchdog function?

 

[t.lamprecht]

Deactivate the Watchdog in the BIOS, it should appear to the kernel nonetheless.

Some BIOS activate the watchdog when this function is enabled, expecting that a driver in the OS updates it, if this is not the case a reset happens.

5 minutes sounds not like an time we use, our watchdog-mux has a 10 seconds granularity and the local resource manager sets it to 60 seconds.

Look in the syslog if watchdog-mux gerts started correctly and as said above, deactivate it in the BIOS and recheck `dmsg | grep -i watchdog`.

 

[Blinkiz]

Having watchdog in Supermicro BIOS active or disabled always gives two log lines in syslog.

Oct  8 14:55:30 vc11 watchdog-mux[1690]: Watchdog driver 'iTCO_wdt', version 0
Oct  8 14:55:30 vc11 kernel: [    0.097016] NMI watchdog: enabled on all CPUs, permanently consumes one hw-PMU counter.

Are you guys saying that Supermicro watchdog is not compatible with Proxmox fencing maybe? Sure that I should not install anything on the system to get this to work?

 

[wolfgang]

We use several different kinds of Supermicro boards with no problems.
But we do not start the watchdog by the bios.