Hardware watchdog (ipmi_watchdog) on Proxmox 5

[chris_lee]

Dear colleagues,

I moved to Proxmox 5 in a dev environment and was wondering how to setup the hardware watchdog.
On the same hardware running Proxmox 4, a kernel module ipmi_watchdog has been loaded.
Now I can only find the following modules.

lsmod |grep ipmi
ipmi_ssif             24576  0
ipmi_si                57344  0
ipmi_devintf        20480  0
ipmi_msghandler  45056  3 ipmi_ssif,ipmi_devintf,ipmi_si

When I manually try to load the ilmi_watchdog module, I get an error. The module ist missing in the drivers directory:

/lib/modules/4.10.17-1-pve/kernel/drivers/watchdog# ls -lah

-rw-r--r--  1 root root  14K Jul 28 14:09 acquirewdt.ko
-rw-r--r--  1 root root  15K Jul 28 14:09 advantechwdt.ko
-rw-r--r--  1 root root  14K Jul 28 14:09 alim1535_wdt.ko
-rw-r--r--  1 root root  17K Jul 28 14:09 alim7101_wdt.ko
-rw-r--r--  1 root root  15K Jul 28 14:09 cadence_wdt.ko
-rw-r--r--  1 root root  15K Jul 28 14:09 cpu5wdt.ko
-rw-r--r--  1 root root 8.5K Jul 28 14:09 da9052_wdt.ko
-rw-r--r--  1 root root 8.8K Jul 28 14:09 da9055_wdt.ko
-rw-r--r--  1 root root  11K Jul 28 14:09 da9062_wdt.ko
-rw-r--r--  1 root root 8.6K Jul 28 14:09 da9063_wdt.ko
-rw-r--r--  1 root root  11K Jul 28 14:09 dw_wdt.ko
-rw-r--r--  1 root root 9.1K Jul 28 14:09 ebc-c384_wdt.ko
-rw-r--r--  1 root root  16K Jul 28 14:09 eurotechwdt.ko
-rw-r--r--  1 root root  23K Jul 28 14:09 f71808e_wdt.ko
-rw-r--r--  1 root root  21K Jul 28 14:09 hpwdt.ko
-rw-r--r--  1 root root  17K Jul 28 14:09 i6300esb.ko
-rw-r--r--  1 root root  14K Jul 28 14:09 ib700wdt.ko
-rw-r--r--  1 root root  14K Jul 28 14:09 ibmasr.ko
-rw-r--r--  1 root root  16K Jul 28 14:09 ie6xx_wdt.ko
-rw-r--r--  1 root root  17K Jul 28 14:09 it8712f_wdt.ko
-rw-r--r--  1 root root  21K Jul 28 14:09 it87_wdt.ko
-rw-r--r--  1 root root 8.1K Jul 28 14:09 iTCO_vendor_support.ko
-rw-r--r--  1 root root  20K Jul 28 14:09 iTCO_wdt.ko
-rw-r--r--  1 root root  15K Jul 28 14:09 kempld_wdt.ko
-rw-r--r--  1 root root  14K Jul 28 14:09 machzwd.ko
-rw-r--r--  1 root root  12K Jul 28 14:09 max63xx_wdt.ko
-rw-r--r--  1 root root  17K Jul 28 14:09 mei_wdt.ko
-rw-r--r--  1 root root 7.6K Jul 28 14:09 mena21_wdt.ko
-rw-r--r--  1 root root 9.8K Jul 28 14:09 menf21bmc_wdt.ko
-rw-r--r--  1 root root  12K Jul 28 14:09 ni903x_wdt.ko
-rw-r--r--  1 root root  18K Jul 28 14:09 nv_tco.ko
-rw-r--r--  1 root root  12K Jul 28 14:09 of_xilinx_wdt.ko
-rw-r--r--  1 root root  16K Jul 28 14:09 pc87413_wdt.ko
-rw-r--r--  1 root root  30K Jul 28 14:09 pcwd_pci.ko
-rw-r--r--  1 root root  24K Jul 28 14:09 pcwd_usb.ko
-rw-r--r--  1 root root 4.8K Jul 28 14:09 pretimeout_panic.ko
-rw-r--r--  1 root root 8.3K Jul 28 14:09 retu_wdt.ko
-rw-r--r--  1 root root  15K Jul 28 14:09 sbc60xxwdt.ko
-rw-r--r--  1 root root 9.9K Jul 28 14:09 sbc_epx_c3.ko
-rw-r--r--  1 root root  12K Jul 28 14:09 sbc_fitpc2_wdt.ko
-rw-r--r--  1 root root  19K Jul 28 14:09 sc1200wdt.ko
-rw-r--r--  1 root root  17K Jul 28 14:09 sch311x_wdt.ko
-rw-r--r--  1 root root  15K Jul 28 14:09 smsc37b787_wdt.ko
-rw-r--r--  1 root root  12K Jul 28 14:09 softdog.ko
-rw-r--r--  1 root root  21K Jul 28 14:09 sp5100_tco.ko
-rw-r--r--  1 root root 9.0K Jul 28 14:09 twl4030_wdt.ko
-rw-r--r--  1 root root  14K Jul 28 14:09 via_wdt.ko
-rw-r--r--  1 root root  14K Jul 28 14:09 w83627hf_wdt.ko
-rw-r--r--  1 root root  15K Jul 28 14:09 w83877f_wdt.ko
-rw-r--r--  1 root root  16K Jul 28 14:09 w83977f_wdt.ko
-rw-r--r--  1 root root  13K Jul 28 14:09 wafer5823wdt.ko
-rw-r--r--  1 root root  16K Jul 28 14:09 wdat_wdt.ko
-rw-r--r--  1 root root  22K Jul 28 14:09 wdt_pci.ko
-rw-r--r--  1 root root  12K Jul 28 14:09 wm831x_wdt.ko
-rw-r--r--  1 root root 9.8K Jul 28 14:09 wm8350_wdt.ko
-rw-r--r--  1 root root  15K Jul 28 14:09 xen_wdt.ko
-rw-r--r--  1 root root  21K Jul 28 14:09 ziirave_wdt.ko

Any idea how to use a hardware watchdog instead of the softdog?

Chris

 

[Alwin]

You should find the module under /lib/modules/4.10.17-1-pve/kernel/drivers/char/ipmi/ipmi_watchdog.ko. What error message do you get, when loading the module?

 

[chris_lee]

Problem solved. After the latest Proxmox update the ipmi_watchdog driver could be loaded and the *.ko file is available.

I tried to implement the hardware watchdog on an old HP DL160 Gen6 machine. I tried ipmi_watchdog as well as hpwdt module, but the output of the ipmitool says:

ipmitool mc watchdog get
Watchdog Timer Use:     Reserved (0x00)
Watchdog Timer Is:      Stopped
Watchdog Timer Actions: No action (0x00)
Pre-timeout interval:   0 seconds
Timer Expiration Flags: 0x00
Initial Countdown:      0 sec
Present Countdown:      0 sec

Whereas on the Dell machines I get:

Watchdog Timer Use:     SMS/OS (0x44)
Watchdog Timer Is:      Started/Running
Watchdog Timer Actions: Power Cycle (0x03)
Pre-timeout interval:   0 seconds
Timer Expiration Flags: 0x10
Initial Countdown:      10 sec
Present Countdown:      9 sec

Any idea?

 

[debi@n]

Hi! same problem here, since proxmox 4.X and HP proliant and watchdog, i read about watchdog external but i don´t know where i can find it.
i work without HA , it was the solution for me.

maybe can help:

https://forum.proxmox.com/threads/hp-proliant-reboots.25523/

https://pve.proxmox.com/wiki/High_Availability_Cluster_4.x#HP_ILO_.28module_.22hpwdt.22_.29

 

[debi@n]

i found this too but i didn´t test it.

https://www.aliexpress.com/item/Hardware-WatchDog-USB-For-Mining-Rig-Unattended-Operation-Crash-Auto-Recover-Server-24-hours-Computer-Sensor/32827081267.html?spm=2114.search0302.3.2.CinZcE&ws_ab_test=searchweb0_0,searchweb201602_0_10073_10152_10151_10536_10538_10537_10539_10176_10055_10154_10056_10155_10059_10312_10314_10534_10313_10533_10060_10084_100031_10083_10184_10107_10547_10546_10307_10548_10341_10065_10142_10340_10068_10343_10541_10342_10103_10345_10102_10344_10303_10325,searchweb201603_0,ppcSwitch_0&algo_pvid=39871637-80d4-42ec-bd6c-7051fada1a93&algo_expid=39871637-80d4-42ec-bd6c-7051fada1a93-0

 

[Alwin]

Most of the time, the watchdog can/needs to be activated in the baseboard management, usually accessible through BIOS or LAN.