UPS NUTS Client to shutdown other node in cluster (not HA).

[vonDiten]

I have a small homelab composed by 3 (old-ish) thinkcentre.
One of them runs home services, and also the home NAS, so I plugged my UPS USB connection there, and NUTs is working fine.

My proxmox is composed by two Proxmox 9.1.5 in cluster configuration, but not HA, one of them have two network interfaces, it acts as a router to incoming request (I also have a traefik to work as a reverse proxy). this node have some LXCs and VMs. And I have NUTs Client working right.

My issue is that the second node (let's say it works as a pure compute node, Docker inside a VM and probably K3s in a couple VMs more) only have connection via 2.5g network with the other node. It doesn't have access to the home network (the 2.5g network is only between this two nodes).

Ideally, a NUTs client that is a client of the other NUTs client, but I cant make it work. So I think some command to trigger upon shutdown of the first node to shutdown the second one.

I now that I could add firewall rules to the node that have the two network interfaces for the second node to conect to the NUTS server, but it was really a pain to have everything working fine as of now. So any idea, greatly appreciated.

Thanks. chers

 

[bitranox]

Since your nodes are in a Proxmox cluster, SSH keys are already exchanged between them.
That makes this pretty painless.

SSH shutdown from node 1's NUT script, Node 1 already has a working NUT client, so you just add a script that SSHs into node 2 before shutting itself down.

Create a script on node 1 at /usr/local/bin/ups-shutdown-all.sh:

#!/bin/bash
NODE2_IP="<node2-ip-on-2.5g-network>"
# Shut down node 2 first (give it time to cleanly stop VMs)
ssh -o ConnectTimeout=5 root@"$NODE2_IP" \
"ha-manager crm-command node-maintenance enable && \
pvesh create /nodes/\$(hostname)/status --command shutdown" &
SSH_PID=$!
# Wait for the SSH command to finish
wait $SSH_PID 2>/dev/null
# Give node 2 a head start on its shutdown
sleep 10
# Now shut ourselves down
/sbin/shutdown -h now

Then in /etc/nut/upsmon.conf on node 1, point it at your script:

# /etc/nut/upsmon.conf on node 1
SHUTDOWNCMD "/usr/local/bin/ups-shutdown-all.sh"

# second method - this would shut down when switch to battery immideately:
# NOTIFYCMD /usr/local/bin/ups-shutdown-all.sh
# NOTIFYFLAG ONBATT SYSLOG+EXEC
# NOTIFYFLAG LOWBATT SYSLOG+EXEC

Quick sanity check first: make sure passwordless SSH actually works from node 1 to node 2.
Proxmox cluster setup should have handled this, but worth confirming:

ssh root@<node2-2.5g-ip> hostname

If that returns node 2's hostname without asking for a password you are done.

PS: oh, I forgot - the switch needs to be connected also to the UPS, otherwise You wouldnt reach node2 ....

Just pimped my UPS with new (and additional) mosfets, better cooling and 4x size battery pack .... (WIP)

 

[vonDiten]

Hey, really thanks. I'll try it.

About the switch on the same UPS, yes. In this case, I used to have a switch between them, but then I discovered that the only 2g5 connection that could take benefit of that are those nodes, so I made a direct nic to nic connection (perhaps this is a kind of no-no thing for network guys, but I'm not a network guy, and iperf2 shown that the speed is right both ways).

Also, great mod on your UPS, I don't really feel comfortable with power stuff (*) , but I modded my home services thinkcentre M720Q to have a LSI SAS controller, soldered an exit wire from a big non populated cap (that I traced to be directly connected to the PSU) so the computer and two spinning drives are connected to the same PSU (the 20v PSU V is switched down to 12v). Also in those small chasis air is a little scarse, so I added a radial fan, solder another 2 pin connector to the mobo, and 3d printed a hood for the heathsink. misteriously, nothing blow up yet, and it's been moths working so far.

I can upload a picture if someone find it interesting. (but is not for the faint of heart)

* I made some years ago, two back to back SCR and optotriac to power my 2 x 2k Watt resistors for my homebrewing machine.