HEALTH_WARN 1 daemons have recently crashed

thanks stark,
happened to me on the 17th of Jan.
many thanks
Damon
 
Is there a period defined that removes errors from gui if not present anymore? I had a error too, it was fixed, but warning did not get away from gui. Can someone share details how ceph errors are handeled in proxmox gui?

Edit: I know howto do it via cli, but is there any check proxmox does to remove them automatically without manual action from users?
 
Last edited:
i used today on command line:

Bash: 
ceph crash ls

that gives a list of archived and new crashes.
then you can archive your crash:

Bash: 
ceph crash archive 2020-10-29_03:47:12.641232Z_843e3d9d-bc56-46dc-8175-9026fa7f44a4

(you must replace the id with your values of course)
(sorry, did not see that you want a GUI solution)
 
Last edited:
  • Like
Reactions: jsterr
Why appear this Warnings ??, what is the problem Ceph or Proxmox if the solution is to archived (ceph crash archive-all) , do have permanent solution for this

Luis
 
Luis Pesaressi said:
Why appear this Warnings ??, what is the problem Ceph or Proxmox if the solution is to archived (ceph crash archive-all) , do have permanent solution for this

Luis
Click to expand...

Its not that you archive the error and it still exists. If you see this error you need to check the PREVIOUSLY crashed service like maybe the MON-service and look why it has crashed. Means everything is fine, but something WAS not fine in the past.

To be fair I dont know if this errors will automatically disappear after a period of time, it maybe should to avoid confusion?
 
Here in Apr 21 2025 this was a helpful thread - never had this issue in 2+ years of running proxmox ceph!
 
Joined just to add, for those wondering why this didn't auto clear, because an automatic clear on something that says "something crashed, multiple times" in a storage context is a really bad idea. Corruption may have occurred, network paths might have been broken causing crash, quorum might have been lost, many of these things you would only discover when a component actually fails. The system is basically saying "Hey, something happened that I can't figure out if it was something you did or I did, can you check for me and if it's all good and you know why then archive these".

Mon daemon in particular seems to crash a bit during simultaneous reboots of multiple nodes + bonded ethernet and spanning tree. There's nothing implicitly wrong with that, it's transient, it recovers but ceph doesn't know that.

Remember, ceph is built for very large storage deployments where random crashes can be a tell tale sign of broader issues that don't show up immediately. If you absolutely don't care about the integrity of your storage you could always setup a nightly cron of ceph crash archive-all but don't come crying when your data goes MIA because you had multiple failures that you didn't see and then your cluster went splat.
 
Last edited:
You must log in or register to reply here.
Top Bottom
Back