local-LVM not available after Kernel update on PVE 7

[fiona]

Hi,

This just started happening to me, but after a power outage, not a software update.

After the outage, the server appeared to hang for a long time, and then finally displayed "Timed out for waiting the udev queue being empty."

It then continued to boot, but without my LVM partition, which caused all my containers to fail to start.

I did the trick of increasing the timeout period, and this did work, but now my server takes an absolute age to boot, and, worse, my containers all take an age to start as well, for no apparent reason.

What actually causes this huge delay, why might it have happened after a power outage, and is there anything I can do to undo the damage?

please check the health of your physical disk, e.g. using smartctl -a /dev/XYZ. Are there any interesting messages in the system logs/journal?

 

[Taz-Matt]

I agree with @fiona. Maybe test the drives properly and replace the faulty one if any. Don't forget backups.

Also, regarding the "trick", I personally removed it from boot time and let the server start without the LVM partition on which the VM disks are. At least I got access to the host and can send the commands manually or thru a script and know what to expect. I hate being in the dark for 10-20 minutes, I prefer booting without LVM and then "having control". Note that this only happens on one of my servers and it is definitely the slowest one I have regarding disks. Others have even bigger disks and disk usage is higher but don't have this issue.

Good luck with that!

 

[DavidGA]

Hi,

please check the health of your physical disk, e.g. using smartctl -a /dev/XYZ. Are there any interesting messages in the system logs/journal?

SMART tests check out just fine, and there's nothing weird in the boot log.

However, I've noticed another symptom, despite the fact that containers to start and run OK, they can no longer be backed up. Telling proxmox to backup any container (I use the "stop" method, and the backup goes to an NFS mounted directory), the backup process simply runs forever, and never completes.

 

[fiona]

SMART tests check out just fine, and there's nothing weird in the boot log.

However, I've noticed another symptom, despite the fact that containers to start and run OK, they can no longer be backed up. Telling proxmox to backup any container (I use the "stop" method, and the backup goes to an NFS mounted directory), the backup process simply runs forever, and never completes.

What if you try to read an LV from start to finish? Does that also hang?

 

[DavidGA]

No, and I've since found that backups do complete, they're just verrrry slow.

 

[stuartxt]

Nice fix

 

[kpmarcin]

event_timeout=600

Thank you so much. This is the only solution that worked for me.
I had this issue with my 20TB hard drive.