How to extend the udev-timer?

L

limefrog

New Member
Jan 31, 2025
15
0
1
Hello,

I'm in need to extend the setting for the udev-timer responsible for the timed out for waiting the udev queue being empty. message.
How to do that?

Best regards
limefrog
 
I had a look at udevadm but it seems, the settings done that way are not persistent after reboot !?

udevadm settle --timeout=300

...did nothing after the next reboot, and I haven't found how to read the value of the setting back.
Think i need to find out, where to alter the default which seems to be set to 120s:

Code: 
udevadm settle [options]

Watches the udev event queue, and exits if all current events are handled.

-t, --timeout=SECONDS
Maximum number of seconds to wait for the event queue to become empty. The default value is 120 seconds. A value of 0 will check if the queue is empty and always return immediately. A non-zero value will return an exit code of 0 if queue became empty before timeout was reached, non-zero otherwise.
 
I found systemctl cat systemd-udev-settle.service to read values and settings:

Code: 
# systemctl cat systemd-udev-settle.service

# /lib/systemd/system/systemd-udev-settle.service
#  SPDX-License-Identifier: LGPL-2.1-or-later
#
#  This file is part of systemd.
#
#  systemd is free software; you can redistribute it and/or modify it
#  under the terms of the GNU Lesser General Public License as published by
#  the Free Software Foundation; either version 2.1 of the License, or
#  (at your option) any later version.

# This service can dynamically be pulled-in by legacy services which
# cannot reliably cope with dynamic device configurations, and wrongfully
# expect a populated /dev during bootup.

[Unit]
Description=Wait for udev To Complete Device Initialization
Documentation=man:systemd-udev-settle.service(8)
DefaultDependencies=no
Wants=systemd-udevd.service
After=systemd-udev-trigger.service
Before=sysinit.target
ConditionPathIsReadWrite=/sys

[Service]
Type=oneshot
TimeoutSec=180
RemainAfterExit=yes
ExecStart=udevadm settle

that reveals /lib/systemd/system/systemd-udev-settle.service as configuration file and default value of 180s for timeout. Change setting...

Code: 
TimeoutSec=180  --> TimeoutSec=300

...and call systemctl daemon-reload as requested...

Code: 
# systemctl cat systemd-udev-settle.service
# Warning: systemd-udev-settle.service changed on disk, the version systemd has loaded is outdated.
# This output shows the current version of the unit's original fragment and drop-in files.
# If fragments or drop-ins were added or removed, they are not properly reflected in this output.
# Run 'systemctl daemon-reload' to reload units.
# /lib/systemd/system/systemd-udev-settle.service
#  SPDX-License-Identifier: LGPL-2.1-or-later
#
#  This file is part of systemd.
#
#  systemd is free software; you can redistribute it and/or modify it
#  under the terms of the GNU Lesser General Public License as published by
#  the Free Software Foundation; either version 2.1 of the License, or
#  (at your option) any later version.

# This service can dynamically be pulled-in by legacy services which
# cannot reliably cope with dynamic device configurations, and wrongfully
# expect a populated /dev during bootup.

[Unit]
Description=Wait for udev To Complete Device Initialization
Documentation=man:systemd-udev-settle.service(8)
DefaultDependencies=no
Wants=systemd-udevd.service
After=systemd-udev-trigger.service
Before=sysinit.target
ConditionPathIsReadWrite=/sys

[Service]
Type=oneshot
TimeoutSec=300
RemainAfterExit=yes
ExecStart=udevadm settle

...after power cycle I got timed out for waiting the udev queue being empty. message again!
At least, the setting was persistent, my 300s are still shown on systemctl cat systemd-udev-settle.service.

Seems I'm on the wrong way! Does anybody have a clue how to solve the situation?
 
Last edited:
I already found /etc/udev/udev.conf:

Code: 
# see udev.conf(5) for details
#[/INDENT]
# udevd is also started in the initrd.  When this file is modified you might
# also want to rebuild the initrd, so that it will include the modified configuration.

#udev_log=info
#children_max=
#exec_delay=
event_timeout=300                    <--    was 180 and commented out.
#timeout_signal=SIGKILL
#resolve_names=early

and altered the timeout accordingly to 300s, so both entries are equal now ... but that does nothing again.
 
Last edited:
The man page systemd-udevd.service reveals parameter for the kernel command line:

Code: 
KERNEL COMMAND LINE

Parameters prefixed with "rd." will be read when systemd-udevd is used in an initrd, those without will be processed both in the initrd and on the host.

   . . .

udev.event_timeout=, rd.udev.event_timeout=
Wait for events to finish up to the given number of seconds. This option might be useful if events are terminated due to kernel drivers taking too long to initialize.

so I edit /etc/default/grub to:

Code: 
# If you change this file, run 'update-grub' afterwards to update
# /boot/grub/grub.cfg.
# For full documentation of the options in this file, see:
#   info -f grub -n 'Simple configuration'

GRUB_DEFAULT=0
GRUB_TIMEOUT=5
GRUB_DISTRIBUTOR=`lsb_release -i -s 2> /dev/null || echo Debian`
GRUB_CMDLINE_LINUX_DEFAULT="video=1024x768 rd.udev.event_timeout=300"          <-- !
GRUB_CMDLINE_LINUX=""

# If your computer has multiple operating systems installed, then you
# probably want to run os-prober. However, if your computer is a host
# for guest OSes installed via LVM or raw disk devices, running
# os-prober can cause damage to those guest OSes as it mounts
# filesystems to look for things.
#GRUB_DISABLE_OS_PROBER=false

# Uncomment to enable BadRAM filtering, modify to suit your needs
# This works with Linux (no patch required) and with any kernel that obtains
# the memory map information from GRUB (GNU Mach, kernel of FreeBSD ...)
#GRUB_BADRAM="0x01234567,0xfefefefe,0x89abcdef,0xefefefef"

# Uncomment to disable graphical terminal
#GRUB_TERMINAL=console

# The resolution used on graphical terminal
# note that you can use only modes which your graphic card supports via VBE
# you can see them in real GRUB with the command `vbeinfo'
GRUB_GFXMODE=1024x768

# Uncomment if you don't want GRUB to pass "root=UUID=xxx" parameter to Linux
#GRUB_DISABLE_LINUX_UUID=true

# Uncomment to disable generation of recovery mode menu entries
#GRUB_DISABLE_RECOVERY="true"

# Uncomment to get a beep at grub start
#GRUB_INIT_TUNE="480 440 1"

and do a update-grub as advised, rebooted ... but that does nothing again.

So I'll turn back all the settings to their defaults.
I need another idea!
 
According to another approach, i tried to handle that with the crowbar:

systemctl mask systemd-udev-settle.service
systemctl mask lvm2-monitor.service

But same as ever ... that does nothing again. Turned everything back in place then:

systemctl unmask systemd-udev-settle.service
systemctl unmask lvm2-monitor.service
 
Last edited:
It seems it has nothing to do with the systemd services, since systemd-analyze blame doesn't show something remarkable to me:

Code: 
# systemd-analyze blame

3.527s networking.service
2.566s pveproxy.service
2.373s pve-guests.service
2.157s pvedaemon.service
1.535s pve-ha-lrm.service
1.510s pvestatd.service
1.481s pvescheduler.service
1.457s pve-ha-crm.service
1.419s pve-firewall.service
1.051s pve-cluster.service
 981ms ifupdown2-pre.service
 979ms systemd-udev-settle.service
 745ms systemd-binfmt.service
 719ms spiceproxy.service
 644ms postfix@-.service
 519ms pvebanner.service
 262ms proc-fs-nfsd.mount
 241ms user@0.service
 215ms systemd-udev-trigger.service
 182ms systemd-journal-flush.service
 168ms systemd-journald.service
 168ms nfs-server.service
 144ms systemd-udevd.service
 138ms rrdcached.service
 138ms ssh.service
 130ms chrony.service
 103ms lvm2-monitor.service
  81ms smartmontools.service
  75ms lxc-net.service
  74ms systemd-modules-load.service
  68ms lxc.service
  64ms nfs-blkmap.service
  62ms nfs-idmapd.service
  60ms apparmor.service
  60ms nfsdcld.service
  60ms systemd-logind.service
  59ms modprobe@efi_pstore.service
  58ms lxcfs.service
  53ms pve-query-machine-capabilities.service
  52ms qmeventd.service
  52ms systemd-tmpfiles-setup.service
  51ms e2scrub_reap.service
  50ms ksmtuned.service
  44ms zfs-mount.service
  42ms dbus.service
  39ms systemd-sysusers.service
  38ms dev-hugepages.mount
  38ms zfs-share.service
  36ms dev-mqueue.mount
  34ms pve-lxc-syscalld.service
  33ms sys-kernel-debug.mount
  32ms sys-kernel-tracing.mount
  32ms pvefw-logger.service
  30ms keyboard-setup.service
  29ms kmod-static-nodes.service
  29ms run-rpc_pipefs.mount
  26ms modprobe@configfs.service
  26ms nfs-mountd.service
  25ms modprobe@dm_mod.service
  25ms modprobe@drm.service
  24ms proc-sys-fs-binfmt_misc.mount
  24ms systemd-random-seed.service
  24ms modprobe@loop.service
  23ms zfs-volume-wait.service
  22ms modprobe@fuse.service
  21ms console-setup.service
  20ms systemd-tmpfiles-setup-dev.service
  18ms zfs-import-cache.service
  16ms rbdmap.service
  16ms systemd-remount-fs.service
  16ms systemd-update-utmp.service
  15ms pvenetcommit.service
  14ms rpc-statd.service
  13ms systemd-sysctl.service
  12ms sys-fs-fuse-connections.mount
  12ms rpcbind.service
  12ms user-runtime-dir@0.service
  11ms systemd-update-utmp-runlevel.service
  10ms sys-kernel-config.mount
   8ms systemd-user-sessions.service
   5ms rpc-statd-notify.service
   2ms postfix.service
 138us blk-availability.service

Sure, I booted the system with two drives only, but the delays are there:

Code: 
# dmesg | grep ioc0

[    3.288986] mptbase: ioc0: Initiating bringup
[    3.441882] ioc0:
[   15.458564] scsi host1: ioc0: LSISAS1078 C2, FwRev=011bbe00h, Ports=1, MaxQ=276, IRQ=16
[   15.463274] mptsas: ioc0: attaching sata device: fw_channel 0, fw_id 0, phy 0, sas_addr 0x1221000000000000
[   16.479918] mptbase: ioc0: WARNING - IOC is in FAULT state (2650h)!!!
[   16.480022] mptbase: ioc0: WARNING - Issuing HardReset from mpt_fault_reset_work!!
[   16.480154] mptbase: ioc0: Initiating recovery
[   16.480245] mptbase: ioc0: WARNING - IOC is in FAULT state!!!
[   16.480339] mptbase: ioc0: WARNING -            FAULT code = 2650h
[   17.503744] mptbase: ioc0: Recovered from IOC FAULT
[   29.655815] mptbase: ioc0: WARNING - mpt_fault_reset_work: HardReset: success
[   29.662540] mptsas: ioc0: attaching sata device: fw_channel 0, fw_id 0, phy 0, sas_addr 0x1221000000000000
[   29.662774] mptsas: ioc0: attaching sata device: fw_channel 0, fw_id 1, phy 1, sas_addr 0x1221000001000000
[   29.683538] mptsas: ioc0: attaching sata device: fw_channel 0, fw_id 1, phy 1, sas_addr 0x1221000001000000
[   30.688020] mptbase: ioc0: WARNING - IOC is in FAULT state (2650h)!!!
[   30.688123] mptbase: ioc0: WARNING - Issuing HardReset from mpt_fault_reset_work!!
[   30.688250] mptbase: ioc0: Initiating recovery
[   30.688341] mptbase: ioc0: WARNING - IOC is in FAULT state!!!
[   30.688434] mptbase: ioc0: WARNING -            FAULT code = 2650h
[   31.711870] mptbase: ioc0: Recovered from IOC FAULT
[   43.875968] mptbase: ioc0: WARNING - mpt_fault_reset_work: HardReset: success

Does anybody have an idea on this?
 
Does anybody know if systemd-udev-settle.service has a persistent log-file somewhere? Or something else, where I can find out what happens here?
 
Very Strange: After rolling back all changes I've seen ONE boot sequence walking completely trough with four drives! So, I have the proof that it is possible. Shame on me: I didn't keep the dmesg output, because i thought i got it - but the next reboot brought me back into nowhere again!
 
The error emanates from udevadm wait - not settle. Check if the command being launched is actually include -t or the timeout argument while waiting (ps auxw). Also make sure the timeout isn’t overridden eg in /etc/udev/udev.conf.d/*.conf - it is usually best to include custom configs in its own .conf file, eg, 99mychanges.conf in order to override any distribution-specific configurations,
 
@guruevi
thanks a lot for your reply.

I checked /etc/udev:

Code: 
/etc/udev# ls

hwdb.d    rules.d  udev.conf

while hwdb.d is empty rule.d contains one file:

Code: 
/etc/udev/rules.d# ls

60-bridge-network-interface.rules

where the file contains nothing.

the udev.conf contains more or less nothing, since everything is commented out:

Code: 
# cat udev.conf
# see udev.conf(5) for details
#
# udevd is also started in the initrd.  When this file is modified you might
# also want to rebuild the initrd, so that it will include the modified configuration.

#udev_log=info
#children_max=
#exec_delay=
#event_timeout=180
#timeout_signal=SIGKILL
#resolve_names=early

When i changed #event_timeout=180 to event_timeout=300 nothing changed.

I've created /etc/udev/udev.conf.d/99_timeout.conf and put event_timeout=600 in there

Code: 
/etc/udev/udev.conf.d# cat 99_timeout.conf

event_timeout=600

According to my expectations that does nothing.

Check if the command being launched is actually include -t or the timeout argument while waiting (ps auxw).
Click to expand...

Sorry, but that needs a bit explanation how to do that (this is the first time I get in contact with udev). Where do i find the command being launched?
 
You must log in or register to reply here.
Top Bottom