got inotify poll request in wrong process - disabling inotify

RolandK

Famous Member
Mar 5, 2019
1,173
325
128
53
i'm seeing a lot of these messages in daemon.log:

what could be causing it ?

Code:
Dec 13 17:40:54 pve1 pveproxy[3288803]: got inotify poll request in wrong process - disabling inotify
Dec 13 17:59:44 pve1 pveproxy[3337246]: got inotify poll request in wrong process - disabling inotify
Dec 13 18:21:29 pve1 pveproxy[3393903]: got inotify poll request in wrong process - disabling inotify
Dec 14 12:55:49 pve1 pveproxy[2052165]: got inotify poll request in wrong process - disabling inotify
Dec 14 13:09:27 pve1 pveproxy[2086816]: got inotify poll request in wrong process - disabling inotify
Dec 14 13:25:01 pve1 pveproxy[2127331]: got inotify poll request in wrong process - disabling inotify
Dec 14 13:35:49 pve1 pveproxy[2154495]: got inotify poll request in wrong process - disabling inotify
Dec 14 13:49:22 pve1 pveproxy[2189097]: got inotify poll request in wrong process - disabling inotify
Dec 14 14:08:19 pve1 pveproxy[2236994]: got inotify poll request in wrong process - disabling inotify
Dec 14 14:50:08 pve1 pveproxy[2352233]: got inotify poll request in wrong process - disabling inotify
Dec 14 15:04:23 pve1 pveproxy[2394605]: got inotify poll request in wrong process - disabling inotify
Dec 14 15:23:25 pve1 pveproxy[2444495]: got inotify poll request in wrong process - disabling inotify
Dec 14 15:31:49 pve1 pveproxy[2466030]: got inotify poll request in wrong process - disabling inotify
Dec 14 15:44:16 pve1 pveproxy[2500998]: got inotify poll request in wrong process - disabling inotify
Dec 14 16:01:40 pve1 pveproxy[2560392]: got inotify poll request in wrong process - disabling inotify
Dec 14 16:15:30 pve1 pveproxy[2596020]: got inotify poll request in wrong process - disabling inotify
Dec 14 16:28:09 pve1 pveproxy[2893547]: got inotify poll request in wrong process - disabling inotify
Dec 14 16:55:25 pve1 pveproxy[3018526]: got inotify poll request in wrong process - disabling inotify
Dec 14 17:03:22 pve1 pveproxy[3214938]: got inotify poll request in wrong process - disabling inotify
Dec 14 17:15:54 pve1 pveproxy[3607122]: got inotify poll request in wrong process - disabling inotify
 
  • Like
Reactions: Sim0n and isaacvv
Hello,

What do the pvestatd and pveprfoxy services status say?

Bash:
systemctl status pvestatd.service
systemctl status pveproxy.service
 
Code:
# systemctl status pvestatd.service
â—Ź pvestatd.service - PVE Status Daemon
     Loaded: loaded (/lib/systemd/system/pvestatd.service; enabled; vendor preset: enabled)
     Active: active (running) since Fri 2022-12-16 19:26:01 CET; 2 days ago
    Process: 4443 ExecStart=/usr/bin/pvestatd start (code=exited, status=0/SUCCESS)
   Main PID: 4473 (pvestatd)
      Tasks: 1 (limit: 309307)
     Memory: 102.8M
        CPU: 1h 31min 57.524s
     CGroup: /system.slice/pvestatd.service
             └─4473 pvestatd

Dec 17 02:24:17 pve1 pvestatd[4473]: status update time (5.365 seconds)
Dec 17 02:35:57 pve1 pvestatd[4473]: status update time (5.370 seconds)
Dec 17 02:51:47 pve1 pvestatd[4473]: status update time (5.369 seconds)
Dec 17 02:57:27 pve1 pvestatd[4473]: status update time (5.385 seconds)
Dec 17 09:44:01 pve1 pvestatd[4473]: status update time (5.363 seconds)
Dec 17 20:06:11 pve1 pvestatd[4473]: status update time (5.381 seconds)
Dec 18 10:43:05 pve1 pvestatd[4473]: status update time (6.235 seconds)
Dec 18 12:11:24 pve1 pvestatd[4473]: status update time (5.364 seconds)
Dec 18 12:43:44 pve1 pvestatd[4473]: status update time (5.350 seconds)
Dec 18 20:01:14 pve1 pvestatd[4473]: status update time (5.382 seconds)


Code:
# systemctl status pveproxy.service
â—Ź pveproxy.service - PVE API Proxy Server
     Loaded: loaded (/lib/systemd/system/pveproxy.service; enabled; vendor preset: enabled)
     Active: active (running) since Fri 2022-12-16 19:26:03 CET; 2 days ago
    Process: 4514 ExecStartPre=/usr/bin/pvecm updatecerts --silent (code=exited, status=0/SUCCESS)
    Process: 4540 ExecStart=/usr/bin/pveproxy start (code=exited, status=0/SUCCESS)
    Process: 1025838 ExecReload=/usr/bin/pveproxy restart (code=exited, status=0/SUCCESS)
   Main PID: 4619 (pveproxy)
      Tasks: 4 (limit: 309307)
     Memory: 233.9M
        CPU: 22min 17.212s
     CGroup: /system.slice/pveproxy.service
             ├─   4619 pveproxy
             ├─1025883 pveproxy worker
             ├─3752873 pveproxy worker
             └─3757781 pveproxy worker

Dec 19 17:53:12 pve1 pveproxy[4619]: worker 3752873 started
Dec 19 17:53:16 pve1 pveproxy[3752872]: got inotify poll request in wrong process - disabling inotify
Dec 19 17:55:14 pve1 pveproxy[3716110]: worker exit
Dec 19 17:55:14 pve1 pveproxy[4619]: worker 3716110 finished
Dec 19 17:55:14 pve1 pveproxy[4619]: starting 1 worker(s)
Dec 19 17:55:14 pve1 pveproxy[4619]: worker 3757781 started
Dec 19 17:55:14 pve1 pveproxy[3752873]: Clearing outdated entries from certificate cache
Dec 19 17:56:09 pve1 pveproxy[3757781]: Clearing outdated entries from certificate cache
Dec 19 17:56:47 pve1 pveproxy[3752872]: worker exit
Dec 19 18:00:02 pve1 pveproxy[1025883]: Clearing outdated entries from certificate cache
 
i found a fix for this (one liner).

is anybody willing to test it before i submit this to pve-devel ?

actually this error happens rarely on the systems where i can add fixes for testing, so i'd be happy if i can get help testing
 
Hello Roland,
Just for your information,
I have the same in my lab.

Code:
pve1 pveproxy[1165]: worker 107438 finished
pve1 pveproxy[1165]: starting 1 worker(s)
pve1 pveproxy[1165]: worker 120699 started
pve1 pveproxy[120695]: got inotify poll request in wrong process - disabling inotify

Thanks,
 
the diff command shows the difference between the original file and the modified version. you can use patch utility to add the diff output to the file.



you can manually add PVE::INotify::inotify_close(); to /usr/share/perl5/PVE/APIServer/AnyEvent.pm as follows:


Code:
sub wait_end_loop {
    my ($self) = @_;

    $self->{end_loop} = 1;

    undef $self->{socket_watch};

    $0 = "$0 (shutdown)" if $0 !~ m/\(shutdown\)$/;

    if ($self->{conn_count} <= 0) {
        $self->{end_cond}->send(1);
        return;
    }

    # fork and exit, so that parent starts a new worker
    if (fork()) {
        exit(0);
    }

    PVE::INotify::inotify_close();

    # else we need to wait until all open connections gets closed
 
Last edited: