rm /usr/local/lib/libz.so.1 -> file is restored automatically?

[astrakid]

hi,
i updated proxmox from 5 to 6 last week and after some issues it works now very well.
unfortunately a small issue is left: the file /usr/local/lib/libz.so.1 is an old version (this caused some troubles during installation already). when i remove that file, everything works smooth, but after some time (no reboot!) it reappears.

how can i solve this?

regards,
andre

 

[Chris]

Probably it get rewritten by the package it belongs to during upgrade.
Check via `dpkg -S /usr/local/lib/libz.so.1`

 

[astrakid]

root@server:/usr/lib# dpkg -S /usr/local/lib/libz.so.1
dpkg-query: Kein Pfad gefunden, der auf Muster /usr/local/lib/libz.so.1 passt

 

[Stoiko Ivanov]

/usr/local is not the path any (clean) debian package would install files into (the /usr/local hierarchy are for local modifications by the admin )
This seems rather odd - check which processes might access that lib (install lsof and check with that) - also look around for cronjobs which might place the file there....

I hope this helps!

 

[astrakid]

will check that, thanks. currently there is no new libz.so.1, so i have to wait until it is regenerated (i think that happens everytime an update via apt is done).

 

[astrakid]

so I think i found the cause for this file to be used:
sshd!
any idea why this could be or how to narrow down that?

 

[Stoiko Ivanov]

This all sounds really strange - and unless you've compiled sshd yourself I would doubt that it needs a library which is not shipped via package...

TBH I would check quite well that the Node has not been compromised....

Why do you think that sshd needs libz from /usr/local/lib?

 

[astrakid]

I checked via lsof:

root@server:~# lsof /usr/local/lib/libz.so.1
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
sshd 1963 root mem REG 8,17 113264 2885827 /usr/local/lib/libz.so.1
sshd 19223 root mem REG 8,17 113264 2885827 /usr/local/lib/libz.so.1
sshd 19315 hno_remote mem REG 8,17 113264 2885827 /usr/local/lib/libz.so.1

but i don't think that sshd needs it, i just observed that this file is used by sshd..

 

[astrakid]

today i installed updates, and i guess that libz.so.1 caused trouble again:

libxcomp3:amd64 (2:3.5.99.19-3+deb10u2) wird eingerichtet ...
pve-kernel-helper (6.0-8) wird eingerichtet ...
tzdata (2019c-0+deb10u1) wird eingerichtet ...

Current default time zone: 'Europe/Berlin'
Local time is now:      Mon Sep 23 07:37:10 CEST 2019.
Universal Time is now:  Mon Sep 23 05:37:10 UTC 2019.
Run 'dpkg-reconfigure tzdata' if you wish to change it.

nx-x11-common (2:3.5.99.19-3+deb10u2) wird eingerichtet ...
lxc-pve (3.1.0-65) wird eingerichtet ...
Warning: The unit file, source configuration file or drop-ins of lxc.service changed on disk. Run 'systemctl daemon-reload' to reload units.
pve-kernel-5.0.21-2-pve (5.0.21-3) wird eingerichtet ...
Examining /etc/kernel/postinst.d.
run-parts: executing /etc/kernel/postinst.d/apt-auto-removal 5.0.21-2-pve /boot/vmlinuz-5.0.21-2-pve
run-parts: executing /etc/kernel/postinst.d/dkms 5.0.21-2-pve /boot/vmlinuz-5.0.21-2-pve
Error!  The dkms.conf for this module includes a BUILD_EXCLUSIVE directive which
does not match this kernel/arch.  This indicates that it should not be built.
run-parts: executing /etc/kernel/postinst.d/initramfs-tools 5.0.21-2-pve /boot/vmlinuz-5.0.21-2-pve
update-initramfs: Generating /boot/initrd.img-5.0.21-2-pve
E: /usr/share/initramfs-tools/hooks/plymouth failed with return 1.
update-initramfs: failed for /boot/initrd.img-5.0.21-2-pve with 1.
run-parts: /etc/kernel/postinst.d/initramfs-tools exited with return code 1
Failed to process /etc/kernel/postinst.d at /var/lib/dpkg/info/pve-kernel-5.0.21-2-pve.postinst line 19.
dpkg: Fehler beim Bearbeiten des Paketes pve-kernel-5.0.21-2-pve (--configure):
 »installiertes pve-kernel-5.0.21-2-pve-Skript des Paketes post-installation«-Unterprozess gab den Fehlerwert 2 zurück
pve-headers-5.0.21-2-pve (5.0.21-3) wird eingerichtet ...
libnx-x11-6:amd64 (2:3.5.99.19-3+deb10u2) wird eingerichtet ...
pve-headers-5.0 (6.0-8) wird eingerichtet ...
dpkg: Abhängigkeitsprobleme verhindern Konfiguration von pve-kernel-5.0:
 pve-kernel-5.0 hängt ab von pve-kernel-5.0.21-2-pve; aber:
  Paket pve-kernel-5.0.21-2-pve ist noch nicht konfiguriert.

dpkg: Fehler beim Bearbeiten des Paketes pve-kernel-5.0 (--configure):
 Abhängigkeitsprobleme - verbleibt unkonfiguriert
libxcompshad3:amd64 (2:3.5.99.19-3+deb10u2) wird eingerichtet ...
nxagent (2:3.5.99.19-3+deb10u2) wird eingerichtet ...
Trigger für libc-bin (2.28-10) werden verarbeitet ...
Trigger für man-db (2.8.5-2) werden verarbeitet ...
Fehler traten auf beim Bearbeiten von:
 pve-kernel-5.0.21-2-pve
 pve-kernel-5.0
E: Sub-process /usr/bin/dpkg returned an error code (1)

and in syslog i see:

Sep 23 07:24:00 server systemd[1]: Starting Proxmox VE replication runner...
Sep 23 07:24:00 server pvesr[26276]: Can't load '/usr/lib/x86_64-linux-gnu/perl5/5.28/auto/RRDs/RRDs.so' for module RRDs: /usr/local/lib/libz.so.1: version `ZLIB_1.2.9' not found (required by /usr/lib/x86_64-linux-gnu/libpng16.so.16) at /usr/lib/x86_64-linux-gnu/perl/5.28/DynaLoader.pm line 187.
Sep 23 07:24:00 server pvesr[26276]:  at /usr/share/perl5/PVE/Cluster.pm line 15.
Sep 23 07:24:00 server pvesr[26276]: Compilation failed in require at /usr/share/perl5/PVE/Cluster.pm line 15.
Sep 23 07:24:00 server pvesr[26276]: BEGIN failed--compilation aborted at /usr/share/perl5/PVE/Cluster.pm line 15.
Sep 23 07:24:00 server pvesr[26276]: Compilation failed in require at /usr/share/perl5/PVE/RPCEnvironment.pm line 12.
Sep 23 07:24:00 server pvesr[26276]: BEGIN failed--compilation aborted at /usr/share/perl5/PVE/RPCEnvironment.pm line 12.
Sep 23 07:24:00 server pvesr[26276]: Compilation failed in require at /usr/share/perl5/PVE/CLI/pvesr.pm line 10.
Sep 23 07:24:00 server pvesr[26276]: BEGIN failed--compilation aborted at /usr/share/perl5/PVE/CLI/pvesr.pm line 10.
Sep 23 07:24:00 server pvesr[26276]: Compilation failed in require at /usr/bin/pvesr line 6.
Sep 23 07:24:00 server pvesr[26276]: BEGIN failed--compilation aborted at /usr/bin/pvesr line 6.
Sep 23 07:24:00 server systemd[1]: pvesr.service: Main process exited, code=exited, status=2/INVALIDARGUMENT
Sep 23 07:24:00 server systemd[1]: pvesr.service: Failed with result 'exit-code'.
Sep 23 07:24:00 server systemd[1]: Failed to start Proxmox VE replication runner.

this message repeats every minute.
after removing libz.so.1 this message reappears and dpkg --configure -a succeeds as well?!

Sep 23 07:49:00 server systemd[1]: Starting Proxmox VE replication runner...
Sep 23 07:49:01 server CRON[19401]: (root) CMD (/opt/oscam/check_oscamreaderdown.sh  >>/opt/oscam/check_oscamreader.log 2>&1)
Sep 23 07:49:01 server systemd[1]: pvesr.service: Succeeded.
Sep 23 07:49:01 server systemd[1]: Started Proxmox VE replication runner.

maybe a server reboot will solve it, will try to perform the reboot today...

 

[Stoiko Ivanov]

as said - having libz in /usr/local/lib can quite sure lead to problems like these - you need to find out where it comes from and why it keeps reappearing.