[SOLVED] upgrade issue's

[svennd]

I wanted to install updates, but there are issue's last time this happened, I had to completely reinstall the machine... I'm not feeling like doing that again ... Could you please advice :

apt-get update
apt-get upgrade : http://pastebin.com/aci33CRT

systemctl status pvedaemon.service

systemctl status pvedaemon.service
● pvedaemon.service - PVE API Daemon
   Loaded: loaded (/lib/systemd/system/pvedaemon.service; enabled)
   Active: active (running) (Result: exit-code) since Thu 2017-02-02 10:02:50 CET; 5 days ago
  Process: 25724 ExecReload=/usr/bin/pvedaemon restart (code=exited, status=2)
Main PID: 3124 (pvedaemon)
   CGroup: /system.slice/pvedaemon.service
           ├─ 3124 pvedaemon
           ├─11299 pvedaemon worker
           ├─11689 pvedaemon worker
           └─26073 pvedaemon worker

Feb 07 10:19:49 rocky pvedaemon[25724]: Compilation failed in require at /usr/share/perl5/PVE/API2/Cluster.pm line 13.
Feb 07 10:19:49 rocky pvedaemon[25724]: BEGIN failed--compilation aborted at /usr/share/perl5/PVE/API2/Cluster.pm line 13.
Feb 07 10:19:49 rocky pvedaemon[25724]: Compilation failed in require at /usr/share/perl5/PVE/API2.pm line 13.
Feb 07 10:19:49 rocky pvedaemon[25724]: BEGIN failed--compilation aborted at /usr/share/perl5/PVE/API2.pm line 13.
Feb 07 10:19:49 rocky pvedaemon[25724]: Compilation failed in require at /usr/share/perl5/PVE/Service/pvedaemon.pm line 8.
Feb 07 10:19:49 rocky pvedaemon[25724]: BEGIN failed--compilation aborted at /usr/share/perl5/PVE/Service/pvedaemon.pm line 8.
Feb 07 10:19:49 rocky pvedaemon[25724]: Compilation failed in require at /usr/bin/pvedaemon line 11.
Feb 07 10:19:49 rocky pvedaemon[25724]: BEGIN failed--compilation aborted at /usr/bin/pvedaemon line 11.
Feb 07 10:19:49 rocky systemd[1]: pvedaemon.service: control process exited, code=exited status=2
Feb 07 10:19:49 rocky systemd[1]: Reload failed for PVE API Daemon.

Trying again results in :

 apt-get upgrade
Reading package lists... Done
Building dependency tree
Reading state information... Done
Calculating upgrade... Done
The following packages have been kept back:
  libpve-common-perl pve-container pve-manager qemu-server
0 upgraded, 0 newly installed, 0 to remove and 4 not upgraded.
26 not fully installed or removed.
After this operation, 0 B of additional disk space will be used.
Do you want to continue? [Y/n] y
Setting up libssl1.0.0:amd64 (1.0.1t-1+deb8u6) ...
Setting up libxpm4:amd64 (1:3.5.12-0+deb8u1) ...
Setting up libuutil1linux (0.6.5.8-pve14~bpo80) ...
Setting up libnvpair1linux (0.6.5.8-pve14~bpo80) ...
Setting up libpve-access-control (4.0-23) ...
Setting up libpve-storage-perl (4.0-73) ...
Setting up libzpool2linux (0.6.5.8-pve14~bpo80) ...
Setting up libzfs2linux (0.6.5.8-pve14~bpo80) ...
Setting up lxcfs (2.0.6-pve1) ...
Setting up lxc-pve (2.0.7-1) ...
Setting up openssl (1.0.1t-1+deb8u6) ...
Setting up pve-kernel-4.4.35-2-pve (4.4.35-79) ...
Examining /etc/kernel/postinst.d.
run-parts: executing /etc/kernel/postinst.d/apt-auto-removal 4.4.35-2-pve /boot/vmlinuz-4.4.35-2-pve
run-parts: executing /etc/kernel/postinst.d/initramfs-tools 4.4.35-2-pve /boot/vmlinuz-4.4.35-2-pve
update-initramfs: Generating /boot/initrd.img-4.4.35-2-pve
run-parts: executing /etc/kernel/postinst.d/zz-update-grub 4.4.35-2-pve /boot/vmlinuz-4.4.35-2-pve
Generating grub configuration file ...
Found linux image: /boot/vmlinuz-4.4.35-2-pve
Found initrd image: /boot/initrd.img-4.4.35-2-pve
Found linux image: /boot/vmlinuz-4.4.19-1-pve
Found initrd image: /boot/initrd.img-4.4.19-1-pve
Found memtest86+ image: /boot/memtest86+.bin
Found memtest86+ multiboot image: /boot/memtest86+_multiboot.bin
done
Setting up spiceterm (2.0-2) ...
Setting up pve-ha-manager (1.0-40) ...
Job for pve-ha-lrm.service failed. See 'systemctl status pve-ha-lrm.service' and 'journalctl -xn' for details.
Job for pve-ha-crm.service failed. See 'systemctl status pve-ha-crm.service' and 'journalctl -xn' for details.
Setting up pve-docs (4.4-3) ...
Setting up pve-manager (4.4-5) ...
Job for pvedaemon.service failed. See 'systemctl status pvedaemon.service' and 'journalctl -xn' for details.
dpkg: error processing package pve-manager (--configure):
subprocess installed post-installation script returned error exit status 1
dpkg: dependency problems prevent configuration of proxmox-ve:
proxmox-ve depends on pve-manager; however:
  Package pve-manager is not configured yet.

dpkg: error processing package proxmox-ve (--configure):
dependency problems - leaving unconfigured
Setting up tcpdump (4.9.0-1~deb8u1) ...
Setting up zfsutils-linux (0.6.5.8-pve14~bpo80) ...
Installing new version of config file /etc/cron.d/zfsutils-linux ...
Installing new version of config file /etc/sudoers.d/zfs ...
Setting up zfs-initramfs (0.6.5.8-pve14~bpo80) ...
Setting up zfs-zed (0.6.5.8-pve14~bpo80) ...
Setting up libnvpair1 (0.6.5.8-pve14~bpo80) ...
Setting up libuutil1 (0.6.5.8-pve14~bpo80) ...
Setting up libzfs2 (0.6.5.8-pve14~bpo80) ...
Setting up libzpool2 (0.6.5.8-pve14~bpo80) ...
Setting up zfsutils (0.6.5.8-pve14~bpo80) ...
Processing triggers for libc-bin (2.19-18+deb8u7) ...
Processing triggers for initramfs-tools (0.120+deb8u2) ...
update-initramfs: Generating /boot/initrd.img-4.4.35-2-pve
Errors were encountered while processing:
pve-manager
proxmox-ve
E: Sub-process /usr/bin/dpkg returned an error code (1)

Just to be sure the entire installation is broken by doing a simple upgrade :

pveversion -v
Can't locate PVE/RESTEnvironment.pm in @INC (you may need to install the PVE::RESTEnvironment module) (@INC contains: /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.20.2 /usr/local/share/perl/5.20.2 /usr/lib/x86_64-linux-gnu/perl5/5.20 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.20 /usr/share/perl/5.20 /usr/local/lib/site_perl .) at /usr/share/perl5/PVE/RPCEnvironment.pm line 6.
BEGIN failed--compilation aborted at /usr/share/perl5/PVE/RPCEnvironment.pm line 6.
Compilation failed in require at /usr/share/perl5/PVE/API2/APT.pm line 20.
BEGIN failed--compilation aborted at /usr/share/perl5/PVE/API2/APT.pm line 20.
Compilation failed in require at /usr/bin/pveversion line 7.
BEGIN failed--compilation aborted at /usr/bin/pveversion line 7.

help ?

 

[svennd]

apt-get dist-upgrade ...

 

[deadbird]

Had the very same problem, started sweating, found this solution, solved the problem. Thanks a lot!!

 

[svennd]

Glad I'm not the only one

Tho this is a fair issue, normal "apt-get update's" should not break working setups, contrary to dist-upgrade that could break ... but I'm just glad I got out of this issue

 

[fabian]

Glad I'm not the only one

Tho this is a fair issue, normal "apt-get update's" should not break working setups, contrary to dist-upgrade that could break ... but I'm just glad I got out of this issue

PVE is a rolling release distribution, and therefore different rules apply (compared to stock Debian). We need to pull in new dependencies from time to time, and this only works with dist-upgrade (or the newer frontend called "apt", instead of "apt-get").

 

[Josh Douglas]

So what is the proper update procedure? It's debian based, so my experience has always been:

apt-get update; apt-get upgrade; apt-get dist-upgrade; apt-get autoremove

Is there a better way, other the update through the web interface?

 

[Ashley]

So what is the proper update procedure? It's debian based, so my experience has always been:

apt-get update; apt-get upgrade; apt-get dist-upgrade; apt-get autoremove

Is there a better way, other the update through the web interface?

apt-get update; apt-get dist-upgrade;

I would hold off the autoremove until you have checked everything is working correctly.

 

[Q-wulf]

Is there a better way, other the update through the web interface?

Afaik the way of the "web interface" is

apt-get update (when you hit the refresh button)
apt-get dist-upgrade (when you trigger an upgrade)

 

[svennd]

So what is the proper update procedure? It's debian based, so my experience has always been:

apt-get update; apt-get upgrade; apt-get dist-upgrade; apt-get autoremove

Is there a better way, other the update through the web interface?

apt-get update : updates repo cache
apt-get upgrade : updates all packages with security patches (not breaking updates)
apt-get dist-upgrade : updates all packages and things like kernel which might cause issue's and hence need to be tested prior to production.

Seems I was wrong, and dist-upgrade should be ran instead.

 

[fabian]

apt-get update : updates repo cache
apt-get upgrade : updates all packages with security patches (not breaking updates)
apt-get dist-upgrade : updates all packages and things like kernel which might cause issue's and hence need to be tested prior to production.

Seems I was wrong, and dist-upgrade should be ran instead.

that's not quite true

• "apt-get update" reloads package repositories
• "apt-get upgrade" upgrades installed packages, but does not remove any of them or install any new packages not already installed
• "apt-get dist-upgrade" upgrades installed packages, but does remove installed packages and/or install new packages to satisfy the dependencies of the updated packages

historically, "apt-get upgrade" was used for upgrades while running a stable release like Debian - where updated packages in a stable release will (almost) never introduce changed dependencies, because it is security and severe bug fixes only. "apt-get dist-upgrade" was used to upgrade to the next stable release, which usually meant installing and removing quite a lot of packages because of new upstream versions, changes in packaging, etc.pp. the above distinction does not apply to PVE with a rolling release model, because any update might introduce a new dependency (or conflict), and if you use apt-get upgrade you will partly upgrade packages, which is a bad idea

to make matters even more confusing, there is a new dpkg frontend called "apt", which unifies several of the "apt-*" tools, and offers:

• "apt update" works like "apt-get update", but with colors and progress
• "apt upgrade" works like "apt-get upgrade", but will pull in new dependencies (nice for things like kernel upgrades)
• "apt full-upgrade" or "apt dist-upgrade" works like "apt-get dist-upgrade"

none of the above cares about whether upgrades are security upgrades, feature upgrades, bug fixes, new upstream versions, ... in Debian, security upgrades get their own part of the repositories, so you can pin / script / .. your systems to only install security upgrades - many people auto-install security updates, but manually install point releases (with non-security bug fixes) for example. this does not work for PVE, as we don't package security fixes separately from other updates (this is one fundamental difference between our and e.g., Debian's, release model).

TL;DR: use PVE's built-in upgrade functionality, or use "apt-get update; apt-get dist-upgrade"!

 

[fabian]

and a small post-script: it's possible to configure apt-get / apt to break the above assumptions, e.g. by telling it to always auto-remove packages which are "not needed" anymore ("not needed" means marked as automatically installed as dependency of another package, but with nothing depending on it anymore now). such an auto-removal can also happen on an "upgrade" operation, but this behaviour is not enabled by default.

 

[svennd]

Actually I was looking for a way yesterday to disable "apt-get upgrade" so that nobody runs it in production... but I could't get it to work... I tried wrapping apt-get but failed... maybe you guys can give it a try... ?

 

[fabian]

Actually I was looking for a way yesterday to disable "apt-get update" so that nobody runs it in production... but I could't get it to work... I tried wrapping apt-get but failed... maybe you guys can give it a try... ?

I think/hope you mean "apt-get upgrade"?

I think the solution is an organizational one here - whoever has root access on your box needs to know how to correctly install updates on it.. PVE has a built-in upgrade mechanism that "does the right thing"™, so just tell your admins to use it.. you could of course swap out the apt-get binary for a script that replaces "upgrade" with "dist-upgrade" or something like that, but what about people running aptitude? apt? insert-favorite-dpkg/apt-frontend-here?

 

[svennd]

Yeah but before the interface could not handle updates to well, and it was recommended to use the console so

 

[fabian]

Yeah but before the interface could not handle updates to well, and it was recommended to use the console so

you can also use "pveupdate" and "pveupgrade" on the shell - but those are just very thin wrappers around "apt-get update" and "apt-get dist-upgrade".

 

[Jean-François Dagenais]

Broke my system... for the second time on an update. Last time was the grub boot problem. Why would anyone want to update their grub on a stable system... I am booted and in the system already! This is only risk. So I pinned the grub-pc package to workaround it.

Yes I falsely assumed an apt-get upgrade was "less risky" than a dist-upgrade. But guys, your packages should contain required dependency information as to not allow a package to be upgraded if it required another one to be install. Thus the apt-get upgrade should tell me that such and such packages have been held back. I'm no deb packages expert, but you guys should!

This is bad. I don't know if you have different maintenance policies on the enterprise repos, but this is bad.

I am so scared of upgrading now.