[SOLVED] Upgrade hanging

[Fladi]

Hi,

today I wanted to dist-upgrade my cluster. After the three "VM"-Server I wanted to upgrade the other 3 ceph-severs. But on the first one the "apt-get dist-upgrade" stucked.

[...]
Setting up ceph-common (10.2.6-1~bpo80+1) ...
Setting system user ceph properties..usermod: no changes
..done
Fixing /var/run/ceph ownership....done

After some hour of waiting I killed the process, removed the lock-file and did a "dpkg --configure -a". This now hangs again (same output).

The dpkg.log shows:

2017-03-08 19:11:22 status installed pve-kernel-4.4.40-1-pve:amd64 4.4.40-82
2017-03-08 19:11:22 configure ceph-common:amd64 10.2.6-1~bpo80+1 <none>
2017-03-08 19:11:22 status half-configured ceph-common:amd64 10.2.6-1~bpo80+1

Output of pveversion:

pve-manager/4.4-12/e71b7a74 (running kernel: 4.4.35-2-pve)

What would be the best way to proceed?

Thanks
Tim

 

[jrosengren]

Same thing happened to me... I killed off the "ceph.target start" process and the upgrade stopped hanging and completed.

 

[Fladi]

Thanks! this worked for me as well!

 

[jesusjimenez]

Same thing happened to me... I killed off the "ceph.target start" process and the upgrade stopped hanging and completed.

worked for me too

 

[oversite]

thanks. found it
"root 7155 7126 0 5622 2488 6 03:23 pts/1 00:00:00 /bin/systemctl start ceph.target"

 

[fabian]

there will be a fixed pve-manager package available soon, which will need to be installed before upgrading the ceph packages (the easiest way is to temporarily comment out the ceph repository in your APT sources). the workaround for hanging upgrades (killing the offending systemctl process) should have no side effects though, so if you have a non-HA setup this also works.

 

[fabian]

(note: only applies if you are using Ceph Jewel and used our upgrade howto or pveceph install!)

you can also apply the (proposed) fix manually before doing the upgrades:

1. change "WantedBy=ceph.target" to "WantedBy=multi-user.target" in /etc/systemd/system/ceph.service
2. run "systemctl daemon-reload; systemctl disable ceph.service; systemctl enable ceph.service"

now, "systemctl list-dependencies ceph.target" should no longer list PVE's "ceph.service", and the ceph-common package should no longer trigger a hang on upgrading.

 

[HFE]

Hi,

today I wanted to dist-upgrade my cluster. After the three "VM"-Server I wanted to upgrade the other 3 ceph-severs. But on the first one the "apt-get dist-upgrade" stucked.

[...]
Setting up ceph-common (10.2.6-1~bpo80+1) ...
Setting system user ceph properties..usermod: no changes
..done
Fixing /var/run/ceph ownership....done

After some hour of waiting I killed the process, removed the lock-file and did a "dpkg --configure -a". This now hangs again (same output).

The dpkg.log shows:

2017-03-08 19:11:22 status installed pve-kernel-4.4.40-1-pve:amd64 4.4.40-82
2017-03-08 19:11:22 configure ceph-common:amd64 10.2.6-1~bpo80+1 <none>
2017-03-08 19:11:22 status half-configured ceph-common:amd64 10.2.6-1~bpo80+1

Output of pveversion:

pve-manager/4.4-12/e71b7a74 (running kernel: 4.4.35-2-pve)

What would be the best way to proceed?

Thanks
Tim

executing 'systemctl daemon-reexec' worked for me

 

[eXtremeSHOk]

Did this as a fix on our HA clusters.
Run the following either before the update or whilst the update has frozen

sed -i "s/=ceph.target/=multi-user.target/" /etc/systemd/system/ceph.service
systemctl daemon-reload; systemctl disable ceph.service; systemctl enable ceph.service; systemctl daemon-reexec