PVE Wont Start (Freezes) After Apt Package Install

alexinux

Member
Aug 21, 2023
31
2
13
Hey all,

I attempted to make one of our PVE Hosts a Local Apt Repository since we have a stand-alone environment and everything was going good until I attempted to install "dpkg-dev" package. I first installed "apache2" to created the repository to where the packages would be advertised without an issue but when I attempted to install "dpkg-dev" package it wouldn't install via the USB repository we have. I thought my repository did not have all dependencies or packages for this so I went and tried apt-download on another system to download "dpkg-dev" and all dependencies to then install them using the "dpkg -i *.deb" command. After this last command I rebooted but now the system stalls/freezes on the line on the screenshot below. I cannot even boot to any other kernel/option on grub because it causes a "kernel panic" and the "recovery options" on grub freeze as well. Even trying the debug option freezes so I don't know what to do anymore.

The questions are:

1. Did I break the system or the Apt packages within the system?
2. Is there a way to install the below to get the system to boot properly?
3. If there is no going back, can I recover the cluster configuration files by booting to Live Debian or something and then reinstalling?
4. Is there a way to revert the package installations and reinstall everything again?
** I ran apt-mirror again on my USB Repository to make sure I had not messed anything up. **

Any help or guidance is appreciated and I can provide more screens regarding the boot as well.

PVE Startup Error.jpg
 
Hello,

On the Proxmox 8 and 9 ISO you have a rescue disk option in the Advanced section. You can try to see if he can boot your server loading your Proxmox configuration.
 
Hello,

On the Proxmox 8 and 9 ISO you have a rescue disk option in the Advanced section. You can try to see if he can boot your server loading your Proxmox configuration.
I tried that and I'm getting stuck in the blow line. I'm using the latest ISO.

Rescue Boot JPG.jpg
 
Maybe you can try something like this to get your configuration file in a first time, then try to install package to correct your error : https://wiki.debian.org/RescueLive
I got to boot with the Installer Debug Mode. I guess now is to mount the pve-root directory and extract everything in /etc?

What would be the files that I need to extract so when I reinstall everything can go back to normal? I have Proxmox on an NVME and a Zpool which holds the VMs and respective disks separately.
 
You can backup the /etc folder but be sure that the /etc/pve directory is available with data, this directory is mounted by the service pve-cluster so be sure that the service is running.

You can also backup the config.db according to this documentation : https://pve.proxmox.com/wiki/Proxmox_Cluster_File_System_(pmxcfs)#_recovery

Normally after an OS reinstall you will be able to import your existing zfs pool that store your VM disk : https://unix.stackexchange.com/ques...ol-and-storage-data-after-a-system-re-install
 
I did get into a Debian Rescue Boot and when attempting to repair the packages I get the following:

If I do "apt --fix-broken install" nothing gets fixed. Is there something I can do to get this packages fixed individually or via other means?

Debian Rescue Boot.jpg
 
You can backup the /etc folder but be sure that the /etc/pve directory is available with data, this directory is mounted by the service pve-cluster so be sure that the service is running.

You can also backup the config.db according to this documentation : https://pve.proxmox.com/wiki/Proxmox_Cluster_File_System_(pmxcfs)#_recovery

Normally after an OS reinstall you will be able to import your existing zfs pool that store your VM disk : https://unix.stackexchange.com/ques...ol-and-storage-data-after-a-system-re-install

I can backup all those folders but I checked the /etc/pve and its empty, all others have something but not this one. I can't seem to start the service in chroot mode from the debian rescue environment.

** Also I found with the cat /var/log/dpkg.log |grep 'install' that the last packages installed are ubuntu and not debian, hence the issue.

*** Is there a way to replace this packages with the debian ones? I downloaded them all and tried the dpgk command but it gets me a bunch of errors from dependencies.

**** Doing the apt purge <packagename> it gets the same errors regarding dependencies.
 
There is no apt --fix-broken install executed in the picture. Only apt-get upgrade -y
Have you run apt --fix-broken install actually?
What is the output of it?
I have.... it doesn't fix anything broken. I was attempting to install all those packages manually but there is still the problem of the ubuntu packages.
 
*** Is there a way to replace this packages with the debian ones? I downloaded them all and tried the dpgk command but it gets me a bunch of errors from dependencies.

WHAT dpkg command exactly? Please quote it precisely. And the output.
It's difficult to try to help you when you aren't showing exact commands and results.
 
Last edited:
WHAT dpkg command exactly? Please quote it precisely. And the output.
It's difficult to try to help you when you aren't showing exact commands and results.
cd /path/to/.debfiles
dpkg -i *.deb

This was to install everything inside the directory.