no network after restarting PVE

Mar 20, 2022
19
7
8
I was unable to access the web UI in my proxmox host for days (that I noticed, but probably longer), even though networking on VMs was working perfectly fine. I decided to power cycle it in hopes that would get things running properly again, but I got the following errors:

Code:
  Found volume group "cheap" using metadata type lvm2
  2 logical volume(s) in volume group "cheap" now active
[FAILED] Failed to start systemd-jo...ush Journal to Persistent Storage.
[FAILED] Failed to start systemd-ud...To Complete Device Initialization.
[DEPEND] Dependency failed for zfs-...port ZFS pools by device scanning.
[DEPEND] Dependency failed for zfs-... - Import ZFS pools by cache file.
[FAILED] Failed to start ifupdown2-... synchronize boot up for ifupdown.
[DEPEND] Dependency failed for netw...vice - Network initialization.

All the messages being truncated makes it very difficult to understand what's going on.

After about 30-45 minutes it looks like proxmox is running, but cannot access the UI or ping the server. This time the VMs also have no network access. Any guidance on what I can try? I think I can at least login to the local terminal as I get the PVE welcome screen.
 
Thank you.

Looks like these have failed, not sure what to do with this yet, but posting here while I research it.

- ifupdown2-pre.service
- smartmontools.server (this did not fail the first several reboots, but did on the last one)
- systemd-journal-flush.service
- systemd-udev-settle.service

systemd-journal-flush is the first one that appeared on the console log, so figured I'd start with that one. It's failing due to a "timeout", which isn't really that helpful. I tried the solution posted here (https://unix.stackexchange.com/questions/740162), but that did not resolve anything (it was a shot in the dark as that directory did exist for me)
 
Last edited:
I think the issue is a disk is bad or going bad. I'm having trouble reading the output on the host console and tried adjusting the resolution through grub. Doing a `update-grub` is taking an extremely long time and I got an error when it tried to copy the kernel

Code:
Copying and configuring kernels on /dev/disk/by-uuid/2383-B3D3
        Copying kernal 5.15.158-2-pve
cp: error writing '/var/tmp/espmounts/2383-B3D3/initrd.img-5.15.158-2-pve': Input/output error

My proxmox instance is installed to a mirrored set of SSDs, so hopefully this is something I can recover from. Seems like reading files is working file, but writing them is slow (even writing small text files).
 
One of the mirrored drives was bad. Disconnecting it allowed proxmox to boot up with no problems. Going to check how to attach a new drive to replace it.