[SOLVED] lxc upgrade to bullseye trouble

R

RobFantini

Famous Member
May 24, 2012
2,022
107
133
Boston,Mass
hello, while doing apt dist-upgrade to bullseye we have seen upgrades get stuck at this point:

Code: 
Installing new version of config file /etc/systemd/resolved.conf ...
Installing new version of config file /etc/systemd/system.conf ...
Installing new version of config file /etc/systemd/user.conf ...
Created symlink /etc/systemd/system/sysinit.target.wants/systemd-pstore.service -> /lib/systemd/system/systemd-pstore.service.
Failed to get load state of systemd-networkd.socket: No buffer space available
Failed to try-restart systemd-networkd.service: Transport endpoint is not connected
See system logs and 'systemctl status systemd-networkd.service' for details.
Failed to try-restart systemd-resolved.service: Transport endpoint is not connected
See system logs and 'systemctl status systemd-resolved.service' for details.

after a few minutes the upgrade sometimes continues.

the lxc .conf is :

Code: 
arch: amd64
cpulimit: 4
cpuunits: 1024
features: nesting=1
hostneme: emby
memory: 8192
net0: bridge=vmbr0,gw=172.30.24.2,hwaddr=32:64:31:34:32:38,ip=172.30.24.17/24,name=eth0,type=veth
onboot: 1
ostype: debian
parent: preUp
protection: 1
rootfs: lxc2:subvol-22217-disk-0,size=320G
swap: 1024
lxc.mount.entry: /ht  ht  none bind,create=dir,optional 0 0

so apt dist-upgrade completed

however after that :
- can not shutdown the pct from pve: command 'lxc-stop -n 22217 --nokill --timeout 60' failed: exit code 1

- stopped from cli.
- then started
- the pct has no network address after starting:
Code: 
# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0@if26: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 32:64:31:34:32:38 brd ff:ff:ff:ff:ff:ff link-netnsid 0

this hangs, does not return anything:
systemctl status


Any ideas on getting this system fixed?
 
after rebooting the pve host the lxc got a network address [ probably restarting a systemd unit would have done the same thing.]

however this still freezes:
systemctl status

and media server does not work.

I'll try starting in debug mode later on.
 
debug mode output
Code: 
# cat start-debug-mode   
lxc-start -n $1     -F --logfile=lxc.log --logpriority=debug

root@pve33:[/fbc/pve/lxc]:# ./start-debug-mode 22217
systemd 247.3-6 running in system mode. (+PAM +AUDIT +SELINUX +IMA +APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +ZSTD +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=unified)
Detected virtualization lxc.
Detected architecture x86-64.

Welcome to Debian GNU/Linux 11 (bullseye)!

Set hostname to <emby-bullseye>.
/etc/systemd/system/rc-local.service:11: Support for option SysVStartPriority= has been removed and it is ignored
Queued start job for default target Graphical Interface.
[  OK  ] Created slice system-container\x2dgetty.slice.
[  OK  ] Created slice system-modprobe.slice.
[  OK  ] Created slice system-postfix.slice.
[  OK  ] Created slice User and Session Slice.
[  OK  ] Started Dispatch Password Requests to Console Directory Watch.
[  OK  ] Started Forward Password Requests to Wall Directory Watch.
[  OK  ] Reached target Local Encrypted Volumes.
[  OK  ] Reached target Remote File Systems.
[  OK  ] Reached target Slices.
[  OK  ] Reached target Swap.
[  OK  ] Listening on Device-mapper event daemon FIFOs.
[  OK  ] Listening on LVM2 poll daemon socket.
[  OK  ] Listening on Syslog Socket.
[  OK  ] Listening on initctl Compatibility Named Pipe.
[  OK  ] Listening on Journal Audit Socket.
[  OK  ] Listening on Journal Socket (/dev/log).
[  OK  ] Listening on Journal Socket.
         Mounting Huge Pages File System...
         Mounting POSIX Message Queue File System...
[  OK  ] Finished Availability of block devices.
         Starting Monitoring of LVM2 mirrors, snapshots etc. using dmeventd or progress polling...
         Starting Journal Service...
         Starting Remount Root and Kernel File Systems...
         Starting Apply Kernel Variables...
         Starting Helper to synchronize boot up for ifupdown...
[  OK  ] Mounted Huge Pages File System.
[  OK  ] Mounted POSIX Message Queue File System.
[  OK  ] Finished Remount Root and Kernel File Systems.
         Starting Initial Check File System Quotas...
         Starting Create System Users...
[  OK  ] Finished Helper to synchronize boot up for ifupdown.
[  OK  ] Finished Initial Check File System Quotas.
[  OK  ] Finished Create System Users.
         Starting Create Static Device Nodes in /dev...
[  OK  ] Finished Apply Kernel Variables.
[  OK  ] Finished Create Static Device Nodes in /dev.
[  OK  ] Started Journal Service.
         Starting Flush Journal to Persistent Storage...
[  OK  ] Finished Monitoring of LVM2 mirrors, snapshots etc. using dmeventd or progress polling.
[  OK  ] Reached target Local File Systems (Pre).
[  OK  ] Reached target Local File Systems.
         Starting Raise network interfaces...
[  OK  ] Finished Flush Journal to Persistent Storage.
         Starting Create Volatile Files and Directories...
[  OK  ] Finished Create Volatile Files and Directories.
[  OK  ] Reached target System Time Set.
[  OK  ] Reached target System Time Synchronized.
         Starting Update UTMP about System Boot/Shutdown...
[  OK  ] Finished Update UTMP about System Boot/Shutdown.
[  OK  ] Reached target System Initialization.
[  OK  ] Started ACPI Events Check.
[  OK  ] Started Daily apt download activities.
[  OK  ] Started Daily apt upgrade and clean activities.
[  OK  ] Started Periodic ext4 Online Metadata Check for All Filesystems.
[  OK  ] Started Daily rotation of log files.
[  OK  ] Started Daily man-db regeneration.
[  OK  ] Started Updates mlocate database every day.
[  OK  ] Started Daily Cleanup of Temporary Directories.
[  OK  ] Reached target Paths.
[  OK  ] Reached target Timers.
[  OK  ] Listening on ACPID Listen Socket.
[  OK  ] Listening on D-Bus System Message Bus Socket.
[  OK  ] Listening on UUID daemon activation socket.
[  OK  ] Reached target Sockets.
[  OK  ] Reached target Basic System.


note I restored backup to a diff pct number .

the broken one is still available to be worked on.
 
this lxc was originally created using Turnkey Debian 8 template some years ago [ when Debian 8 was stable ].

also the lxc restored from backup does have some issues
Code: 
# systemctl list-units --state=failed
  UNIT               LOAD   ACTIVE SUB    DESCRIPTION                                            
● inithooks.service  loaded failed failed inithooks: firstboot and everyboot initialization scripts
● networking.service loaded failed failed Raise network interfaces

I suppose next those should be fixed then upgrade to bullseye attempted.
 
Last edited:
I'll get back to this later on. here is more info
Code: 
# systemctl status inithooks.service
● inithooks.service - inithooks: firstboot and everyboot initialization scripts
   Loaded: loaded (/lib/systemd/system/inithooks.service; enabled; vendor preset: enabled)
   Active: failed (Result: exit-code) since Wed 2021-09-29 01:56:50 EDT; 19min ago
  Process: 73 ExecStart=/bin/sh -c      FGCONSOLE=$(fgconsole);      openvt -f -c 8 -s -w -- ${INITHOOKS_PATH}/run;      chvt $FGCONSOL
 Main PID: 73 (code=exited, status=208/STDIN)
      CPU: 807us

Sep 29 01:56:50 emby-536 systemd[1]: Starting inithooks: firstboot and everyboot initialization scripts...
Sep 29 01:56:50 emby-536 systemd[73]: inithooks.service: Failed to set up standard input: No such file or directory
Sep 29 01:56:50 emby-536 systemd[73]: inithooks.service: Failed at step STDIN spawning /bin/sh: No such file or directory
Sep 29 01:56:50 emby-536 systemd[1]: inithooks.service: Main process exited, code=exited, status=208/STDIN
Sep 29 01:56:50 emby-536 systemd[1]: inithooks.service: Failed with result 'exit-code'.
Sep 29 01:56:50 emby-536 systemd[1]: Failed to start inithooks: firstboot and everyboot initialization scripts.
Sep 29 01:56:50 emby-536 systemd[1]: inithooks.service: Consumed 807us CPU time


:# systemctl status networking.service
● networking.service - Raise network interfaces
   Loaded: loaded (/lib/systemd/system/networking.service; enabled; vendor preset: enabled)
   Active: failed (Result: exit-code) since Wed 2021-09-29 01:56:50 EDT; 22min ago
     Docs: man:interfaces(5)
  Process: 94 ExecStart=/sbin/ifup -a --read-environment (code=exited, status=1/FAILURE)
 Main PID: 94 (code=exited, status=1/FAILURE)
      CPU: 71ms

Sep 29 01:56:50 emby-536 systemd[1]: Starting Raise network interfaces...
Sep 29 01:56:50 emby-536 ifup[94]: udhcpc: SIOCGIFINDEX: No such device
Sep 29 01:56:50 emby-536 ifup[94]: ifup: failed to bring up eth1
Sep 29 01:56:50 emby-536 systemd[1]: networking.service: Main process exited, code=exited, status=1/FAILURE
Sep 29 01:56:50 emby-536 systemd[1]: networking.service: Failed with result 'exit-code'.
Sep 29 01:56:50 emby-536 systemd[1]: Failed to start Raise network interfaces.
Sep 29 01:56:50 emby-536 systemd[1]: networking.service: Consumed 71ms CPU time.
 
interfaces has eth0 and eth1 , when pve networking config has just one device . weird.
Code: 
# cat interfaces
# UNCONFIGURED INTERFACES
# remove the above line if you edit this file

auto lo
iface lo inet loopback

auto eth0
iface eth0 inet static
        address 172.30.24.17/24
        gateway 172.30.24.2

auto eth1
iface eth1 inet dhcp

so i'll remove the eth1 part and the comments at top

then reboot:

that fixed network error line .

now just this:

Code: 
# systemctl list-units --state=failed
  UNIT              LOAD   ACTIVE SUB    DESCRIPTION                                             
● inithooks.service loaded failed failed inithooks: firstboot and everyboot initialization scripts
 
RobFantini said:
systemctl status inithooks.service
Click to expand...
This is not a service that plain Debian use (nowadays), probably from TurnKey? May be worth to ask about that one in their support channels then. It could be that it tries to open the wrong /dev/ tty/console device.

RobFantini said:
Sep 29 01:56:50 emby-536 ifup[94]: udhcpc: SIOCGIFINDEX: No such device
Sep 29 01:56:50 emby-536 ifup[94]: ifup: failed to bring up eth1
Click to expand...
It tries to bring up eth1 but your ip a output only showed eth0 ?
 
  • Like
Reactions: RobFantini
t.lamprecht said:
This is not a service that plain Debian use (nowadays), probably from TurnKey? May be worth to ask about that one in their support channels then. It could be that it tries to open the wrong /dev/ tty/console device.


It tries to bring up eth1 but your ip a output only showed eth0 ?
Click to expand...


- i had a left over from testing eth1 in interfaces . removing that fixed .

- the cause of my other issue is probably due to removing /etc/apt/sources.list.d/turnkey.list some years ago - then not getting updates.

thank you
 
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom