Lxc start error- subvol-ID-disk -No such file or directory

P

penguin2020

Active Member
Jul 2, 2018
16
1
43
Hi everybady

I have been testing (in addition to being using it in production) an installation with ZFS in root and in the pools where I have the disks of my VM and CT

I have updated to the latest version in the laboratory (cluster with physical equipment for testing and development) and the only container I have, it is not possible to start, something happens like it cannot find the disk that is mounted (or should) in a zfs subvolume...
But if I create a new container, using the same storage, it is created fine, and it starts correctly
With the VM (qemu) there is no problem.
I think the issue is due to the ZFS subvolumes, that the system is not mounting them correctly, but I can not find the solution

Some data of the system and the container that does not start:

When wanting to start it:

TASK ERROR: zfs error: /saspool/containers/subvol-106-disk-0: No such file or directory

PVE Version

Code: 
root@pvedev1:~# pveversion -v
proxmox-ve: 6.2-2 (running kernel: 5.4.65-1-pve)
pve-manager: 6.2-15 (running version: 6.2-15/48bd51b6)
pve-kernel-5.4: 6.2-7
pve-kernel-helper: 6.2-7
pve-kernel-5.4.65-1-pve: 5.4.65-1
pve-kernel-5.4.44-2-pve: 5.4.44-2
pve-kernel-5.4.41-1-pve: 5.4.41-1
pve-kernel-5.4.34-1-pve: 5.4.34-2
ceph-fuse: 14.2.11-pve1
corosync: 3.0.4-pve1
criu: 3.11-3
glusterfs-client: 5.5-3
ifupdown: 0.8.35+pve1
ksm-control-daemon: 1.3-1
libjs-extjs: 6.0.1-10
libknet1: 1.16-pve1
libproxmox-acme-perl: 1.0.5
libpve-access-control: 6.1-3
libpve-apiclient-perl: 3.0-3
libpve-common-perl: 6.2-4
libpve-guest-common-perl: 3.1-3
libpve-http-server-perl: 3.0-6
libpve-storage-perl: 6.2-10
libqb0: 1.0.5-1
libspice-server1: 0.14.2-4~pve6+1
lvm2: 2.03.02-pve4
lxc-pve: 4.0.3-1
lxcfs: 4.0.3-pve3
novnc-pve: 1.1.0-1
proxmox-backup-client: 1.0.1-1
proxmox-mini-journalreader: 1.1-1
proxmox-widget-toolkit: 2.3-10
pve-cluster: 6.2-1
pve-container: 3.2-2
pve-docs: 6.2-6
pve-edk2-firmware: 2.20200531-1
pve-firewall: 4.1-3
pve-firmware: 3.1-3
pve-ha-manager: 3.1-1
pve-i18n: 2.2-2
pve-qemu-kvm: 5.1.0-6
pve-xtermjs: 4.7.0-2
qemu-server: 6.2-19
smartmontools: 7.1-pve2
spiceterm: 3.1-1
vncterm: 1.6-2
zfsutils-linux: 0.8.4-pve2

CT Configuration

Code: 
arch: amd64
cores: 4
hostname: odo
memory: 2048
net0: name=eth0,bridge=vmbr0,gw=192.168.30.1,hwaddr=EE:EC:99:26:25:ED,ip=192.168.30.190/24,type=veth
ostype: debian
rootfs: containers:subvol-106-disk-0,size=128G
swap: 1024

Storage Configuration


Code: 
root@pvedev1:~# cat /etc/pve/storage.cfg
dir: local
        path /var/lib/vz
        content iso,images,backup,vztmpl
        maxfiles 0
        shared 0

zfspool: local-zfs
        pool rpool/data
        content rootdir,images
        sparse 1

zfspool: vm-disk
        pool saspool/vm-disk
        content images
        mountpoint /saspool/vm-disk
        nodes pvedev2,pvedev1
        sparse 1

zfspool: containers
        pool saspool/containers
        content images,rootdir
        mountpoint /saspool/containers
        nodes pvedev2,pvedev1
        sparse 0

dir: dump-isos
        path /saspool/dump-isos
        content backup,iso,vztmpl,images,snippets
        maxfiles 3
        nodes pvedev2,pvedev1
        shared 0

nfs: nas_omv11
        export /export/NAS_omv11
        path /mnt/pve/nas_omv11
        server 10.0.3.3
        content images,iso,backup
        maxfiles 4

zfs list

Code: 
root@pvedev1:/saspool# zfs list -r -o name,mountpoint,mounted
NAME                                  MOUNTPOINT                             MOUNTED
rpool                                 /rpool                                     yes
rpool/ROOT                            /rpool/ROOT                                yes
rpool/ROOT/pve-1                      /                                          yes
rpool/data                            /rpool/data                                yes
saspool                               /saspool                                    no
saspool/containers                    /saspool/containers                         no
saspool/containers/subvol-106-disk-0  /saspool/containers/subvol-106-disk-0       no
saspool/dump-isos                     /saspool/dump-isos                          no
saspool/vm-disk                       /saspool/vm-disk                            no
saspool/vm-disk/vm-101-disk-0         -                                            -
saspool/vm-disk/vm-1010-disk-0        -                                            -
saspool/vm-disk/vm-102-disk-0         -                                            -
saspool/vm-disk/vm-1020-disk-0        -                                            -
saspool/vm-disk/vm-103-disk-0         -                                            -
saspool/vm-disk/vm-104-disk-0         -                                            -

File systems


Code: 
root@pvedev1:/# df -HT
Filesystem                 Type      Size  Used Avail Use% Mounted on
udev                       devtmpfs   17G     0   17G   0% /dev
tmpfs                      tmpfs     3.4G  9.6M  3.4G   1% /run
rpool/ROOT/pve-1           zfs       579G  5.0G  574G   1% /
tmpfs                      tmpfs      17G   66M   17G   1% /dev/shm
tmpfs                      tmpfs     5.3M     0  5.3M   0% /run/lock
tmpfs                      tmpfs      17G     0   17G   0% /sys/fs/cgroup
rpool                      zfs       574G  132k  574G   1% /rpool
rpool/ROOT                 zfs       574G  132k  574G   1% /rpool/ROOT
rpool/data                 zfs       574G  132k  574G   1% /rpool/data
/dev/fuse                  fuse       32M   33k   32M   1% /etc/pve
10.0.3.3:/export/NAS_omv11 nfs       2.0T  993G  976G  51% /mnt/pve/nas_omv11
tmpfs                      tmpfs     3.4G     0  3.4G   0% /run/user/0

pvesm status

Code: 
root@pvemgua1:~# pvesm status
Name              Type     Status           Total            Used       Available        %
containers     zfspool     active       946800896          937712       945863184    0.10%
dump-isos          dir     active       564781312         4847744       559933568    0.86%
local              dir     active       564781312         4847744       559933568    0.86%
local-zfs      zfspool     active       559933684              96       559933588    0.00%
nas_omv11          nfs     active      1921554432       969037824       952500224   50.43%
vm-disk        zfspool     active      1125358292       179495108       945863184   15.95%

lxc-start verbose

Code: 
root@pvedev1:~# lxc-start -n 106 -F -l DEBUG -o /tmp/lxc-106.log
lxc-start: 106: conf.c: run_buffer: 323 Script exited with status 2
lxc-start: 106: start.c: lxc_init: 797 Failed to run lxc.hook.pre-start for container "106"
lxc-start: 106: start.c: __lxc_start: 1896 Failed to initialize container "106"
lxc-start: 106: conf.c: run_buffer: 323 Script exited with status 1
lxc-start: 106: start.c: lxc_end: 964 Failed to run lxc.hook.post-stop for container "106"
lxc-start: 106: tools/lxc_start.c: main: 308 The container failed to start
lxc-start: 106: tools/lxc_start.c: main: 314 Additional information can be obtained by setting the --logfile and --logpriority options

cat /tmp/lxc-106.log

Code: 
root@pvedev1:~# cat /tmp/lxc-106.log
lxc-start 106 20201111181813.127 INFO     lsm - lsm/lsm.c:lsm_init:29 - LSM security driver AppArmor
lxc-start 106 20201111181813.128 INFO     conf - conf.c:run_script_argv:340 - Executing script "/usr/share/lxc/hooks/lxc-pve-prestart-hook" for container "106", config section "lxc"
lxc-start 106 20201111181813.755 DEBUG    conf - conf.c:run_buffer:312 - Script exec /usr/share/lxc/hooks/lxc-pve-prestart-hook 106 lxc pre-start produced output: cannot open directory //saspool/containers/subvol-106-disk-0: No such file or directory

lxc-start 106 20201111181813.769 ERROR    conf - conf.c:run_buffer:323 - Script exited with status 2
lxc-start 106 20201111181813.769 ERROR    start - start.c:lxc_init:797 - Failed to run lxc.hook.pre-start for container "106"
lxc-start 106 20201111181813.769 ERROR    start - start.c:__lxc_start:1896 - Failed to initialize container "106"
lxc-start 106 20201111181813.769 INFO     conf - conf.c:run_script_argv:340 - Executing script "/usr/share/lxcfs/lxc.reboot.hook" for container "106", config section "lxc"
lxc-start 106 20201111181814.272 INFO     conf - conf.c:run_script_argv:340 - Executing script "/usr/share/lxc/hooks/lxc-pve-poststop-hook" for container "106", config section "lxc"
lxc-start 106 20201111181814.990 DEBUG    conf - conf.c:run_buffer:312 - Script exec /usr/share/lxc/hooks/lxc-pve-poststop-hook 106 lxc post-stop produced output: umount: /var/lib/lxc/106/rootfs: not mounted

lxc-start 106 20201111181814.990 DEBUG    conf - conf.c:run_buffer:312 - Script exec /usr/share/lxc/hooks/lxc-pve-poststop-hook 106 lxc post-stop produced output: command 'umount --recursive -- /var/lib/lxc/106/rootfs' failed: exit code 1

lxc-start 106 20201111181815.207 ERROR    conf - conf.c:run_buffer:323 - Script exited with status 1
lxc-start 106 20201111181815.218 ERROR    start - start.c:lxc_end:964 - Failed to run lxc.hook.post-stop for container "106"
lxc-start 106 20201111181815.224 ERROR    lxc_start - tools/lxc_start.c:main:308 - The container failed to start
lxc-start 106 20201111181815.227 ERROR    lxc_start - tools/lxc_start.c:main:314 - Additional information can be obtained by setting the --logfile and --logpriority options
lxc-start 106 20201111182737.869 INFO     lsm - lsm/lsm.c:lsm_init:29 - LSM security driver AppArmor
lxc-start 106 20201111182737.869 INFO     conf - conf.c:run_script_argv:340 - Executing script "/usr/share/lxc/hooks/lxc-pve-prestart-hook" for container "106", config section "lxc"
lxc-start 106 20201111182738.605 DEBUG    conf - conf.c:run_buffer:312 - Script exec /usr/share/lxc/hooks/lxc-pve-prestart-hook 106 lxc pre-start produced output: cannot open directory //saspool/containers/subvol-106-disk-0: No such file or directory

lxc-start 106 20201111182738.616 ERROR    conf - conf.c:run_buffer:323 - Script exited with status 2
lxc-start 106 20201111182738.616 ERROR    start - start.c:lxc_init:797 - Failed to run lxc.hook.pre-start for container "106"
lxc-start 106 20201111182738.616 ERROR    start - start.c:__lxc_start:1896 - Failed to initialize container "106"
lxc-start 106 20201111182738.616 INFO     conf - conf.c:run_script_argv:340 - Executing script "/usr/share/lxcfs/lxc.reboot.hook" for container "106", config section "lxc"
lxc-start 106 20201111182739.119 INFO     conf - conf.c:run_script_argv:340 - Executing script "/usr/share/lxc/hooks/lxc-pve-poststop-hook" for container "106", config section "lxc"
lxc-start 106 20201111182739.826 DEBUG    conf - conf.c:run_buffer:312 - Script exec /usr/share/lxc/hooks/lxc-pve-poststop-hook 106 lxc post-stop produced output: umount: /var/lib/lxc/106/rootfs: not mounted

lxc-start 106 20201111182739.826 DEBUG    conf - conf.c:run_buffer:312 - Script exec /usr/share/lxc/hooks/lxc-pve-poststop-hook 106 lxc post-stop produced output: command 'umount --recursive -- /var/lib/lxc/106/rootfs' failed: exit code 1

lxc-start 106 20201111182739.836 ERROR    conf - conf.c:run_buffer:323 - Script exited with status 1
lxc-start 106 20201111182739.837 ERROR    start - start.c:lxc_end:964 - Failed to run lxc.hook.post-stop for container "106"
lxc-start 106 20201111182739.837 ERROR    lxc_start - tools/lxc_start.c:main:308 - The container failed to start
lxc-start 106 20201111182739.837 ERROR    lxc_start - tools/lxc_start.c:main:314 - Additional information can be obtained by setting the --logfile and --logpriority options


I really appreciate your comments
 
Finally I was can start the container

Code: 
# rm -rf /saspool/containers/
# zfs mount saspool/containers/subvol-106-disk-0

this sounds like Proxmox wanted to mount it, but couldn't
Any ideas to fix it and keep it on every reboot?
 
Hello, dear friends of the community


since i upgrade from version 6.1-x to 6.2-x, my containers are not starting
I have created a new one, and when restarting, it does not start giving this error
Any idea what might be happening
Thanks in advance

Code: 
mount_entry: 1892 No such file or directory - Failed to mount "run" on "/usr/lib/x86_64-linux-gnu/lxc/rootfs/run"
lxc_setup: 3256 Failed to setup mount entries
do_start: 1224 Failed to setup container "107"
__sync_wait: 41 An error occurred in another process (expected sequence number 5)
__lxc_start: 1950 Failed to spawn container "107"
TASK ERROR: startup for container '107' failed
 
I'm in this situation

Code: 
root@pvemgua1:~# zfs list -o name,mounted
NAME                            MOUNTED
rpool                               yes
rpool/ROOT                          yes
rpool/ROOT/pve-1                    yes
rpool/data                          yes
saspool                              no
saspool/containers                   no
saspool/dump-isos                    no
saspool/vm-disk                      no
saspool/vm-disk/vm-101-disk-0         -
saspool/vm-disk/vm-1010-disk-0        -
saspool/vm-disk/vm-102-disk-0         -
saspool/vm-disk/vm-1020-disk-0        -
saspool/vm-disk/vm-103-disk-0         -
saspool/vm-disk/vm-104-disk-0         -
 
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom
Back