LV for storage directory timing out at boot (Timed out waiting for device (...) failed with result 'dependency')

[Chris.M]

Greetings

I have set-up two storage directory, one for VZDump backup files and one for ISO images.
Those two directory are ext3 LVs on an existing VG (disk2).
VG disk2 is composed of only one PV (/dev/sdc1).

Issue: Each time I boot I get a `Timed out waiting for device` for each of my LVs used for storage directory.
In order to complete the boot, I have to the mount the LVs myself in the emergency mode shell (`mount -a`).

It's my first time setting up storage directory btw so I suspect I might have done something incorectly.
I'm sure someone more experienced will figure this out ;-)

Lastly I am using Proxmox in a homelab for educationnal purposes.

-----------------------
Proxmox version = 7.4-17
Linux HostnameRedacted 5.15.126-1-pve #1 SMP PVE 5.15.126-1 (2023-10-03T17:24Z) x86_64 GNU/Linux


Journalctl:

systemd[1]: Timed out waiting for device /dev/mapper/disk2-disk2--3.
systemd[1]: Dependency failed for File System Check on /dev/mapper/disk2-disk2--3.
systemd[1]: Dependency failed for /mnt/disk2/disk2-3.
systemd[1]: Dependency failed for Local File Systems.
systemd[1]: local-fs.target: Job local-fs.target/start failed with result 'dependency'.
systemd[1]: local-fs.target: Triggering OnFailure= dependencies.
systemd[1]: mnt-disk2-disk2\x2d3.mount: Job mnt-disk2-disk2\x2d3.mount/start failed with result 'dependency'.
systemd[1]: systemd-fsck@dev-mapper-disk2\x2ddisk2\x2d\x2d3.service: Job systemd-fsck@dev-mapper-disk2\x2ddisk2\x2d\x2d3.service/start failed with result 'dependency'.
systemd[1]: dev-mapper-disk2\x2ddisk2\x2d\x2d3.device: Job dev-mapper-disk2\x2ddisk2\x2d\x2d3.device/start failed with result 'timeout'.
systemd[1]: dev-mapper-disk2\x2ddisk2\x2d\x2d4.device: Job dev-mapper-disk2\x2ddisk2\x2d\x2d4.device/start timed out.
systemd[1]: Timed out waiting for device /dev/mapper/disk2-disk2--4.
systemd[1]: Dependency failed for File System Check on /dev/mapper/disk2-disk2--4.
systemd[1]: Dependency failed for /mnt/disk2/disk2-4.
systemd[1]: mnt-disk2-disk2\x2d4.mount: Job mnt-disk2-disk2\x2d4.mount/start failed with result 'dependency'.
systemd[1]: systemd-fsck@dev-mapper-disk2\x2ddisk2\x2d\x2d4.service: Job systemd-fsck@dev-mapper-disk2\x2ddisk2\x2d\x2d4.service/start failed with result 'dependency'.
systemd[1]: dev-mapper-disk2\x2ddisk2\x2d\x2d4.device: Job dev-mapper-disk2\x2ddisk2\x2d\x2d4.device/start failed with result 'timeout'.

pvscan:

PV /dev/sdc1 VG disk2 lvm2 [<7.28 TiB / <283.41 GiB free]
PV /dev/sdb1 VG disk3 lvm2 [<931.51 GiB / 48.00 MiB free]
PV /dev/sda3 VG pve lvm2 [<118.74 GiB / 14.75 GiB free]
Total: 3 [8.30 TiB] / in use: 3 [8.30 TiB] / in no VG: 0 [0 ]

vgscan:

Found volume group "disk2" using metadata type lvm2
Found volume group "disk3" using metadata type lvm2
Found volume group "pve" using metadata type lvm2

lvscan (some lines ommited):

ACTIVE '/dev/disk2/disk2-3' [4.00 TiB] inherit
ACTIVE '/dev/disk2/disk2-4' [1.00 TiB] inherit

LVs used for storage directory are (disk 2-3 and disk 2-4 on VG disk2)
lvdisplay (some lines ommited):

--- Logical volume ---
LV Name disk2-3
VG Name disk2
LV UUID Dbi1kZ-X0mW-7dVa-bGnl-0IFJ-U5Y1-ctfvUO
LV Write Access read/write
LV Creation host, time queen, 2021-07-25 17:48:36 -0400
LV Pool metadata disk2-3_tmeta
LV Pool data disk2-3_tdata
LV Status available
# open 1
LV Size 4.00 TiB
Allocated pool data 0.00%
Allocated metadata 10.42%
Current LE 1048576
Segments 1
Allocation inherit
Read ahead sectors auto
- currently set to 8192
Block device 253:33

--- Logical volume ---
LV Name disk2-4
VG Name disk2
LV UUID MYBPzu-O4Nv-oaBZ-v9UL-SgFx-KTCw-dxkdSJ
LV Write Access read/write
LV Creation host, time queen, 2021-09-19 10:43:42 -0400
LV Pool metadata disk2-4_tmeta
LV Pool data disk2-4_tdata
LV Status available
# open 1
LV Size 1.00 TiB
Allocated pool data 0.00%
Allocated metadata 10.42%
Current LE 262144
Segments 1
Allocation inherit
Read ahead sectors auto
- currently set to 2048
Block device 253:36

Storage config
/etc/pve/storage.cfg (some lines ommited):

dir: local-disk2-3
path /mnt/disk2/disk2-3
content backup
prune-backups keep-last=3,keep-monthly=1,keep-weekly=1,keep-yearly=1
shared 0

dir: local-disk2-4
path /mnt/disk2/disk2-4
content iso
prune-backups keep-all=1
shared 0

In storage.cfg there is no conflict with those two dir paths.

fstab (some lines ommited):

/dev/mapper/disk2-disk2--3 /mnt/disk2/disk2-3 ext4 defaults 0 2
/dev/mapper/disk2-disk2--4 /mnt/disk2/disk2-4 ext4 defaults 0 2

ls -l /dev/mapper/ | grep disk2

lrwxrwxrwx 1 root root 8 Oct 27 17:10 disk2-disk2--1 -> ../dm-10
lrwxrwxrwx 1 root root 7 Oct 27 17:10 disk2-disk2--1_tdata -> ../dm-7
lrwxrwxrwx 1 root root 7 Oct 27 17:10 disk2-disk2--1_tmeta -> ../dm-6
lrwxrwxrwx 1 root root 7 Oct 27 17:10 disk2-disk2--1-tpool -> ../dm-9
lrwxrwxrwx 1 root root 8 Oct 27 17:10 disk2-disk2--2 -> ../dm-30
lrwxrwxrwx 1 root root 8 Oct 27 17:10 disk2-disk2--2_tdata -> ../dm-12
lrwxrwxrwx 1 root root 8 Oct 27 17:10 disk2-disk2--2_tmeta -> ../dm-11
lrwxrwxrwx 1 root root 8 Oct 27 17:10 disk2-disk2--2-tpool -> ../dm-29
lrwxrwxrwx 1 root root 8 Oct 27 17:10 disk2-disk2--3 -> ../dm-33
lrwxrwxrwx 1 root root 8 Oct 27 17:10 disk2-disk2--3_tdata -> ../dm-32
lrwxrwxrwx 1 root root 8 Oct 27 17:10 disk2-disk2--3_tmeta -> ../dm-31
lrwxrwxrwx 1 root root 8 Oct 27 17:10 disk2-disk2--4 -> ../dm-36
lrwxrwxrwx 1 root root 8 Oct 27 17:10 disk2-disk2--4_tdata -> ../dm-35
lrwxrwxrwx 1 root root 8 Oct 27 17:10 disk2-disk2--4_tmeta -> ../dm-34
lrwxrwxrwx 1 root root 8 Oct 27 17:10 disk2-vm--104--disk--0 -> ../dm-37

Node -> Disks:


Node -> LVM:


Node -> LVM-Thin:

(Not sure why disk2-3 and disk2-4 show up as LVM-Thin)


-----------------------
How I created the two volumes (on existing VG disk2):

lvcreate -L 4096G -c 256K -T -n disk2-3 disk2
mkfs.ext4 /dev/disk2/disk2-3

lvcreate -L 1024G -c 256K -T -n disk2-4 disk2
mkfs.ext4 /dev/disk2/disk2-4

Added them to fstab then `mount -a` ...

Then via the web interface:
Datacenter -> Storage -> add -> select Directory -> select its type ... -> enter the directory mount path ...



Writing this post I am still scratching my head as why I get the mount error at boot...
But again, I'm sure someone more experienced will figure this out ;-)

Thanks so much for your help!