Mount point not found after power outage

[1j1o]

Hi all,

My house had an electrical outage that caused my server to reboot. As I tried to starting my Nextcloud container, I've run into this error:
TASK ERROR: volume 'usb-hdd:100/vm-100-disk-0.raw' does not exist

I use an old laptop with an external SSD connected via USB.

I'm quite new with Proxmox and I'm not sure how to troubleshoot this. Is anyone familiar with this issue and may help?

Thank you!

 

[bbgeek17]

Is the USB disk connected?
What does "cat /etc/pve/storage.cfg" show?
What about "pvesm status" ?

---

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox

 

[1j1o]

Is the USB disk connected?
What does "cat /etc/pve/storage.cfg" show?
What about "pvesm status" ?

---

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox

Thanks for replying!

> Yep, the USB disk is connected.
> Here is what storage.cfg shows:
dir: local path /var/lib/vz content vztmpl,iso,backup lvmthin: local-lvm thinpool data vgname pve content rootdir,images dir: usb-hdd path /mnt/usb-hdd content rootdir,images prune-backups keep-all=1 shared 0

(Note: I'm not sure if this would do anything, but I did try changing "shared 0" to "shared 1". Didn't seem to have an effect)
> "pvesm status" shows:
Name Type Status Total Used Available % local dir active 60837724 3720188 53994748 6.11% local-lvm lvmthin active 160022528 15682207 144340320 9.80% usb-hdd dir active 60837724 3720188 53994748 6.11%

 

[flames]

as you see from "pvesm status" your usb disk is not mounted to /mnt/usb-hdd anymore (it has same size as your local). most probably it is not /dev/sda anymore, but /dev/sdx (x some other letter). i assume you mounted your disk with /dev/sdx1 in /etc/fstab, you should mount it by uuid or label in fstab, so it is found even if it changes its device path by connection order.
please rapport output of following:
lsblk
lsusb
ls -lah /dev/disk/by-label/
ls -lah /dev/disk/by-uuid/
ls -lah /dev/disk/by-id/
ls -lah /dev/disk/by-path/
ls -l /dev/disk/by-partlabel/ <- this,..
ls -l /dev/disk/by-partuuid/ <- or this you use in fstab
cat /etc/fstab
you will see the difference then

and keep it "shared 0". that does not concern you, until cluster with multiple nodes and more understanding in proxmox storage with NAS (SMB and NFS), SAN and vSAN (as CEPH).

 

[apoc]

ls -lah /dev/disk/by-id/
ls -lah /dev/disk/by-path/

Always use static references from these paths.
/dev/sdx is only providing trouble.
Also be aware that some devices like USB bridges don't provide unique device identifiers.
So it makes sense to even double check those!

 

[Dunuin]

And you didn't enabled the "is_mountpoint" option for your directory storage. You really should do that, so a failed mount will prevent PVE from filling up your entire root filesystem, until the server crashes.

You can do that by running once: pvesm set usb-hdd --is_mountpoint yes

 

[flames]

please remove the quote or take the right lines. could confuse the new user

 

[1j1o]

as you see from "pvesm status" your usb disk is not mounted to /mnt/usb-hdd anymore (it has same size as your local). most probably it is not /dev/sda anymore, but /dev/sdx (x some other letter). i assume you mounted your disk with /dev/sdx1 in /etc/fstab, you should mount it by uuid or label in fstab, so it is found even if it changes its device path by connection order.
please rapport output of following:
lsblk
lsusb
ls -lah /dev/disk/by-label/
ls -lah /dev/disk/by-uuid/
ls -lah /dev/disk/by-id/
ls -lah /dev/disk/by-path/
ls -l /dev/disk/by-partlabel/ <- this,..
ls -l /dev/disk/by-partuuid/ <- or this you use in fstab
cat /etc/fstab
you will see the difference then

and keep it "shared 0". that does not concern you, until cluster with multiple nodes and more understanding in proxmox storage with NAS (SMB and NFS), SAN and vSAN (as CEPH).

You're incredible, thank you. I can't believe it was this simple of a fix.
I just remounted the disk:
mount /dev/sda1 /mnt/usb-hdd and everything's back to work.

Thanks for explaining it in detail as well.

 

[flames]

You're incredible, thank you. I can't believe it was this simple of a fix.
I just remounted the disk:
mount /dev/sda1 /mnt/usb-hdd and everything's back to work.

Thanks for explaining it in detail as well.

please, add the mount to your /etc/fstab and NOT as /dev/sda1, but as corresponding label or uuid, which you can find via:

ls -l /dev/disk/by-partlabel/ <- label/name of partition, or
ls -l /dev/disk/by-partuuid/ <- uuid of partition

do NOT mount it via /dev/sda1, becuase this is not permanent path!

 

[1j1o]

please, add the mount to your /etc/fstab and NOT as /dev/sda1, but as corresponding label or uuid, which you can find via:

ls -l /dev/disk/by-partlabel/ <- label/name of partition, or
ls -l /dev/disk/by-partuuid/ <- uuid of partition

do NOT mount it via /dev/sda1, becuase this is not permanent path!

Ah sorry, I got ahead of myself.
Got it, thank you.

 

[flames]

good, everyone has a start. happy it works