Lost VM Disk - Accidently deleted storage

JayK · Feb 27, 2024

Well I messed up. I deleted a storage device from Datacenter>Storage. This was intentional as I want to repurpose the drive. However... I accidently installed one VM on it instead of installing on "local" like EVERYTHING ELSE. I didn't realize it until I couldn't start the VM. Fortunately, I had not destroyed the volume in Proxmox>Disks>Directory, so I re-added the drive, and everything seemed to be fine. VMs and LXCs have access to the drive again. But, the one VM still wont start. I get

"Error: volume 'Backup:104/vm-104-disk-0.qcow2' does not exist"

This is my TrueNAS VM. I REALLY don' want to have to rebuild my NAS. I can, I have backups, but, it's definitely not something I want to do if I don't have to.

Thanks for any help.

bbgeek17 · Feb 27, 2024

did you add the storage back under the same name, in the same exact way?
What is "Backup" storage? (cat /etc/pve/storage.cfg). What is current storage state? (pvesm status).

If is a file based storage, as you suggest, is the file there?

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox

Kingneutron · Feb 27, 2024

Open a shell / root prompt. If not already installed, ' apt update; apt install mlocate '
After that finishes, ' time updatedb '

After ^^ finishes, ' locate *.qcow2 '

Note for future readers - mlocate is a dummy pkg that points to plocate, so use that if it gets removed

bbgeek17 · Feb 27, 2024

Kingneutron said:
Open a shell / root prompt. If not already installed, ' apt update; apt install mlocate '
After that finishes, ' time updatedb '

After ^^ finishes, ' locate *.qcow2 '

I did not read it as if he deleted the data, but just removed the storage pool configuration and then added it back. May be I am wrong.

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox

JayK · Feb 27, 2024

bbgeek17 said:
I did not read it as if he deleted the data, but just removed the storage pool configuration and then added it back. May be I am wrong.

Correct. I removed the storage from Datacenter and re-added it. All my data is still there. I thought I added it the exact same way I did the first time, but maybe not.
Backup is just what I called the drive.

Code:

dir: local
        path /var/lib/vz
        content backup,rootdir,vztmpl,images,iso
        shared 0

dir: Backup
        path /mnt
        content images,vztmpl,rootdir,backup,iso,snippets
        prune-backups keep-all=1
        shared 0

So after I looked at storage.cfg the first time, content only included images. I added the rest after as I'm pretty sure they were there originally. I rebooted Proxmox just to be sure. Still get the same error starting TrueNAS VM.

bbgeek17 · Feb 27, 2024

For starters don't use /mnt as mount path. It has special meaning in PVE.
Either use subdirectory or a different top level path.
Add mountpoint flag.

Report the output of "mount", "df", and whether the needed file is there. Ie "find . -name *.qcow2"

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox

JayK · Feb 27, 2024

bbgeek17 said:
For starters don't use /mnt as mount path. It has special meaning in PVE.

Ahh. I didn't know that. I can't edit the mount path, but I can remove the storage an re-add it somewhere else. My concern would be that the VM wouldn't be able to find it, but I guess that's already the case.

The .qcow file is there in the images/104 folder. As far as I can tell this is where it was before. It does not show up in the UI. This being the VM image, can I move it to where it would be on "local" storage (where it should have been in the first place) and point the VM to that? Ultimately this is what I want anyway.

mount:

Code:

sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,relatime)
udev on /dev type devtmpfs (rw,nosuid,relatime,size=24673728k,nr_inodes=6168432,mode=755,inode64)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,noexec,relatime,size=4941484k,mode=755,inode64)
/dev/mapper/pve-root on / type ext4 (rw,relatime,errors=remount-ro)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev,inode64)
tmpfs on /run/lock type tmpfs (rw,nosuid,nodev,noexec,relatime,size=5120k,inode64)
cgroup2 on /sys/fs/cgroup type cgroup2 (rw,nosuid,nodev,noexec,relatime)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
bpf on /sys/fs/bpf type bpf (rw,nosuid,nodev,noexec,relatime,mode=700)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20120)
tracefs on /sys/kernel/tracing type tracefs (rw,nosuid,nodev,noexec,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,nosuid,nodev,noexec,relatime)
mqueue on /dev/mqueue type mqueue (rw,nosuid,nodev,noexec,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime,pagesize=2M)
sunrpc on /run/rpc_pipefs type rpc_pipefs (rw,relatime)
fusectl on /sys/fs/fuse/connections type fusectl (rw,nosuid,nodev,noexec,relatime)
configfs on /sys/kernel/config type configfs (rw,nosuid,nodev,noexec,relatime)
/dev/sdf1 on /mnt/pve/Backup type ext4 (rw,relatime)
lxcfs on /var/lib/lxcfs type fuse.lxcfs (rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other)
/dev/fuse on /etc/pve type fuse (rw,nosuid,nodev,relatime,user_id=0,group_id=0,default_permissions,allow_other)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,nosuid,nodev,noexec,relatime)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=4941480k,nr_inodes=1235370,mode=700,inode64)

df:

Code:

Filesystem            1K-blocks       Used  Available Use% Mounted on
udev                   24673728          0   24673728   0% /dev
tmpfs                   4941484       1460    4940024   1% /run
/dev/mapper/pve-root  470831648  243313956  207337240  54% /
tmpfs                  24707416      43680   24663736   1% /dev/shm
tmpfs                      5120          0       5120   0% /run/lock
/dev/sdf1            9688302352 6044267872 3155696308  66% /mnt/pve/Backup
/dev/fuse                131072         20     131052   1% /etc/pve
tmpfs                   4941480          0    4941480   0% /run/user/0

JensF · Feb 27, 2024

JayK said:
/dev/sdf1 9688302352 6044267872 3155696308 66% /mnt/pve/Backup

It seems to me you need this path in your storage.cfg:

Code:

dir: Backup
        path /mnt/pve/Backup
        content images,vztmpl,rootdir,backup,iso,snippets
        prune-backups keep-all=1
        shared 0

bbgeek17 · Feb 27, 2024

JensF said:
It seems to me you need this path in your storage.cfg:

possibly. Depending on how Op mounts the device on boot.
If there is a manual entry in fstab, then best approach is to remove it, then redo "directory" storage. PVE will automatically create the right thing:
https://forum.proxmox.com/threads/persistent-directory-mount.125981/#post-549881

If one insists on manually mounting to /mnt/pve/xxxx then, yes the path needs to be specified correctly, and the "is-mountpoint" flag should be set.

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox

JayK · Feb 27, 2024

JensF said:
It seems to me you need this path in your storage.cfg:

Code:

dir: Backup path /mnt/pve/Backup content images,vztmpl,rootdir,backup,iso,snippets prune-backups keep-all=1 shared 0

That did it! Thank you!

The only thing I added to fstab was for the smb share from TrueNAS. Though I realize that will not likely work since Proxmox looks at fstab, then loads VMs... I think. Don't really know yet since I wasn't able to try it.

Since this is running now, how do I move the image?

bbgeek17 · Feb 27, 2024

JayK said:
Since this is running now, how do I move the image?

use GUI, VM>hardware>select disk>top disk action >move storage

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox

JayK · Feb 27, 2024

bbgeek17 said:
use GUI, VM>hardware>select disk>top disk action >move storage

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox

Thanks. That was a dumb question. I didn't look before I asked.

homelabenthusiast · Feb 27, 2024

@JayK do you see vm-104-disk-0.qcow2 under Backup datastore after you readded it?

Kingneutron · Feb 27, 2024

JayK said:
That did it! Thank you!

The only thing I added to fstab was for the smb share from TrueNAS. Though I realize that will not likely work since Proxmox looks at fstab, then loads VMs... I think. Don't really know yet since I wasn't able to try it.

Since this is running now, how do I move the image?

Glad it worked out for you

now DON'T FORGET TO ENABLE REGULAR BACKUPS

Search

Search

Lost VM Disk - Accidently deleted storage

JayK

Member

bbgeek17

Distinguished Member

Kingneutron

Active Member

bbgeek17

Distinguished Member

JayK

Member

bbgeek17

Distinguished Member

JayK

Member

JensF

Well-Known Member

bbgeek17

Distinguished Member

JayK

Member

bbgeek17

Distinguished Member

JayK

Member

homelabenthusiast

Member

Kingneutron

Active Member