[SOLVED] How to restore VM disks after Local ZFS delete

[Vaillant]

Hello,

I made a silly mistake and need some guidance. I unfortunately delete my local-zfs with all my VMs disks. Because of backup I was able to restore most VMs but two of them are not working for me one of them is because I did not have any backup. The other is because I do not have enough memory to do the restore.

Here is the data I get when listing the ZFS

root@Homelab1:~# zfs list
NAME USED AVAIL REFER MOUNTPOINT
rpool 740G 160G 96K /rpool
rpool/ROOT 139G 160G 96K /rpool/ROOT
rpool/ROOT/pve-1 139G 160G 139G /
rpool/data 304G 160G 96K /rpool/data
rpool/data/vm-100-disk-0 29.1G 160G 29.1G -
rpool/data/vm-101-disk-0 6.59G 160G 6.59G -
rpool/data/vm-101-disk-1 35.0G 160G 35.0G -
rpool/data/vm-102-disk-0 8.55G 160G 8.55G -
rpool/data/vm-200-disk-0 74.8G 160G 47.0G -
rpool/data/vm-200-state-BeforeUPS 2.94G 160G 2.94G -
rpool/data/vm-200-state-Updated 6.13G 160G 6.13G -
rpool/data/vm-201-disk-0 11.8G 160G 11.8G -
rpool/data/vm-202-disk-0 41.2G 160G 41.2G -
rpool/data/vm-301-disk-0 17.7G 160G 17.7G -
rpool/data/vm-302-disk-0 20.9G 160G 20.9G -
rpool/data/vm-401-disk-0 49.7G 160G 49.7G -
rpool/vm-101-disk-0 16.3G 169G 6.59G -
rpool/vm-101-disk-1 81.3G 205G 35.4G -
rpool/vm-102-disk-0 36.6G 188G 8.50G -
rpool/vm-301-disk-0 65.0G 207G 17.7G -
rpool/vm-302-disk-0 32.5G 171G 21.5G -
rpool/vm-401-disk-0 65.0G 174G 50.6G -

I need to restore vm-100 : I can see the disk in data but have no clue how to bring it to rpool (I'm a beginner)
I also need to restore vm-200 but I have a backup which is not working. Thus I suppose the best option is to follow the same process as vm-100

I would like to know how to delete vm-201 and vm-202 as they got deleted from the interface but because of my mistake they are still in the pool.

 

[aj@root]

ZFS Basics

I'd recommend checking out a primer on ZFS. This is also a case where, although I wouldn't recommend taking what it says as gospel, you could chat with an LLM about it to get the gist of it and get to the point where you have the vocabulary to know what questions to ask.

Just knowing about the various `zfs` commands to list, create, destroy, send, receive, etc and how they work (real copy vs linked snapshot) is very helpful.

That said... I don't think we need any of that to fix your problem.

Restoring your local Storage

I think what you may have done here is just delete the reference to the local storage in the Web UI, like this:


Yes?

So all you need to do, is add it back!



Managing Storage

You can use zfs destroy to delete those stray volumes but you can also use the storage manager on each node:

You won't be able to delete them until their owning VM has been deleted.

If you added back a `local` already, you may need to do a little dance of creating a `local2`, moving things to that, removing `local` (once its empty), removing `local2` (just the reference in the Datacenter), and then adding it back as `local`.

Migrating Disks

You can use the zfs send and zfs receive to copy volumes between datasets, but you can also use the Web UI


Since you're out of disk space, you may need to delete unreferenced disks from storage before moving any of the others. If you remove them with the Web UI, it has checks to prevent oopsies. If you use zfs destroy, it just do what you tell it to do.

P.S. Also, to avoid confusion "memory" is RAM and "storage" is disk. I know in many circles "memory" gets thrown around to mean lots of things, but in the IT world they're distinct.

 

[aj@root]

Depending on how mixed up things are, it may also be helpful to edit some files in /etc/pve/qemu-server/*.conf to change a few lines by hand:

For example, you might change

scsi0: local:101/vm-101-disk-0.qcow2,cache=writethrough,discard=on,size=32G,ssd=1

to

scsi0: local2:101/vm-101-disk-0.qcow2,cache=writethrough,discard=on,size=32G,ssd=1

and then later back again, depending on how you're trying to shuffle things.

I recommend copying the file, editing it, then copying it back:

mkdir -p ~/Backups
cp -RPp /etc/pve/qemu-server/101.conf ~/Backups/101.conf
cp -RPp /etc/pve/qemu-server/101.conf /tmp/101.conf
nano /tmp/101.conf
mv /tmp/101.conf /etc/pve/qemu-server/

Or you may not need to do that at all. You might be able to do everything in the Web UI just fine... but I imagine they may be some errors about "cannot find disk" or some such, in which case ^^ you do that to point it the the right Datacenter Storage ID.

You should be able to get everything back in order without too much trouble.

 

[Vaillant]

Depending on how mixed up things are, it may also be helpful to edit some files in /etc/pve/qemu-server/*.conf to change a few lines by hand:

For example, you might change

scsi0: local:101/vm-101-disk-0.qcow2,cache=writethrough,discard=on,size=32G,ssd=1

to

scsi0: local2:101/vm-101-disk-0.qcow2,cache=writethrough,discard=on,size=32G,ssd=1

and then later back again, depending on how you're trying to shuffle things.

I recommend copying the file, editing it, then copying it back:

mkdir -p ~/Backups
cp -RPp /etc/pve/qemu-server/101.conf ~/Backups/101.conf
cp -RPp /etc/pve/qemu-server/101.conf /tmp/101.conf
nano /tmp/101.conf
mv /tmp/101.conf /etc/pve/qemu-server/

Or you may not need to do that at all. You might be able to do everything in the Web UI just fine... but I imagine they may be some errors about "cannot find disk" or some such, in which case ^^ you do that to point it the the right Datacenter Storage ID.

You should be able to get everything back in order without too much trouble.

Thank you so much for your help, I was able to return to where I wanted to be.

 

[aj@root]

@Vaillant You're welcome. I'm glad that helped get things back on track.
Would you mind marking this topic as Solved?