Is it recoverable?

[alscx]

I'm really desperate and I could use any help I get.

I'd been using Proxmox successfully for a couple of months and I decided to reinstall it this weekend. My Proxmox was installed on an NVME and I have also 3 6TB disks in RAIDZ1 that I shared via passthrough to a VM (OMV) and 3 more spare disks with varied sizes (LUKS encrypted). What happened is that I made a mistake when I was reinstalling Proxmox and instead of choosing the ext4 option I used before I chose zfs (RAID0). What happened is that Proxmox created a huge zfs pool with all my disks instead of using only the NVME.

I'm desperate. This data comes from more than 15 years of my life. In fact I bought these disks recently in order to copy my stuff from my old disks and have 2 copies (the original in the old disks and the data from the new raid) and I hadn't even finished copying the data yet, since I was slowly sorting what I would keep, etc. Now all that's left seems to be a single huge pool that's using all of my disks.

Could someone with more experience give me some tip? I've turn the PC off and now I'm trying to find whether it's recoverable.

I really appreciate any suggestion.

 

[leesteken]

I did not expect the Proxmox installed to default to a ZFS pool over all your drives, but apparently it did. This will be very difficult and it's a good thing you shut it down (to prevent overwriting more data).
Do you have backups of your valuable data (from a little while back)? Personally, I'm not sure how to fix this remotely, especially via the text interface of this forum.
Do you have a copy of the previous partitioning information by any chance? Do you have additional drives with enough space that you can copy the current situation (to experiment without corruption any more data)?
Maybe go to a specialized data recovery company with your drives and explain the situation to them (and be prepared to pay for their time and efforts)?

 

[alexskysilk]

I'm desperate. This data comes from more than 15 years of my life

zfs isnt friendly to most end user accessible data recovery tools. the good news-

Maybe go to a specialized data recovery company with your drives and explain the situation to them (and be prepared to pay for their time and efforts)?

This is very likely to be successful.

 

[alscx]

I did not expect the Proxmox installed to default to a ZFS pool over all your drives, but apparently it did. This will be very difficult and it's a good thing you shut it down (to prevent overwriting more data).
Do you have backups of your valuable data (from a little while back)? Personally, I'm not sure how to fix this remotely, especially via the text interface of this forum.
Do you have a copy of the previous partitioning information by any chance? Do you have additional drives with enough space that you can copy the current situation (to experiment without corruption any more data)?
Maybe go to a specialized data recovery company with your drives and explain the situation to them (and be prepared to pay for their time and efforts)?

I appreciate you take the time to help.

Well, the only copies I had were on those 3 spare disks (one 8TB, one 4 TB and the other 2 TB). Ironically I was worried about having any trouble with those drives and loose my things that's why I got these 3 new 6TB disks and created this RAID and was in the process of copying them...

The following was the line I used to create the pool (I saved it because as I'm not very experienced with ZFS I imagined I could reuse in the case I had to rebuild the raid):

sudo zpool create -o failmode=continue -o autoexpand=on -o autotrim=on -o feature@async_destroy=enabled -o feature@empty_bpobj=enabled -o feature@lz4_compress=enabled -o feature@multi_vdev_crash_dump=enabled -o feature@spacemap_histogram=enabled -o feature@enabled_txg=enabled -o feature@hole_birth=enabled -o feature@extensible_dataset=enabled -o feature@embedded_data=enabled -o feature@bookmarks=enabled -o feature@filesystem_limits=enabled -o feature@large_blocks=enabled -o feature@large_dnode=enabled -o feature@sha512=enabled -o feature@skein=enabled -o feature@edonr=enabled -o feature@userobj_accounting=enabled -o feature@project_quota=enabled -o feature@device_removal=enabled -o feature@obsolete_counts=enabled -o feature@zpool_checkpoint=enabled -o feature@spacemap_v2=enabled -o feature@allocation_classes=enabled -o feature@resilver_defer=enabled -o feature@bookmark_v2=enabled -o feature@redaction_bookmarks=enabled -o feature@redacted_datasets=enabled -o feature@bookmark_written=enabled -o feature@log_spacemap=enabled -o feature@livelist=enabled -o feature@device_rebuild=enabled -o feature@zstd_compress=enabled -o feature@draid=enabled -O encryption=on -O keyformat=raw -O keylocation=/etc/zfs/zpool.key zpool raidz /dev/disk/by-id/ata-QEMU_DVD-ROM_QM00003 /dev/disk/by-id/ata-QEMU_HARDDISK_QM00005 /dev/disk/by-id/ata-QEMU_HARDDISK_QM00007

Unfortunately the only disks I have are these. I have a couple of external USB disks, but they are all small ones.

They are already showing as part of the new pool (rpool) Proxmox created, so I believe there's nothing left of my old pool.

 

[leesteken]

Well, the only copies I had were on those 3 spare disks (one 8TB, one 4 TB and the other 2 TB). Ironically I was worried about having any trouble with those drives and loose my things that's why I got these 3 new 6TB disks and created this RAID and was in the process of copying them...

This is not what I meant. You'll need additional drives to experiment or lose your data. I'm now assuming you did not have any backups for data that you feel is valuable.

Unfortunately the only disks I have are these. I have a couple of external USB disks, but they are all small ones.

Then I recommend against trying to fix this yourself or trying to get it fixed by other people trying to fix it remotely.

They are already showing as part of the new pool (rpool) Proxmox created, so I believe there's nothing left of my old pool.

That might be the case but ZFS uses COW, so the data might still be recoverable by someone (more) knowledgeable (than me) who specializes in data recovery.

This is very likely to be successful.

Other people seem to agree that the best approach is to contact specialists and pay them to (try to) recover your valuable data.