Proxmox 3.4 ZFS issue --> Data lost?

[GUENTER]

Hello,

I install Proxmox 3.4 with three 4TB SATA HDD's and ZFS in Raid Z1 mode. Then I place all my data folders in /rpool (/rpool/data /rpool/mp3...) folder, after several reboots (update the Kernel to 3.10 for PCI pass through support) the folders in /rpool are gone and also a scrub did not fix it.

I AM NOT ALLOWED TO POST CODE TAGS SO NO WAY TO SHOW YOU COMMANDLINE OUTPUTS ALSO WITHOUT CODE TAGS THE FORUM SW IS SHOUTING IF I WANT TO POST THE TEXT RESULT of zpool list!

The problem now is that the used size shown in "zpool list" indicates that the data still there but I could not find it anywhere!

The main question is what happens here to have this data los?
Should I do not use the /rpool folder?
Is there a chance to get the data back?
Should I reinstall the server with Proxmox 4 and restore the Data from Backup?

Many Thanks' for your help in advance!

Guenter

 

[wolfgang]

Hi,
this folders what you are talking, there where subvol or only normal dir's on rpool?

 

[tom]

new member are not allowed to post links - this is a spam protection measure. marking all red will improve your reputation.

do you use latest ZFS (0.6.5-1)?

regarding your problem: search the forum, there are quite some threads with valuable hints.

 

[GUENTER]

Hello,

I create "normal" subfolders in /rpool which are gone now.

And yes I search not only in the forum but do not find any other hint than scrub and "there is no fsck for ZFS"...

Many thanks for your help in advanve!

Guenter

 

[GUENTER]

Hello Tom,

new member are not allowed to post links - this is a spam protection measure.

A CODE tag is not an link or?

marking all red will improve your reputation.

If you already dig in to an issue and after days create an account in the support forum and prepare a detail post and then get an message like "no links allowed" even when there are NO links in, maybe you then also post a line in red ;-)

do you use latest ZFS (0.6.5-1)?

No the version before but I now update to the latest version but this also do not solve the issue after a update and reboot...

Kind Regards

Gunter

 

[sigxcpu]

zfs mount -a

and work from there. You may need to remove some empty folders so it is a good idea would be to do a pool wide snapshot before fiddling

zfs snapshot -r rpool@saved

If you mess things up when you remove the empty folders (like, being actually mounted)

zfs rollback rpool/mp3@saved

 

[GUENTER]

Hello sigxcpu,

zfs mount -a

and work from there. You may need to remove some empty folders so it is a good idea would be to do a pool wide snapshot before fiddling

zfs snapshot -r rpool@saved

If you mess things up when you remove the empty folders (like, being actually mounted)

zfs rollback rpool/mp3@saved

YOU ARE THE MAN!

All data back, but I have to umount / empty it first with this:

umount /rpool/ROOT && rm /rpool/* -Rf && zfs mount -a

Many manny thanks'

The only question is now would i survive an reboot? --> I will move now my folders do an other self created root folder...

Guenter

 

[sigxcpu]

I don't like the idea of having root and data on the same disks, so my home install scenario is with another disc dedicated to Proxmox. I have two servers, one has a 30GB SATA SSD, another one has a 32GB USB SSD. Installation on these "OS" disks is using standard LVM, not ZFS. This way, the big drives are dedicated to storage.
Even more, my main home NAS has a 512GB SSD for VMs and containers (standard ZFS storage plugin in Proxmox) and the "data" folders are simply bind mounted inside the corresponding containers.

@Proxmox: did you explore a scenario to do a distribution like SmartOS? That would be absolute killer.

 

[vkhera]

@Proxmox: did you explore a scenario to do a distribution like SmartOS? That would be absolute killer.

It would be neat to boot the nodes from a USB stick, but SmartOS also requires you to have a dedicated controller machine. I don't see how you could do a proxmox style distributed configuration with a read-only USB stick boot image.

For your particular set up, I don't understand why you wouldn't install the boot volume as ZFS. Just tell the installer to ignore the data drives when making the pool. Then make a new pool manually from the data drives once booted up.

 

[sigxcpu]

It would be neat to boot the nodes from a USB stick, but SmartOS also requires you to have a dedicated controller machine. I don't see how you could do a proxmox style distributed configuration with a read-only USB stick boot image.

For your particular set up, I don't understand why you wouldn't install the boot volume as ZFS. Just tell the installer to ignore the data drives when making the pool. Then make a new pool manually from the data drives once booted up.

1. SmartOS does not require a controller machine. SDC does.
2. The persistent data is written to the local storage pool, for both SmartOS and SDC.

If you want, I can answer questions about SmartOS, no need to ask. I've had it for a long time until Proxmox 4 beta arrived. SDC is used as our private cloud infrastructure at my main job, so I know a thing or two about them.

SmartOS has two disadvantages compared to a Linux setup:

- power consumption, for a home setup, as you don't have deep C-states (that means power, heat and power again to cool it)
- lower virtual NIC performance (~500Mbit vs ~4-5 Gbit) in KVM

I just don't want the complication that (may) arise from ZFS-booting a Linux system (there are tons of issues with this, some right here, in this forum).

So, a read-only stick for booting Linux (maybe PXE boot, too), local storage for persistent data (VMs & configuration) along with Proxmox's cluster code for a non-SPOF configuration would kick ass. Adding to that the possibilities for shared storage and such (that may complicate storing configuration on local if there is no local) would be a killer app.