Storage: Any difference between 'ZFS' & 'Directory'?

[EvertM]

Hi,

I have installed Proxmox on a box with an existing zfs pool. First of all: Proxmox works like a charm and I love it!

I am curious though about how do best use my ZFS pool. I can add it under storage as 'ZFS' or as 'Directory'. Both work fine, When I create a VM on the ZFS mounted as directory I see its files (config & virtual disks) appear in the folder.
However, when I create a VM on that same ZFS but choose 'ZFS' as storage I don't see any files being created. (The VM does work though, so it must be somewhere...)

I'm an old-fashioned guy so currently I like the mount-ZFS-as-directory option better, since I see files.
Are there any advantages of using the other method though?

 

[wolfgang]

Hi,

If you use it as dir what is not recommended by the way, you can only use raw images.
qcow2 is very slow and it can come in some cases to datacorruption, because you should never use a copy on write fs on a copy on write fs.

So you have if you use it as dir no snapshots and no linked clones.

The config is there where they always are under /etc/pve/nodes/<nodename>/qemuserver/<VMID>.conf

if you like to see you images you can see them with "zfs list"

 

[EvertM]

Thanks for the quick reply

Weird, but I am seeing the opposite: When I access the zfs storage as directory, I can use all 3 kinds of images. When I access it as ZFS, raw is my only option.

 

[wolfgang]

I did not say you can't use qcow2 or vmdk. I say you should not use it.

 

[EvertM]

I must have misinterpreted 'you can only use raw images.'...

 

[mir]

Another advantage with using ZFS storage versus ZFS as dir. Given that only raw is safe on dir you loose the option of thin provision. ZFS storage uses ZFS volumes which can be thin provisioned. Another advantage with ZFS storage is that you can use ZFS send/receive on a specific volume where as ZFS in dir will require a ZFS send/receive on the entire filesystem (dataset) or in worst case the entire pool.

 

[EvertM]

Another advantage with using ZFS storage versus ZFS as dir. Given that only raw is safe on dir you loose the option of thin provision. ZFS storage uses ZFS volumes which can be thin provisioned. Another advantage with ZFS storage is that you can use ZFS send/receive on a specific volume where as ZFS in dir will require a ZFS send/receive on the entire filesystem (dataset) or in worst case the entire pool.

But how would I add an existing disk?
Example:
'zfs list' gives me:
NAME USED AVAIL REFER MOUNTPOINT
datastore 694G 4.52T 185K /datastore
datastore/proxmox1 693G 4.52T 660G /datastore/proxmox1
datastore/proxmox1/vm-109-disk-1 33.0G 4.53T 20.4G -

If I want to add a 2nd disk to vm-109, how do I do this? The 'upload' button is greyed out when I view the contents of the ZFS storage.

 

[mir]

If I want to add a 2nd disk to vm-109, how do I do this? The 'upload' button is greyed out when I view the contents of the ZFS storage.

You add and create disks from the gui (hardware tab -> add). This will create an empty disk which is instantly visible within the VM.

 

[EvertM]

I see I expressed myself a bit unclear. I want to add an existing disk image to a vm. With the ZFS as dir I created a new disk via the GUI, and replaced that disk with my existing image. But how do I do this with ZFS storage?

 

[mir]

Since the volume is seen as a block device to the system and provided the disks is of equal size dd should do it. dd if=/dev/old_disk of=/dev/new_disk bs=4096 (for 4k block size volume 8192 for 8k block size volume)

 

[EvertM]

Since the volume is seen as a block device to the system and provided the disks is of equal size dd should do it. dd if=/dev/old_disk of=/dev/new_disk bs=4096 (for 4k block size volume 8192 for 8k block size volume)

Thanks! I'll give that a try

 

[santiagoviejo]

Check this out https://forum.proxmox.com/threads/zfs-storage-vs-mounted-zfs-directory-storage.26236/ it probably can help you
Hope it helps

 

[bogo22]

I had the same question. Thanks for the clarification that Storage->ZFS is recommended and not Storage->Dir->ZFS-Pool.

(1) So, this scenario would be ok, right ?:
Storage->ZFS for Disk-Image, Container (dataset: /ssdpool/vm-data)
Storage->Dir for Container Template, ISO Image, VZDUMP file under the same zfs pool but different dataset (/ssdpool/vm-iso-backup)

(2) Scenario two: It is recommended to install proxmox to zfs-mirror during installtion? That way storage->local and local-lvm would be already on zfs and i don't need to add an extra zfs-storage.

Thanks