ZFS storage vs mounted ZFS Directory Storage

[pcmofo]

I have a ZFS pool that I want to store all of my proxmox files on.

zpool/proxmox/vm
zpool/proxmox/iso
zpool/proxmox/vmbackup
zpool/proxmox/templates

etc

I mounted the zpool and can access all of those files on the host proxmox system at those exact locations. If I add storage to Proxmox as ZFS I can add either the root zpool/proxmox or individual datasets as storage. If I add ZFS type storage I can only choose from Disk Image and Container. If I add Storage as "Directory" then choose the ZFS dataset above I can choose any of the data types including Backup and Templates.

I was able to then download templates to the templates dataset using the GUI. I cant seem to create a new LXC in the /vm folder or move an existing one, the error says "could not open.

What is the best way to handle mounting ZFS datasets as storage in Proxmox so that I can store templates, isos, containers, and backups on my pool?

On a related note, how can I move an existing LXC? Physically copy the directory and edit the config file? Or can I use the GUI like I can for the VMs?

 

[LnxBil]

If you want to store files, you need a filesystem. If you want to store a thin provisioned volume for KVM, then use zvol. There are only these two

You can of course create multiple filesystems and mount it to get the files you want. Technically you can put e.g. qcow2 on zfs filesystem, but this is an additional layer.

 

[VGusev2007]

Don't use directory storage if you want to have a good performance

 

[pcmofo]

So I am currently using FreeNAS with ZFS to store all of my media, backups, etc etc. The idea is to move to Proxmox, let Proxmox handle the ZFS pool that holds that data along with running various LXCs and VMs that provide various services using that shared ZFS pool data.

So where to store the LXC and VMs then? On the ZFS pool makes the most sense to me as it will be backed up, it is redundant, and very fast (800MB/s+) Proxmox will be run from a USB drive. The whole point of this setup is Proxmox can crash and burn and the VMs are safe on the ZFS pool due to redundancy and backups I can be up and running quickly. Proxmox will handle ZFS scrubs and other useful pool management.

So what I don't understand is why I cannot create or move a VM into a ZFS dataset I just created after adding a "Directory" in Proxmox that points to that /zpool/dataset/ when things like the Templates downloaded and worked just fine? As I said earlier, when I add a "ZFS" type storage to Proxmox I can't choose Templates or Backups as a type of data to be stored.

How else should I be going about this? Where "Should" my data be stored?

 

[mir]

Why not simply export a storage to Proxmox through NFS for LXC?
This could also be used for KVM images (Qcow2 or raw).

 

[LnxBil]

Again:
KVM on zvol - the rest in filesystem. I can use container and backup on a directory I added from my ZFS pool.

If you use everything in the filesystem - it also works perfectly - but is not as fast as it could be with a zvol.

 

[pcmofo]

Again:
KVM on zvol - the rest in filesystem. I can use container and backup on a directory I added from my ZFS pool.

If you use everything in the filesystem - it also works perfectly - but is not as fast as it could be with a zvol.

Thanks! I think I understand you now.

I now have the ZFS pool mounted to proxmox and added as "ZFS" storage to hold VMs (disk image, container) and then I have another dataset on the pool mounted as a "Directory" to handle ISO, template, and backup. This appears to be working fine and I can create new VMs as well as run backups and download templates now.

In the near future I will need to move all of these files into "production" on an existing zpool. It seems pretty clear on how to move the VM containers, but how do I move the LXC containers? manually cp to the correct directory and edit the config file? I tried Backup/Restore but it really doesn't like me trying to backup the LXC after adding a mp0: mountpoint to various zfs datasets. I'm not sure how to exclude the mountpoints from the backup.