Importing ZFS pool and dataset configuration

[BeyondEvil]

I have a brand new machine running Proxmox VE 8.2.2 and I'm trying to prepare the storage before adding any VMs and LXCs.

I have 4 NVMEs:

First one is a 1TB with Proxmox itself setup on. I'm planning to use this to store ISOs, Templates etc. Basically things that are easily replaced by either downloading again or configuring again (IaC basically, I use Terraform and Ansible).

Then I have a 1TB WD Red that I'm going to use for caching and transcoding duties etc.

Finally I have two 1TBs setup as a ZFS mirrored pool. This was setup outside of Proxmox, so I now have to import it.

My plan is to run a media center (plex, jellyfin or emby) in an LXC and a Kubernetes cluster. For the Kubernetes cluster I'll start small with 3 VMs (1 master, 2 workers). Inside Kubernetes I'll run all my applications/services, including a Postgres cluster. *arrs, HomeAssistant, etc.

The content for media center is going to be stored on a couple of HDDs managed by mergerfs and snapraid (really "out of scope" for my questions).

The zfs pool currently has two datasets(?):

$ zfs list
NAME                  USED  AVAIL  REFER  MOUNTPOINT
persistent           2.16M   899G    96K  /mnt/persistent
persistent/postgres   160K   899G    96K  /mnt/postgres

So the idea is to store things I quickly want to be able to recover and want higher degree of persistence on than content. VM disks, CT Volumes and database(s).

What should my thinking be for datasets?

Should I create a separate dataset for each VM and LXC?

When importing a dataset into Proxmox I can enable "thin provisioning". When is that a good idea? Specifically regarding for what I plan to store?

There's also a "block size" setting, what should I put there?

Sorry if these questions are trivial and/or dumb, but I'm new to both Proxmox and ZFS.

 

[v95klima]

Since I use sanoid and Syncoid, my zfs datasets are defined based on how I want backup to be packaged = restored (if needed).
I don’t change block size.
Thin provisioning I believe only applies to Zvol, (block storage often used with iSCSI) which I don’t use.

 

[BeyondEvil]

Since I use sanoid and Syncoid, my zfs datasets are defined based on how I want backup to be packaged = restored (if needed).
I don’t change block size.
Thin provisioning I believe only applies to Zvol, (block storage often used with iSCSI) which I don’t use.

Sanoid or Syncoid are new to me.

So I can put "everything" into one dataset (if I want everything backed up)? Any downsides to that?

So if I don't input anything into the "block size" field, it will use whatever the pool or dataset was created with?

 

[v95klima]

For Disk 1 with Proxmox OS no need to tweak block size. Try leaving as is default during install but as ZFS disk.

Then after install, and after mounting mounting other NVME on to OS
Use
zfs import
There should be no Block size question importing.

https://docs.oracle.com/cd/E19253-0...le.com/cd/E19253-01/819-5461/gbchy/index.html

 

[BeyondEvil]

For Disk 1 with Proxmox OS no need to tweak block size. Try leaving as is default during install but as ZFS disk.

Then after install, and after mounting mounting other NVME on to OS
Use
zfs import
There should be no Block size question importing.

https://docs.oracle.com/cd/E19253-0...le.com/cd/E19253-01/819-5461/gbchy/index.html

I'm using the UI:

 

[v95klima]

Btw, I personally switched back to Proxmox OS being EXT4 for performance reasons. Not ZFS for os. But my backup of VMs I put on several disks including other mounted NVME with ZFS and a couple of data sets.

 

[v95klima]

I'm using the UI:

View attachment 71531

Aah ok, if unsure, with cli create a new data set or pool, put some files on it and then mount in gui. I don’t allow the gui to mount my Persistent core backup. Only less important pool/dataset go straight into Proxmox gui. Only Syncoid and command line use for my true persistent backup. (Sometimes a have hard shutdowns of Proxmox…)

 

[BeyondEvil]

Aah ok, if unsure, with cli create a new data set or pool, put some files on it and then mount in gui. I don’t allow the gui to mount my Persistent core backup. Only less important pool/dataset go straight into Proxmox gui. Only Syncoid and command line use for my true persistent backup. (Sometimes a have hard shutdowns of Proxmox…)

The Proxmox OS is not on ZFS, it's on the first disk (xfs or ext4, can't remember).

But if it's not in Proxmox, I can't create a Directory, meaning I can't use it for VM disks etc?

 

[v95klima]

Correct if that is what you want.
For performance and wear reasons I don’t put running VMs onto and on top of ZFS.