Pass ZFS zvol/dataset to a guest VM

[fahadshery]

Hi,

I have a local zfs pool that I created in the proxmox.
Then I created a dataset for my Postgres database as:

zfs create tank/postgres -o ashift=12 -o compression=lz4

I can see that it is mounted in my proxmox at:

tank/postgres

I now wish to pass this dataset to my ubuntu server VM.
How do I do it?

The reason for this is that I want to have persistent storage for my Postgres database. I could then easily take snapshots of this dataset and send it to a remote server for backup.

Thanks and have a brilliant festive season!

 

[Dunuin]

You can't pass that directly to a VM. Bind-Mounting only works with LXCs. With VMs you can only bring your dataset into the VM by using network shares. So you need to turn your PVE somehow into a NAS by installing SMB/NFS server to the host or by running a LXC that acts as a NAS where you could bind-mount your dataset into, to be shared. But running a DB of a network share wouldn't be fast because of all the overhead of the network stack.

Why not just use a virtual disk which would create a zvol? You can also snapshot and incrementally replicate zvols.

 

[fahadshery]

You can't pass that directly to a VM. Bind-Mounting only works with LXCs. With VMs you can only bring your dataset into the VM by using network shares. So you need to turn your PVE somehow into a NAS by installing SMB/NFS server to the host or by running a LXC that acts as a NAS where you could bind-mount your dataset into, to be shared. But running a DB from a network share wouldn't be fast because of all the overhead of the network stack.

In that case, I would rather keep the zfs pool in truenas VM and then use iSCSI to mount it within the VM.

thank you for your reply you've been a great help. wish there was a direct message facility where I could DM you

 

[Dunuin]

In that case, I would rather keep the zfs pool in truenas VM and then use iSCSI to mount it within the VM.

Jep, also the best way I can think of.

 

[LnxBil]

In that case, I would rather keep the zfs pool in truenas VM and then use iSCSI to mount it within the VM.

That cannot be fast.

Best way would be to use PostgreSQL directly on the ZFS dataset itself and configure it according to best practice. It'll be faster than running it on ext4 or any other filesystem in between due to the fact that you can disable full page writes. Every write on ZFS is already atomic. Compression is also a big plus.

 

[fahadshery]

That cannot be fast.

Best way would be to use PostgreSQL directly on the ZFS dataset itself and configure it according to best practice. It'll be faster than running it on ext4 or any other filesystem in between due to the fact that you can disable full page writes. Every write on ZFS is already atomic. Compression is also a big plus.

The thing is that I have a Postgres dataset within TrueNas. My Postgres instance is managed by Kubernetes. I am mounting the TrueNas dataset using NFS within the VM.
I am not sure how I could optimise it further? The only other option is perhaps to create a Postgres jail within TrueNas and mount the dataset in that jail. This makes it very difficult to manage/test/upgrade I guess???

 

[domsl]

Hi,

I am very new to this sort of thing. So I guess I add to your question.

First, thanks to the community for all the answers, especially these newbie questions.

Right now I have a similar problem. Big data folder that gets shared via docker->syncthing inside a VM. I am using a VM disk (zvol) on the zfs of the host and pass is through to the VM, where it is mounted with an ext4 filesystem.
I don't know how bad that is for space and performance.
Additionally, as far as I understand, I can't just create instantaneous zfs snapshots on the zvol, because that would "look" like I snapshotted the VM disk as if I pulled the power, right? Like the docker container does not know that it will have to flush to disk before the snapshot happens?

Because of that I am considering moving the docker containers (probably the other ones too), directly to the proxmox host and to create a native zfs dataset for docker/syncthing to mount.
Then I would have "native" performance, but loose the possibility to easily move the whole VM with the VM disk to a different server. But I could make easy hourly snapshots of my files, because hopefully the docker container knows how to handle an impending snapshot.
Would that be a good idea?

@fahadshery:
In your case, that would mean, run your Postgres instance with Kubernetes directly on host and point your TrueNAS there.
TrueNas sounds like you would have user data as well, where are you storing that right now? Also on a block device?

 

[fahadshery]

@fahadshery:
In your case, that would mean, run your Postgres instance with Kubernetes directly on host and point your TrueNAS there.
TrueNas sounds like you would have user data as well, where are you storing that right now? Also on a block device?

My research suggests that running any database over NFS is a bad idea. Postgres runs great on FreeBSD - so I am gone use a jail instead within TrueNas.
Postgres compiles cleanly and neatly on FreeBSD, and by using a jail on a TrueNas host, you combine the NAS's large amounts of memory along with fastest possible disk I/O, and you will get a MUCH more performant SQL/Postgres server running it on the TrueNas host than you would get on a NFS-mounted SQL/Postgres server.
It would be prudent to follow common ZFS rules for databases, including adding extra memory for the SQL/Postgres server itself (if you don't have a ton), preferring mirror vdevs, and keeping pool occupancy lower than maybe 60%. Lots of common performance issues shared with block storage.
Hope this helps!

 

[LnxBil]

My research suggests that running any database over NFS is a bad idea. Postgres runs great on FreeBSD - so I am gone use a jail instead within TrueNas.
Postgres compiles cleanly and neatly on FreeBSD, and by using a jail on a TrueNas host, you combine the NAS's large amounts of memory along with fastest possible disk I/O, and you will get a MUCH more performant SQL/Postgres server running it on the TrueNas host than you would get on a NFS-mounted SQL/Postgres server.

Yes, as I tried to say ... PostgreSQL is very good directly on ZFS. Also use the optimizations from the postgresql wiki for running directly on ZFS.