Storage Management - Help required

[Luca1337]

Hi Everyone,

I am new to this Forum and hope to receive some help with my Storage Management.
I am aware of the basics and different types of storage and so on, but i don't know how to manage all of this in a "real life usecase".

My Setup:
1x Nvme - 250Gib -> Proxmox OS installed (Ref. as Disk A)
1x Nvme - 500Gib -> Setup as ZFS for container disks / "Proxmox Data" (Ref. as Disk B)
1x internal 5Tib HDD (sata) -> To be used as "shared" storage? (Ref. as Disk C)
1x external 5Tib HDD (usb) -> To be used as "backup" storage? (Ref. as Disk D)

My desired setup:
I'd like to run a PLEX, Nextcloud and Bitwarden Server.
Getting the server / LXCs up and running is no problem. Where i stuggle is the Storage Management.
I am coming from Docker where the Storage setup was straight forward. One disk where the files are stored, and the container mounts a certain path. Even if a docker container is corrupt the files are still on the local storage and remain untouched.

What's the Problem?
I don't get how I should setup the storage to be fast but also reliable.
I've seen solutions where people setup a TrueNas Server, "pre-manged" the storage and then passed it through to PVE. Is this really the "common" solution?

What's the best case scenario? I have tought of a few options, maybe you can verify if any of these is valid?

Eg. Plex Server
Option 1
1. Disk B for Container Data (OS)
2. Disk C setup as ZFS (Shared Storage)
3. Disk D setup as ZFS (for Backups)
3. Mount Point to Disk C ZFS with a fixed size
4. Backup Container to Disk D including the Mount Points

Option 2
1. Disk B for Container Data (OS)
2. Disk C setup as Directory (Shared Storage)
3. Disk D setup as ZFS (for Backups)
3. Mount Point to Disk C with a fixed size
4. Backup Container to Disk D including the Mount Points

Can someone help me out how I should setup the storage? I wanted it to be more like Docker, but i think I'm taking the wrong route here

 

[Luca1337]

BTW. I would not mind to buy any additional SSD / HDD / NVME if that would open new options

 

[LnxBil]

also reliable.

I don't see anything reliable here. I recommend having at least mirroring / RAID1 in order to circumvent the obvious and most probable error you will encounter: a disk failure.

I am coming from Docker where the Storage setup was straight forward.

It is everything but straight forward. Docker has also the same problems if you do it right. I recommend (for Docker as well as anything PVE related (mostly LXC) on single-hosts) to use ZFS. You will have snapshots, compression and quota and can do what you want without thinking about "which disk". ZFS is built to have EVERYTHING in one pool, but that requires a more homogeneous storage backend and not 4 different drives with two different technologies and three different bus types. With snapshots, you will also do not need local backups so that you can use your external USB drive as a separate ZFS pool to replicate your pool or use as separate disk for backups.

 

[Luca1337]

I don't see anything reliable here. I recommend having at least mirroring / RAID1 in order to circumvent the obvious and most probable error you will encounter: a disk failure.


It is everything but straight forward. Docker has also the same problems if you do it right. I recommend (for Docker as well as anything PVE related (mostly LXC) on single-hosts) to use ZFS. You will have snapshots, compression and quota and can do what you want without thinking about "which disk". ZFS is built to have EVERYTHING in one pool, but that requires a more homogeneous storage backend and not 4 different drives with two different technologies and three different bus types. With snapshots, you will also do not need local backups so that you can use your external USB drive as a separate ZFS pool to replicate your pool or use as separate disk for backups.

Copy that.
So your suggestion is, to buy new drives (SSDs same size same model). Set them up as ZFS Pool and then? Do I share the pool as NFS to the container? Or do I mount the ZFS as mountpoint directly with a fixed size?

 

[Dunuin]

So your suggestion is, to buy new drives (SSDs same size same model). Set them up as ZFS Pool and then?

And you want proper SSDs with ZFS. So Datacenter/Enterprise SSD with enough DWPD and power-loss protection.

 

[LnxBil]

Do I share the pool as NFS to the container? Or do I mount the ZFS as mountpoint directly with a fixed size?

Yes, internally used as a mountpoint. Each ZFS dataset can have quota so that is what you mean by "fixed size".