Way to setup following few services (from storage POV)

[hlab]

[P.S. I see similar questions asked few times, is there a discussion board (like matrix room) for these kinds of questions]

I'm planning to have few services running on proxmox to start with (namely NAS, Jellyfin, NVR).
Please let me know if I understood this correctly.
I don't think I need TrueNAS separately since I'm using proxmox for VMs.

NAS:
I can configure HDDs in RAID5/6 ZFS pool.
Install `nfs-kernel-server` package and share ZFS pool
Since proxmox is already taking care of ZFS and RAID, I don't see much need to just run `nfs` in container or VM, however if someone has different experience please let me know.

Jellyfin:
Can setup VM with access to ZFS pool

NVR:
Can setup VM with RW access to ZFS pool

What is general suggestion for storing VMs (OS drive is RAID1 ZFS, Data drives are RAID5/6 ZFS - yet to create).

 

[Dunuin]

What is general suggestion for storing VMs

On enterprise SSDs. Raid1 or even better raid10 if you care about performance.

NAS:
I can configure HDDs in RAID5/6 ZFS pool.
Install `nfs-kernel-server` package and share ZFS pool
Since proxmox is already taking care of ZFS and RAID, I don't see much need to just run `nfs` in container or VM, however if someone has different experience please let me know.

Will work but...
1.) more to learn because you need to secure and configure your NFS/SMB shares using the CLI. Limit specific NFS shares only to specific client IPs, create users and groups, set rights and ACL for auch dataset, ...all stuff where the PVE webUI won'T help you.
2.) harder to restore in case of a disaster.The webUI only offers you to backup VMs and LXCs and not the PVE host itself. This can still be done but again more complex and CLI required.

 

[hlab]

On enterprise SSDs. Raid1 or even better raid10 if you care about performance.

Ok so looks like I'll put them on OS drive (already NVMe and RAID1), since it's already setup in RAID1 and I don't have more SSDs (physical restraints - already at max number of drives)

Will work but...
1.) more to learn because you need to secure and configure your NFS/SMB shares using the CLI. Limit specific NFS shares only to specific client IPs, create users and groups, set rights and ACL for auch dataset, ...all stuff where the PVE webUI won'T help you.

Access is local only so wasn't planning to figure out groups / rights, idea is to just present a big mountable drive for LAN computers. But always better to have options if overhead is not bad

2.) harder to restore in case of a disaster.The webUI only offers you to backup VMs and LXCs and not the PVE host itself. This can still be done but again more complex and CLI required.

This definitely would make me hesitate to go with barebone approach. In that case what would be low overhead approach?

 

[Dunuin]

Access is local only so wasn't planning to figure out groups / rights, idea is to just present a big mountable drive for LAN computers. But always better to have options if overhead is not bad

And then think of all the people using your Wifi with Android smartphones that never received a security update for years while at the same time installing some random free apps of the app store. I personally don't want that such devices can access my private files on the NAS.

Ok so looks like I'll put them on OS drive (already NVMe and RAID1), since it's already setup in RAID1 and I don't have more SSDs (physical restraints - already at max number of drives)

The key phrase there was "enterprise" grade. Depending on the number of VMs and type of workloads those consumer SSDs with cheap NAND (especially QLC NAND) and missing power-loss protection can be terribly slow when continuously writing to it and might not last that long. Sometimes even slower than an HDD. So you will have to test it if that works for you.

In that case what would be low overhead approach?

How do you define "low overhead"? And also depends how serious you take your backups. For a homelab with 3-2-1 rule I would get another server with even more HDD capacity + SSDs as special devices. Then install Proxmox Backup Server (PBS) to where you could store your VMs/LXCs and even the datasets of your ZFS pool and PVE configs by using the proxmox-backup-client.
Then even more HDDs, but external ones, which you rotate for a offline+offsite backup and where you rsync your most important files and vzdump backups to. Keep in mind that raid is not a backup and backups stored in the same server often won't help much.

For a commercial use case I would set up multiple PBS servers at different locations and sync them + tape archive.

 

[hlab]

And then think of all the people using your Wifi with Android smartphones that never received a security update for years while at the same time installing some random free apps of the app store. I personally don't want that such devices can access my private files on the NAS.

This makes more sense now.
Ok so I should check on either LXC with just configured NFS, OpenMediaVault VM, TrueNAS VM (more on this on 3rd point)

The key phrase there was "enterprise" grade. Depending on the number of VMs and type of workloads those consumer SSDs with cheap NAND (especially QLC NAND) and missing power-loss protection can be terribly slow when continuously writing to it and might not last that long. Sometimes even slower than an HDD. So you will have to test it if that works for you.

Now this I never thought of. I got 2 NVMe (TLC) and configured them in RAID1.
I'll keep this in mind going forward, currently all hardware is new in server so hopefully it gives me at least an year of a time to figure out backups.

How do you define "low overhead"? And also depends how serious you take your backups. For a homelab with 3-2-1 rule I would get another server with even more HDD capacity + SSDs as special devices. Then install Proxmox Backup Server (PBS) to where you could store your VMs/LXCs and even the datasets of your ZFS pool and PVE configs by using the proxmox-backup-client.
Then even more HDDs, but external ones, which you rotate for a offline+offsite backup and where you rsync your most important files and vzdump backups to. Keep in mind that raid is not a backup and backups stored in the same server often won't help much.

I've been using low overhead wrongly - I don't mean performance wise, I mostly mean that TrueNAS provide many more functions which look unnecessary (Jails).
Maybe I should look at OpenMediaVault or something which is lightweight (directed NAS functionality) and then install it in LXC or VM.
As of now I only found TrueNAS, OMV and Linux LXC + nfs-kernel-server, I'm still looking.

As for backup plan what you pointed out looks good but I may need to wait to implement it (or work towards it slowly) already spent bunch on current system. Meantime hope for nothing to fail.

 

[hlab]

I found https://youtu.be/Hu3t8pcq8O0, going to go with that for network share