Yet another very newbie question about strategic use of proxmox (VM nas, raids, passthroughs etc)

[valk]

Hi and sry for that awful name of the thread but I can't make it more narrow.
So I am just starting with proxmox and my head is dangerously close to exploding. I need advice on how to plan the structure of my homelab server.
Core things I need from my system:
--Raid-5 (4x4TB hdds) on vm NAS xpenology.
--another VM xpenology with 1 hdd for video surveillance (don't need redundancy here and I don't want to include recordings in shanpshots or backups)
--LXC with Jellyfin server using NFS mounted share from VM NAS as media library and GPU passthough for transcoding
--vpn server on LXC or VM (which is better?) for LTE connection to home lan.

I am gonna use an usb hdd drive for backup of proxmox and VMs+LXCs but I can only afford 1 snapshot spacewise + I am gonna use amazon cloud as back up for the critical data.

So here comes some questions:
1) Where do I setup my raid? On proxmox (can I even do that there?) and just virtualize whole space on VM NAS as jbod? Or do I passthrough each of 4 HDDS to NAS VM and build raid there? Loosing snapshots as I understand that way... 3rd option would be to buy an PCI sata card and passthrough it to NAS so I can do snapshots, right?
2) How to setup a Surveillance VM (xpenology) so it would use a physical dedicated HDD and I would include this VM in the backup but not the recordings.
3) Do I want ZFS for homelab use? I don't want to spend more on ram but I am not rly familiar with ZFS and maybe missing a good opportunity here? Not rly afraid of losing family photos with cloud backup + onsite.

 

[Dunuin]

--Raid-5 (4x4TB hdds) on vm NAS xpenology.
--another VM xpenology with 1 hdd for video surveillance (don't need redundancy here and I don't want to include recordings in shanpshots or backups)

Didn't used xpenology yet, but I would bet it is possible to have a raid5 and a single disk in the same VM. So you probably don'T need to waste ressources buy running two NAS VMs.

LXC with Jellyfin server using NFS mounted share from VM NAS as media library and GPU passthough for transcoding

Keep in mind that you can't mount NFS shares in a unprivileged LXC. You need a privileged LXC for that and it requires that you enable the NFS feature first.

--vpn server on LXC or VM (which is better?) for LTE connection to home lan.

VMs are more secure because of better isolation, so I personally run everything in a VM that is accessible from the internet. So I would use a VM for that.

1) Where do I setup my raid? On proxmox (can I even do that there?) and just virtualize whole space on VM NAS as jbod?

That is one option. Creating the raid on the PVE host and then using a single virtual disk to store your data for the xpenology VM.

Or do I passthrough each of 4 HDDS to NAS VM and build raid there?

Also possible. But keep in mind that the xpenology will still only work with 4 virtual disks that are mapped to the 4 physical disks. So SMART inside the VM won't be working and you get a bit of additional overhead. Only option that would allow the VM to directly access the real hardware would be to PCI passthrough the whole disk controller with all of its ports (which probably would require to buy a HBA card).

Loosing snapshots as I understand that way...

3rd option would be to buy an PCI sata card and passthrough it to NAS so I can do snapshots, right?

Depends...only creating the raid on the PVE host without any passthrough would allow PVE to create snapshots. But maybe xpenology supports some filesystems like ZFS or btrfs that also llow snapshots. Then you could let xpenology do the snapshotting on the guest level.

3) Do I want ZFS for homelab use? I don't want to spend more on ram but I am not rly familiar with ZFS and maybe missing a good opportunity here? Not rly afraid of losing family photos with cloud backup + onsite.

You should do some research on bit rot. Having onsite and offsite backups won't prevent that you lose your family fotos. All bits will rot over time causing data corruption. If your filesystem and RAM can't detect these corruptions, you just backup corrupted data without knowing it. Backups won't help if all backups contain the same corrupted file because it corrupted years ago without you noticing it. So ZFS with its bit rot protection and ECC RAM is in my opinion a must have if you really care about your data. But yes, SSD wear will be bad with ZFS, you lose a lot of capacity (pools shouldn't be filled more than 80%), you will need to buy more expensive hardware (CMR HDDs, enterprise SSDs and ECC RAM) and need more RAM and CPU power.

 

[valk]

Didn't used xpenology yet, but I would bet it is possible to have a raid5 and a single disk in the same VM. So you probably don'T need to waste ressources buy running two NAS VMs.

I need 2nd VM cos of the licensing issues, NAS vm has only 2 camera licenses, surveillance VM has 8. So the question still stands how to setup storage in that VM in a way to be able to back up VM but not the recordings?

Keep in mind that you can't mount NFS shares in a unprivileged LXC. You need a privileged LXC for that and it requires that you enable the NFS feature first.

Yeah i found out that the hard way after like 3 hours of frustration with "mount".

That is one option. Creating the raid on the PVE host and then using a single virtual disk to store your data for the xpenology VM.

Also possible. But keep in mind that the xpenology will still only work with 4 virtual disks that are mapped to the 4 physical disks. So SMART inside the VM won't be working and you get a bit of additional overhead. Only option that would allow the VM to directly access the real hardware would be to PCI passthrough the whole disk controller with all of its ports (which probably would require to buy a HBA card).

Depends...only creating the raid on the PVE host without any passthrough would allow PVE to create snapshots. But maybe xpenology supports some filesystems like ZFS or btrfs that also llow snapshots. Then you could let xpenology do the snapshotting on the guest level.

So what would be (in your opinion) my 1st choice here? I am asking cos I am pretty sure I don't have enough knowledge about VE in general to be able to make this decision.

You should do some research on bit rot. Having onsite and offsite backups won't prevent that you lose your family fotos. All bits will rot over time causing data corruption. If your filesystem and RAM can't detect these corruptions, you just backup corrupted data without knowing it. Backups won't help if all backups contain the same corrupted file because it corrupted years ago without you noticing it. So ZFS with its bit rot protection and ECC RAM is in my opinion a must have if you really care about your data. But yes, SSD wear will be bad with ZFS, you lose a lot of capacity (pools shouldn't be filled more than 80%), you will need to buy more expensive hardware (CMR HDDs, enterprise SSDs and ECC RAM) and need more RAM and CPU power.

I dove a bit into bit rot topic and damn... thats a lot of info to process and learn. But when i was talking about not being afraid of loosing family photos - I meant that it's not enterprise data and I am ok with the idea of loosing some of those pics through the years. Xpenology has btrfs though, so there is that.