Use system with proxmox as NAS

[Hammerhand]

Hello. I have a powerful system with proxmox installed and several lxc, all of them with very light resources usage: one with home assistant, another one for a wireguard server and another one for docker (for now the main resources usage due to frigate).

I want to create a vm or lxc as a NAS for photo/video storage and I read about xpenology which seems to be a good option but I already have passthrough video to the docker lxc (for frigate and I will create a plex/jellyfin).

So my questions are: 1. any other way to create a NAS/like in proxmox? 2. any way to passthrough video card to two lxc?

Thanks in advance.

 

[Dunuin]

1. any other way to create a NAS/like in proxmox?

a) have storage on PVE host and run NFS/SMB server on PVE host
b) have storage on PVE host and bind-mount folders into a NAS LXC
c) have storage on PVE host and use virtual disks for a NAS VM/LXC
d) disk passthrough some disks into a NAS VM and create the storage in the guest OS
e) PCI passthrough a NVMe SSD or disk controller into a NAS VM and create the storage in the guest OS

2. any way to passthrough video card to two lxc?

Didn't tried that yet but should work.

 

[Hammerhand]

Well if I can passthough video card I guess this is a beggining. I've seen nextcloud is good for my needs so I will try it and then will go for plex/jellyfin.

I will post results when I start the process.

 

[Nollimox]

So my questions are: 1. any other way to create a NAS/like in proxmox? 2. any way to passthrough video card to two lxc?

I am using Open Media Vault since it's a NAS based on Debian and only requires 1GB RAM as opposed to TrueNAS which requires 8GB RAM and a recommended of 16GB...OMV has webGUI so, no need to passthrough any video card...

 

[sb-jw]

OMV has webGUI so, no need to passthrough any video card...

I don't think he wants to use the graphics card for the desktop in the VM, more for transcoding the videos in Plex

1. any other way to create a NAS/like in proxmox?

But remember that you are creating a chicken/egg problem for yourself here. If you put something important on the VM but your HV is no longer running, you won't be able to access it anymore. I have also always found that the GUI is somewhat unresponsive when a storage is no longer there.

 

[Dunuin]

I have also always found that the GUI is somewhat unresponsive when a storage is no longer there.

Yes, thats because of pvestatd locking up.
Can even prevent you from logging in at all: https://forum.proxmox.com/threads/unavailable-storage-will-make-the-webui-unusable.119107/

But remember that you are creating a chicken/egg problem for yourself here. If you put something important on the VM but your HV is no longer running, you won't be able to access it anymore.

Jup. For that I can highly recommend running at least 2 PVE nodes + have recent backups on both nodes or even on a third PBS server. If then a node will fail you could at least restore a recent version of your guests on the remaining node. My personal homelab favorite if you actually care about your data. Two TrueNAS VMs with identical pools on different nodes and then replicating everything between them works also great. Or three servers (2x PVE + 1x PBS as qdevice) with HA and then ZFS replication direktly between the two PVE Nodes. But probably too much footprint and cost for most people...

I've seen nextcloud is good for my needs

While it offers some NAS-like functionalities via webDAV Its not really great as a NAS. Its more if you want some private cloud ala Dropbox/OneDrive/GoogleDrive for sharing stuff online and syncing between devices.

 

[Milkwyrm]

Or a simple Linux (of your choice) VM/LXC with the 45 drives add-on for cockpit to manage samba/NFS shares. if you want external access then do it via a VPN or, of if you want, a nextcloud container and mount it inside that. By far the least resource intensive way to do it.

 

[Hammerhand]

Wow, reading you guys I stepped reading more about the options you provided and I think I maybe was complicating too much.

What I want to do is:

- To have several hdd attached to my system, some through SATA, some through a multi-hdd enclosure and being able to access its files through all my network (these way I can use some of these disks also for backups)
- To create a plex/jellyfin system which can access all these files and transcode them for entertaiment.
- To create a self-hosted cloud for video and photo backups from phone.

So maybe the best solution is:

- A lxc for plex/jellyfin and try to passthrough graphic card to it.
- A simple linux lxc as @Milkwyrm and a nextcloud container.

What do you think?

 

[Nollimox]

Two TrueNAS VMs

Do you find that it requires 16GB RAM for each?

 

[alexskysilk]

- To have several hdd attached to my system, some through SATA, some through a multi-hdd enclosure and being able to access its files through all my network (these way I can use some of these disks also for backups)

for simplicity (lack of complexity) you can run that on the hypervisor. the "storage" is then accessible even before any guests (lxc or qemu) are booted. This requires manual share and permission management. For ease of management, a nas distro managing the share. In low security (eg, home use) environments I'd lean toward the former method but I'm comfortable with cli management.

- To create a plex/jellyfin system which can access all these files and transcode them for entertaiment.

Im guessing you want to use your gpu; use lxc since it doesnt require exclusive access to it. you can share the storage directly as a bind mount (if you have the storage managed by the host) or as a network mount.

To create a self-hosted cloud for video and photo backups from phone.

same as above. the simplest way is to use tteck's excellent helper scripts
https://tteck.github.io/Proxmox/

 

[Dunuin]

Do you find that it requires 16GB RAM for each?

One is running with 12GB, one with 24GB and one with 32GB. Depends on different things like if you want to use iSCSI, services you are running, amount of storage, ...

 

[Hammerhand]

for simplicity (lack of complexity) you can run that on the hypervisor. the "storage" is then accessible even before any guests (lxc or qemu) are booted. This requires manual share and permission management. For ease of management, a nas distro managing the share. In low security (eg, home use) environments I'd lean toward the former method but I'm comfortable with cli management.


Im guessing you want to use your gpu; use lxc since it doesnt require exclusive access to it. you can share the storage directly as a bind mount (if you have the storage managed by the host) or as a network mount.


same as above. the simplest way is to use tteck's excellent helper scripts
https://tteck.github.io/Proxmox/

I see I have some knowledge problems with how storage works in proxmox. I set up the system more than one year ago and I have problem understanding english wiki.

I've asked here about it. I will start with improving my storage setup and then will keep with other lxcs.

Would be very glad if you can help me there too.

 

[Risker]

Jup. For that I can highly recommend running at least 2 PVE nodes + have recent backups on both nodes or even on a third PBS server. If then a node will fail you could at least restore a recent version of your guests on the remaining node. My personal homelab favorite if you actually care about your data. Two TrueNAS VMs with identical pools on different nodes and then replicating everything between them works also great. Or three servers (2x PVE + 1x PBS as qdevice) with HA and then ZFS replication direktly between the two PVE Nodes. But probably too much footprint and cost for most people...

Can you please elaborate on the topology?

If I understood correctly there are 2x TrueNAS VMs (one per node) with identical pools and replicating the data between the two.

How do you replicate the data across the two VMs?

I am currently only familiar with having one TruneNAS VM with HA on the PVE level, with replication to the other node.
To summarize, I have 2x N100 each with 1x SATA, 1x M.2 NVMe nodes, with an RasPi as the qdevice.
I have created a zpool for the 2TB M.2 NVMe drives, for VM replication and HA.
I want to use half of the M.2 NVMe storage (1TB) as highly available network storage.
My current thinking would be to have 1x TrueNAS VM with 1TB storage, that will be replicated across the nodes

 

[Dunuin]

If I understood correctly there are 2x TrueNAS VMs (one per node) with identical pools and replicating the data between the two.

How do you replicate the data across the two VMs?

Correct. TrueNAS comes with replication tasks (push or pull, also supports encryption) that you can use to keep two TrueNAS setups in sync.
Its not HA (no virtual IPs for failover of SMB/NFS) but at least you got a copy of all your data you could later replicate back or access via SMB/NFS using another IP.