help

[johndoe297]

so i'm new to self hosting, and my goal was to set up a little home server with a few services, nothing too crazy. the config i decided to go with was proxmox base, with an lxc running ubuntu server with docker and portainer to manage my containers. which all worked fine. however, my problem is that i have two drives in my machine, an ssd, and an hdd. i'd like to keep all the boot type stuff on the ssd for speed, and use the hdd for mass storage. i tried creating a directory in the proxmox host and mounting it to the CT, which seemed to work, however neither portainer, nor any containers running in there could use that drive. i couldn't add a volume of it to portainer, and i couldn't add it to nextcloud external storage either. i ended up breaking the whole system with chown commands so i had to wipe it all and start over. so my question is;

can anyone walk me through the proper process to create that setup, or should i just create another CT or VM for an nfs or smb share and do it that way? i feel like it should be doable without that, but hey, i don't really know what i'm doing.

planned setup: lxc running ubuntu
then install docker
then install portainer
then use portainer to install and mgm CTs, particularly nextcloud which i'd like to keep the boot on the ssd, and mass media on the hdd. the other CTs are just small services like pihole, cloudflare tunnel, maybe a small web server or two which i don't care if they use the ssd, they won't use much space.

 

[Dunuin]

Least frustating way: Run something like OpenMediaVault in a VM, use disk passthrough to passthrough the HDD into that VM, share the HDD using SMB and then create a VM for your portainer where you then could mount that SMB share.

There are some things to keep in mind when working with LXCs:
1.) you can`t mount SMB/NFS shares inside unprivileged LXCs (only on the less secure privileged LXCs and only after enabling that feature)
2.) when working with unprivileged LXC you need to work with user/group remapping where UID/GID 0 - 655535 inside the LXC get mapped to 100000 - 165535 on the host
3.) when using LXCs you very often encounter limitations, as LXCs share the kernel/hardware with the PVE host. You either increase isolation which might break stuff or you lower the isolation which makes the LXC less secure

 

[johndoe297]

i see. i was thinking of just setting up a truenas vm or something similar, making an nfs or smb and adding it to nextcloud that way, because i think that can be done easily enough. if i were to run my ubuntu docker portainer in a vm wouldn't it be more resource hungry? because the lxc with everything running was barely using anything.

 

[Dunuin]

if i were to run my ubuntu docker portainer in a vm wouldn't it be more resource hungry? because the lxc with everything running was barely using anything.

Sure, it would be, but still the recommended way:

If you want to run application containers, for example, Docker images, it is recommended that you run them inside a Proxmox QEMU VM. This will give you all the advantages of application containerization, while also providing the benefits that VMs offer, such as strong isolation from the host and the ability to live-migrate, which otherwise isn’t possible with containers.

 

[johndoe297]

alright, i guess i will do it that way. learning is fun to an extent, but after two days of head scratching i'm kind of over it. lol

 

[greyskull]

alright, i guess i will do it that way. learning is fun to an extent, but after two days of head scratching i'm kind of over it. lol

Have you figured this out by any chance? Right now I am looking to do something similar but am stuck using an inefficient and resource hungry Windows 2019 Server VM to share the disks over my LAN.

 

[johndoe297]

Have you figured this out by any chance? Right now I am looking to do something similar but am stuck using an inefficient and resource hungry Windows 2019 Server VM to share the disks over my LAN.

No, unfortunately I did not. I'm not good enough with code and permissions or whatever shenanigans id blocking this. I ended up just spinning up a truenas vm and making an smb. I can add that volume to portainer no problem. However I did find out that creating a VM and passing through directories to them is as simple as just adding it to the VM storage in the creation menu, or just going to the hardware tab on a running VM and clicking add storage. So, the lxc is the problem unfortunately.

 

[greyskull]

Thank you, thank you, thank you! In fact this was my intended use case anyway - wasn't sure which to try between Plex, Jellyfin, OpenMediaVault etc.
I have two drives I just mapped one of the drives to iScsi using my virtualized TrueNAS. Hitting the same 1gbps network bottleneck until I upgrade as expected. iScsi is interesting but seems unnecessarily convoluted but that's Windows/Microsoft for you.

Next little homelab project will be OpenMediaVault and I can retire this Win2019Server VM resource hog. Though I do like Windows Server more than Win10/11.

Motherboard	SuperMicro X10SLL-F [Supermicro.com Details]
CPU	Intel(R) Xeon(R) CPU E3-1231 v3 @ 3.40GHz (4 core 8 thread)
RAM	32GB Unbuffered ECC UDIMM DDR3-1600MHz Sk-Hynix
PCI-E x16​	NVIDIA GT1030 if could physically fit my old GTX1650 in there I would. (Planning on using it for hardware encoding/transcoding)
PCI-E x8​	SATA 4-port Expansion Card using The Asmedia1062 chipset for additional drives. [Amazon link to card details]
PCI-E x4​	NVIDIA GT720 Specifically for my Hackintosh Monterey VM. It's really awesome if you happen to have the correct hardware.
SATA-0 (v3)	SPCC Solid State Disk 500GB
SATA-1 (v3)	Seagate Barracuda 1TB STIOOODM010-2EP102 7200rpm 64Mb cache
SATA-2 (v2)	WD Caviar Black 1 TB WD1001FALS 7200rpm 32Mb Cache
SATA-3 (v2)	Seagate Barracuda 1TB ST1000DM003-1CH162 7200rpm 64Mb cache
SATA-4 (v2)	Seagate Barracuda 1TB ST1000DM003-1CH162 7200rpm 64MB cache
SATA-5 (v2)	Seagate Barracuda 1TB ST31000528AS 7200rpm 32Mb cache
Asmedia1062-1	TOSHIBA 2TB DT01ACA200 7200rpm 64Mb cache (Pass-Through to VM) *iScsi via TrueNAS
Asmedia1062-2	WDC RED 4TB CMR WD40EFZX 5400rpm 128Mb cache (Pass-Through to VM) *These are the HDD's I am considering purchasing and replacing the 1TB RaidZ1 with. Just temporary as I acquire 3 more.

OS: Proxmox-ve: 8.0.1 (running kernel: 6.2.16-3-pve) on a 500GB SSD
TrueNas as primary VM: TrueNAS-13.0-U5.2

Least frustating way: Run something like OpenMediaVault in a VM, use disk passthrough to passthrough the HDD into that VM, share the HDD using SMB and then create a VM for your portainer where you then could mount that SMB share.

There are some things to keep in mind when working with LXCs:
1.) you can`t mount SMB/NFS shares inside unprivileged LXCs (only on the less secure privileged LXCs and only after enabling that feature)
2.) when working with unprivileged LXC you need to work with user/group remapping where UID/GID 0 - 655535 inside the LXC get mapped to 100000 - 165535 on the host
3.) when using LXCs you very often encounter limitations, as LXCs share the kernel/hardware with the PVE host. You either increase isolation which might break stuff or you lower the isolation which makes the LXC less secure

 

[johndoe297]

Thank you, thank you, thank you! In fact this was my intended use case anyway - wasn't sure which to try between Plex, Jellyfin, OpenMediaVault etc.
I have two drives I just mapped one of the drives to iScsi using my virtualized TrueNAS. Hitting the same 1gbps network bottleneck until I upgrade as expected. iScsi is interesting but seems unnecessarily convoluted but that's Windows/Microsoft for you.

Next little homelab project will be OpenMediaVault and I can retire this Win2019Server VM resource hog. Though I do like Windows Server more than Win10/11.

Motherboard	SuperMicro X10SLL-F [Supermicro.com Details]
CPU	Intel(R) Xeon(R) CPU E3-1231 v3 @ 3.40GHz (4 core 8 thread)
RAM	32GB Unbuffered ECC UDIMM DDR3-1600MHz Sk-Hynix
PCI-E x16​	NVIDIA GT1030 if could physically fit my old GTX1650 in there I would. (Planning on using it for hardware encoding/transcoding)
PCI-E x8​	SATA 4-port Expansion Card using The Asmedia1062 chipset for additional drives. [Amazon link to card details]
PCI-E x4​	NVIDIA GT720 Specifically for my Hackintosh Monterey VM. It's really awesome if you happen to have the correct hardware.
SATA-0 (v3)	SPCC Solid State Disk 500GB
SATA-1 (v3)	Seagate Barracuda 1TB STIOOODM010-2EP102 7200rpm 64Mb cache
SATA-2 (v2)	WD Caviar Black 1 TB WD1001FALS 7200rpm 32Mb Cache
SATA-3 (v2)	Seagate Barracuda 1TB ST1000DM003-1CH162 7200rpm 64Mb cache
SATA-4 (v2)	Seagate Barracuda 1TB ST1000DM003-1CH162 7200rpm 64MB cache
SATA-5 (v2)	Seagate Barracuda 1TB ST31000528AS 7200rpm 32Mb cache
Asmedia1062-1	TOSHIBA 2TB DT01ACA200 7200rpm 64Mb cache (Pass-Through to VM) *iScsi via TrueNAS
Asmedia1062-2	WDC RED 4TB CMR WD40EFZX 5400rpm 128Mb cache (Pass-Through to VM) *These are the HDD's I am considering purchasing and replacing the 1TB RaidZ1 with. Just temporary as I acquire 3 more.

OS: Proxmox-ve: 8.0.1 (running kernel: 6.2.16-3-pve) on a 500GB SSD
TrueNas as primary VM: TrueNAS-13.0-U5.2

sounds cool. i mean, i'm learning that when it comes to this home lab stuff there is definitely more than one way to skin a cat. my gut tells me that there is a way to do the passthrough to an lxc which would save a lot of resources because truenas is thirsty, but it really comes down to how much of a headache you want. a lot of people say openmediavault is pretty good for managing media and mass storage, but i've not used it, and i don't know much about it. my thinking is whatever nas like software you choose it will be easier and better than proxmox innate management options. truenas seems to be like a high level software, there's a lot that it can do, but it's definitely not beginner friendly, if you're looking for ease of use, i would not choose truenas. omv is probably more streamlined and user friendly, if you've got the time to dump into learning truenas, it would probably be the more powerful option. seems more enterprise orientated to me. a guy on another forum recommended filecloud and said it was the best there is, however i tried it, and in my personal experience it was a real pain, i couldn't even get it installed. probably my fault, but i had no trouble with any of the other stuff i tried. i wasn't able to get a simple ubuntu server running in a proxmox vm, kept having issues with the mirror archives failing, then when i finally got that working, the filecloud install errored on the database so that was a whole day blown. you may have better luck than me. though, if resources are an issue, you may consider just running one rather than both, especially if you're running them in a proxmox base like me. probably dump truenas because it's resource usage is pretty high, i think the recommended min is 2 cores and 16gb ram, you can do it with less though, but you wouldn't really need to, because i think omv can do nfs and smb shares too. if you're planning on using omv as your hypervisor, then again, i don't see the point of running truenas on it. i wouldn't be running it for other than i can't figure out if proxmox does nfs or smb natively or what. it's certainly not straightforward if it does.

as far as plex vs jellyfin, there is a linus video that covers them, i would watch that and go from there, i'll link it if this forum allows links.

https://youtu.be/jKF5GtBIxpM

it essentially just seems to boil down to what features you can live without and whether you want to pay monies. i think if it was me, jellyfin would probably be fine. i'm not a huge media server guy though. you docker either one though, so that would probably be the least resource wasting option.

i would say for ease of use and saving yourself a lot of headaches, probably do the proxmox hypervisor, then do a VM with omv and set up all your mass storage there, and use it's built in docker to run your media whether plex or jellyfin. then if you want any other containers going and it's okay if they're in some weird semi isolated environment, do a separate LXC with docker/portainer and just let that run whatever stuff that doesn't need any connectivity to other vms or conts. that's what i'm doing and that CT barely uses any resources. i have a very similar set up to you, i7-4770, 32gb ram, on ssd and one hdd and with my little homelab, i don't even go over 3% cpu, and i'm usually just shy of 6gb ram at any given time. i probably don't run as much stuff as you though.