Migrating homelab from Ubuntu Server to Proxmox

[verlax]

Hi everyone,

I'm planning to migrate my current homelab to Proxmox (I believe it's a more modular and scalable solution).
My current setup is a server running Ubuntu Server, with local storage and Docker containers for the apps I need/use:

OS: Ubuntu Server​

CPU: 6 cores / 12 threads​

RAM: 32GB​

OS Drive: 512GB M.2​

HDDs: 18TB + 18TB + 8TB + RAID1 (8TB + 8TB)​

Before migrating, I've been testing in a lab environment where I installed Proxmox. But the more I progress, the more doubts I have—I think it's because I'm still looking at it from a standard server perspective rather than a hypervisor one.

My initial idea was to use the following structure:

PROXMOX Node​

├── VMs​

│ ├── VM NAS with storage​

│ │ └── All HDDs​

│ ├── VM APPS con GPU passthrough​

│ │ └── Dockers​

│ └── VM HomeAssistant​

└── CTs​

├── CT Turnkey FileServer (Samba Share)​

├── CT 2FA​

├── CT Adguard​

├── Cloudflared​

└── Tailscale VPN​

However, I now have some doubts:

2. If I create a VM with all the disks and it fails for some reason... will I lose my data? Considering that I'll be backing up the machine itself (OS and system disk), I don't have enough space to backup all that data.

My alternative has been to not create the NAS VM and instead leave the disks in the Proxmox node, sharing them via NFS to the VMs. But this seems less intuitive and requires manual configuration every time I create something that needs access.​

2. Will the LXC containers for Cloudflared and Tailscale VPN consume resources that I could save by installing them directly on the node?

3. My plan for the test environment is to move it to another house and connect it via VPN, so I can keep testing everything I want without risking my "production" homelab, as well as using it as a Proxmox Backup Server (PBS). Would this be possible with Tailscale?

Now, my setup is looking more like this, which in my head feels like three layers:

- General services: Proxmox Node
- App/services: CTs
- Apps: VMs

PROXMOX Node​

├── Tailscale VPN​

├── Storage (Shared via NFS to LAN)​

├── Cloudflared​

├── VMs │ ├── VM APPS with GPU passthrough​

│ │ └── Docker containers​

│ └── VM Home Assistant​

└── CTs​

├── CT 2FA​

├── CT Turnkey FileServer (Samba Share) (with disks mounted via NFS from node)​

└── CT AdGuard​

I'm not sure if I'm overcomplicating things, or if it's really worth moving everything I currently have configured and working... but the more I see/read about Proxmox, the more I like the versatility it offers...

Any advice, similar experiences, or guidance would be greatly appreciated!

 

[Johannes S]

What problem do you want to solve you can't solve with your existing setup?

ProxmoxVE is way more flexible than a bare metal Linux server or a NAS OS like UnRAID, OpenMediaVault or Truenas. It's also great if you want to teach yourself network and system administration

But if you mainly want to host docker containers and are actually not interested in system maintenance a NAS OS with support for VM or docker containers might be less hassle (like UnRAID, OMV maybe TrueNAS (although they change their Virtualization stack quite often, so maybe not the most future-proof option).

My initial idea was to use the following structure:

Your structure looks fine and a lot more sensible than what most people do in their homelabs. I saw that you plan to host your docker containers in a VM with GPU passthrough which avoids the problems with docker inside lxc containers. What I don't get why you want to have a dedicated VM and a Turnkey Fileserver since both basically serves the same purpose. So why do you want to have a NAS VM?
I would also recommend to get some used enterprise ssds with PLP protection for the OS drive and for the VMs/lxcs os for a sufficient VM/LXC performance. HDDs are great as bulk data storage though and should be fine. ZFS software RAID should give you most flexibility and features see https://forum.proxmox.com/threads/f...y-a-few-disks-should-i-use-zfs-at-all.160037/

2. If I create a VM with all the disks and it fails for some reason... will I lose my data? Considering that I'll be backing up the machine itself (OS and system disk), I don't have enough space to backup all that data.

For a NAS VM you really want to have a dedicated controller for the discs see https://www.truenas.com/community/r...guide-to-not-completely-losing-your-data.212/ The discussed reasons are valid for other NAS (unRAID, OMV) os as well.

Regarding the loss of data you really need to think which riscs you are willing to take: In theory you can always have some issue which cause data loss (being an error of you, a hardware failure (e.G. broken discs), hacker attack etc). So you need an backup sufficient enough for all your important data.
And since your backup might get lost too you should have more than one:

The 3-2-1 rule is simple but effective in protecting important data from all sorts of threats, be it fires, natural disasters or attacks on your infrastructure by adversaries. In short, the rule states that one should create 3 backups on at least 2 different types of storage media, of which 1 copy is kept off-site.
https://pbs.proxmox.com/docs/storage.html#the-3-2-1-rule-with-proxmox-backup-server

Please note that I copied the PBS documentation but you don't need to use PBS for your backups (although it's good integration into ProxmoxVE makes it ideal for VMS and LXCs). How you achieve this depends on your budget and other constraints. One cheap (although manual) way is to buy some external USB discs and do you backups to them. One of them you would have at your place, the other one at another location (e.G. a friends or family member splace) and you would swap them out from time to time. It's a bad idea to have backups on the same node as your production since in case your node gets broken you have no way to access your backups. For the same reason it's usually recommended to not install the ProxmoxBackupServer on the ProxmoxVE host (be it directly on the host or as VM or lxc). But if you lack the hardware for a dedicated ProxmoxBackupServer you can do something of a compromise: The ProxmoxBackupServer allows backing up to external USB discs as "removable datastores" so you could do do a first backup directly on the host but from time to time you would attach an external USB drive and then sync to it. Alternatively you could rent a cloud vserver as remote PBS or use a service like tuxis.nl or Inett to have PBS cloud backups. The possibility to backup and restore complete VMs or LXCs to get a working configuration back in case is actually one of the benefits of ProxmoxVE even if you don't need it for your usecase otherwise.

2. Will the LXC containers for Cloudflared and Tailscale VPN consume resources that I could save by installing them directly on the node?

With lxc the overhead is rather minimal so I wouldn't be concerned. In general it's a good idea to not install additional software directly on the node since it's more envolved to get them back working after a reinstall while lxcs or vms can be easily backed up and restored. It also avoids that the vpn software messes with ProxmoxVE host networking, Software-Defined-Networks and firewall features.

3. My plan for the test environment is to move it to another house and connect it via VPN, so I can keep testing everything I want without risking my "production" homelab, as well as using it as a Proxmox Backup Server (PBS). Would this be possible with Tailscale?

I see no reason why not. I myself have tailscale inside several lxcs and vms which also serve as subnet router for the subnet of my ProxmoxVE nodes.
This allows me to access them (and the remainder of my home lan) even when I'm on the road

 

[verlax]

Wow! What an answer, thanks for taking the time, man...

What problem do you want to solve you can't solve with your existing setup?

A real problem? None - I could solve everything on bare metal, but I love to learn and play with new toys, and Proxmox and these migrations are giving me hours of fun. But don't get me wrong, I want to do things properly and the right way. After all, my homelab needs to work as well as it does now.

Maybe having my HomeAssistant VM in a proper way, not in a Virtualbox installed in bare metal.

Regarding the loss of data you really need to think which riscs you are willing to take

That's my newbie doubt: can I trust a VM as much as a bare metal server? I know there are always risks, and I have backups of all important data (original in RAID and [3-2-1 backup]), but since I don't have an HBA card (yet), anything that forces me to restore a VM backup would likely result in some form of data corruption.

That's why I thought about having the storage shared directly from the node via NFS to all the LXC containers and VMs.

What I don't get why you want to have a dedicated VM and a Turnkey Fileserver since both basically serves the same purpose

My bad - I reused the diagrams between the before and after scenarios and got mixed up. The Turnkey FS LXC is my alternative to the NAS VM. It's just a very low-resource server for sharing storage.

use a service like tuxis.nl or Inett to have PBS cloud backups

Yeah, I'll try tuxis free 150Gbs tier.

Again, thanks for your time.