From OpenMediaVault to proxmox server

[RomainD2]

Hi,

I would like some help do choose the best setup to create my first proxmox server. I used to use Openmediavault with docker containers, but after some problems to install OMV7 on my new hardware setup, I choose to install proxmox to separate Home assistant OS and other docker containers from the OMV7 system. But as it is my first proxmox configuration, I would like some help to chose the righ parameters before going on the wrong way.

The Hardware :

• Motherboard with J5040 processor
• 16 Go DDR4
• Two 64 Go SD card to have two proxmox install.
• One 1 To SSD for containers data and VMs
• One 4 To HDD for my backups
• One 10 To HDD for other datas
• One 2 To HDD for other datas

I would like to have the following services :

• NAS to access to my video files with my ShieldTV or from my computer, and make backups with Duplicati for example. I used tu use Openmediavault, but why not try something else (CasaOS, DSM, … ? ) I need to have HDD spin down on all HDD .
• Home assistant pour la maison, avec accès sécurisé depuis l’extérieur
• Nextcloud
• JDownloader
• Portainer to run some Docker containers and have a graphical interface
• An outside access to all previous services : SWAG and Duck DNS ?

I made a slide about this installation to have a better view of what I would like to do :


And I would like some advices with RAM, disk and Core allocation, avec how to manage all drives to have spindown support and that can be reachable with my servicies.

Thanks for your help !

 

[iamralf]

There are some specific applications you want to run there. Who can say how much RAM etc. they would use. Nextcloud can be memory hungry depending on the workload. Etc.
It will be tight with your resources I guess
But Proxmox is very good at managing resources and providing all the necessary services!

 

[zzz09700]

By the look of things you'd need another VM with all HDDs passed through and running NFS/SMB.

 

[gfngfn256]

• Motherboard with J5040 processor
• 16 Go DDR4

Interesting, according to official Intel docs here, the J5040 only supports up to 8gb DDR4.

Two 64 Go SD card to have two proxmox install.

Proxmox OS writes a lot of stuff - those SD cards won't last.

The rest of the stuff is all doable on your HW, albeit RAM could be better, 4 cores no HT will have to do.

 

[RomainD2]

Thanks for your help !

There are some specific applications you want to run there. Who can say how much RAM etc. they would use. Nextcloud can be memory hungry depending on the workload. Etc.
It will be tight with your resources I guess
But Proxmox is very good at managing resources and providing all the necessary services!

All these services was working well on my previous hardware, with OpenMediavault OS and Docker containers, based on a J1900 processor which is a lot worse than the J5040. The easy part with OMV is that we don't care about core / ram / disks allocation.

I understand that it can be difficult to know exactly the allocation, but is it possible to have an idea from users that use theses services ?

By the look of things you'd need another VM with all HDDs passed through and running NFS/SMB.

Ok, if I add a VM with all HDDs passed through, will I be abble to access to these drives with the others VMs ?

Interesting, according to official Intel docs here, the J5040 only supports up to 8gb DDR4.

Yes, but it is working with two 8 Go stick

Proxmox OS writes a lot of stuff - those SD cards won't last.

Is it better to use a separate partition on mu 1 To SSD drive ? How much Go should I allocate to this partition ?

 

[gfngfn256]

Is it better to use a separate partition on mu 1 To SSD drive ? How much Go should I allocate to this partition ?

Definitely better on SSD. How much you want to allocate, depends on your usage. The OS itself really doesn't need a lot. The Proxmox installer will usually divide the Disk, into root, LVM & LVM-thin. You can control in the installer how much (sort of, see here). Read up carefully, the various usages in PVE & then decide what you need & want BEFORE you install. There's plenty on this forum on the subject.

 

[zzz09700]

Well I guess I'll be hated here but
For what OP wants there is no need to mess around with virtualization stuff, you could have a Debian/Ubuntu and get it to do ALL you want, without planning for cross-VM disk accessing or extra memory load brought by multiple Linux kernel running spontaneously (aka kernel trash). Especially by the look of things OP doesn't have much memory to toss in.

If that's just for fun then you can do whatever you want and fiddle around to see what works and what not. If OP just want something that works, you are probably on the wrong direction. Just try to fix what's broken with OMV, that the optimal solution.

If you just need Home Assistant OS (and I do think that's all you need) in a rather seperate environment then install OMV first then VirtualBox headless then PhpVirtualBox then put that HAOS into a virtualbox VM.

 

[RomainD2]

Well I guess I'll be hated here but
For what OP wants there is no need to mess around with virtualization stuff, you could have a Debian/Ubuntu and get it to do ALL you want, without planning for cross-VM disk accessing or extra memory load brought by multiple Linux kernel running spontaneously (aka kernel trash). Especially by the look of things OP doesn't have much memory to toss in.

If that's just for fun then you can do whatever you want and fiddle around to see what works and what not. If OP just want something that works, you are probably on the wrong direction. Just try to fix what's broken with OMV, that the optimal solution.

If you just need Home Assistant OS (and I do think that's all you need) in a rather seperate environment then install OMV first then VirtualBox headless then PhpVirtualBox then put that HAOS into a virtualbox VM.

I am not so confident anymore about OpenMediavault, I think OMV is going to the wrong direction, getting more complicated and have more peculiarities to install docker / proxmox years after years, and for example, pluggins which were a good idea for beginers are abandoned.

I dream of having an OS with the primary functionalities of a NAS with a graphical configuration (secure remote access, access to hard disk files, backup software) associated with a few plugins for other services that can be installed in one click . All without being dependent on a company's hardware (Synology, etc.).

But yes, I am not so confident to migrate to proxmox as I will certainly have some troubles... The main problem is in fact to have HAOS separate from OMV, as if I want to install new version of OMV, I wouldn't have to touch to my domotics.

 

[Dunuin]

I understand that it can be difficult to know exactly the allocation, but is it possible to have an idea from users that use theses services ?

Jdownloader: 3 vCPUs and 4GB RAM (otherwise browsing will be terrible and extracting eats CPU and RAM)
Nextcloud: 2 vCPUs and 4GB RAM (maybe more in case you want apps like clamav and office)
OMV: 2 vCPUs and 4GB RAM (probably less RAM will work but caching is nice for performance...especially in case you want ZFS)
Portainer: totally depends on the type and amount of containers...I gave my Portainer VMs each 4 vCPUs and 8GB RAM...could be hundreds of cores and GBs of RAM if you want to run lots of containers
HomeAssistant: 2vCPUs and 2GB RAM (but 1 vCPU should wrok too I think)

I am not so confident anymore about OpenMediavault, I think OMV is going to the wrong direction, getting more complicated and have more peculiarities to install docker / proxmox years after years, and for example, pluggins which were a good idea for beginers are abandoned.

I dream of having an OS with the primary functionalities of a NAS with a graphical configuration

Then you won't like PVE. PVE is not for beginners. After all it is a full headless linux server and you should be familiar with the CLI. Its waaay more complicated than something like TrueNAS and that is already way more complicated than OMV. And again, PVE got no NAS functionalities. It's a pure hypervisor.

 

[gfngfn256]

I totally agree with Dunuin:

They you won't like PVE. PVE is not for beginners. After all it is a full headless linux server and you should be familiar with the CLI. Its waaay more complicated than something like TrueNAS and that is already way more complcated than OMV. And again, PVE got no NAS functionalities. It's a pure hypervisor.

The only 2 reasons, I see you needing PVE, is the backup feature. For instance your HAOS instance which will sit as a totally independent VM in PVE, can be regularly backed up & restored on-the-fly. You mis-configured something wrong/upsetting/breaking in your VM, at the push of a button, you'll be back to where you were. Secondly, the other redeeming feature for you will be flexibility in adding any LXCs or VMs that you may need in the future.

But don't go over to PVE thinking its a web-page you just click on to get any services your heart desires. Its a far cry from that.

However: it is doable. But the price will be the learning curve - but it's doable, and well worth it in the long run IMO.

I believe - based on the outline you've posted above - you're enough tech-savvy to get the hang of it. In the end you won't regret it. There's a wealth of knowledge (both on this awesome forum & others) to get any help you seek - I still learn something new every time I research a topic.

I've recommended PVE to others who only run HAOS (& appear less tech-savvy than yourself) - & they have nothing but praise for PVE & haven't looked back.

Take PVE slowly & methodically - don't cut corners & don't do anything you don't comprehend - you'll soon see you're a true ace!

 

[panchuz]

Proxmox OS writes a lot of stuff - those SD cards won't last.

Can you please elaborate on that?

Is it just logs? Can we simlink them away to an HDD for example?

Is there other stuff...? Workarrounds?

 

[RomainD2]

I totally agree with Dunuin:


The only 2 reasons, I see you needing PVE, is the backup feature. For instance your HAOS instance which will sit as a totally independent VM in PVE, can be regularly backed up & restored on-the-fly. You mis-configured something wrong/upsetting/breaking in your VM, at the push of a button, you'll be back to where you were. Secondly, the other redeeming feature for you will be flexibility in adding any LXCs or VMs that you may need in the future.

But don't go over to PVE thinking its a web-page you just click on to get any services your heart desires. Its a far cry from that.

However: it is doable. But the price will be the learning curve - but it's doable, and well worth it in the long run IMO.

I believe - based on the outline you've posted above - you're enough tech-savvy to get the hang of it. In the end you won't regret it. There's a wealth of knowledge (both on this awesome forum & others) to get any help you seek - I still learn something new every time I research a topic.

I've recommended PVE to others who only run HAOS (& appear less tech-savvy than yourself) - & they have nothing but praise for PVE & haven't looked back.

Take PVE slowly & methodically - don't cut corners & don't do anything you don't comprehend - you'll soon see you're a true ace!

Thanks,

Yes, I understood that Proxmox is not a graphical NAS service. And I think that I can do it, but I would like to avoid some basics errors and take them into account from now, that's why I created the above graphic in order to choose the right VMs configuration for my needs. And from your perspective and your experience, I think that you can identify the right configuration in few seconds, but as there are a lot of possibilities, maybe I am wrong .

For example, maybe it is better do do only two VMs, one for HAOS, and the other for OpenMediaVault, and run dockers containers inside OMV. Maybe this solution is easier for ressources allocations, specially for disk paththrough.

 

[gfngfn256]

Can you please elaborate on that?

You can search yourself on the subject (on this forum & others) - I'll only link one here, its oldish but rather interesting. There is loads out there.

There are various workarounds - ranging from useless to useful - but IMO none are perfect.

As a rule of thumb - my motto is: Be ready to replace any disk at any time. Maybe use enterprise HW for anything you can't do without.

 

[Dunuin]

Is it just logs? Can we simlink them away to an HDD for example?

Not only log. Logs, metrics of all the graphs and also constant writes to the config DB. And yes, there are hacks to reduce the writes...but not recommended if you care about stability. Power outage or kernel crash and all that then volatile data will be lost. Maybe not that problematic to lose some logs or metrics but losing config files stored in that DB might be bad.

If you care about the services you are running, I would highly recommend to use a mirror of HDDs/SSDs for your system disk. Keep in mind that there is still no official way to backup your PVE host, so it is very annoying to lose the system disk...

For example, maybe it is better do do only two VMs, one for HAOS, and the other for OpenMediaVault, and run dockers containers inside OMV. Maybe this solution is easier for ressources allocations, specially for disk paththrough.

With that less RAM you won't have much fun running VMs. You will have to mostly stick to LXCs. Also keep in mind that you shouldn't overprovision your RAM and that PVE itself needs 2GB. And then you want some GBs of free RAM to not OOM, for caching and for the virtualization overhead (a 4GB VM might actually consume 4.5 or 5GB of RAM as the KVM process needs ressources too). So at most I would assign a sum of 12GB to all guests combined. So 6x 2GB VMs/LXCs, 3x 4GB VMs/LXCs, 1x 4GB + 2x 2GB + 4x 1GB and so on. Thats not much to work with. For what you want I would have bought at least a system with 32GB RAM and free RAM slots to have to option to expand it to 64GB later in case you want some more services.

 

[RomainD2]

With that less RAM you won't have much fun running VMs. You will have to mostly stick to LXCs. Also keep in mind that you shouldn't overprovision your RAM and that PVE itself needs 2GB. And then you want some GBs of free RAM to not OOM, for caching and for the virtualization overhead (a 4GB VM might actually consume 4.5 or 5GB of RAM as the KVM process needs ressources too). So at most I would assign a sum of 12GB to all guests combined. So 6x 2GB VMs/LXCs, 3x 4GB VMs/LXCs, 1x 4GB + 2x 2GB + 4x 1GB and so on. Thats not much to work with. For what you want I would have bought at least a system with 32GB RAM and free RAM slots to have to option to expand it to 64GB later in case you want some more services.

All above services were running well on an oldest processor, a Q1900 motherboard, with only 8 GB RAM.

I thought that a J5040 and 16 Go RAM would be enought to use the same services with virtual machine. I choose this hardware as I don't want a hungry power consumption server.

If I understood well, If I want to use HAOS and OMV on Proxmox, I need a minimum of two VMs as I can't run them into LXCs.

 

[gfngfn256]

HAOS and OMV on Proxmox, I need a minimum of two VMs as I can't run them into LXCs.

AFAIK that is correct. OMV I don't use - but I never got my head around the famous "tteck" making a script to create an OMV LXC. I guess if you only want to access the local LXC disk you'd be OK - but for anything else its pretty useless. There are nasty/destructive workarounds to access other disks (mountpoints etc.. permissions nightmares etc.) - I wouldn't go there. As far as HAOS, that can only be run correctly (in its OS) format as a VM. Some enterprising users have tried to run it in an LXC - see here - not sure of the outcome. You could obviously run HA core in an LXC - but that's not my flavor.

 

[Dunuin]

I choose this hardware as I don't want a hungry power consumption server.

There are cheap N100 MiniPCs that support 32 GB of RAM and Ryzen Mobile MiniPCs that support 96GB.

 

[RomainD2]

Well, I've decided what I'm going to do with my server, and I'm going to try my luck with the J5040 and its 16 GB of RAM (by the way, apparently you can go up to 32 GB), with just 2 VMs to begin:

• VM1: HAOS
• VM2: OMV 7 with the three HDDs in path-through so as to be able to put them to sleep. The Dockers containers will be under OMV, so I'll have access to my HDD data.

However, I still have questions about my 1TB SSD for the OS (50GB), VM data (200GB), and a “temporary” folder (The Rest) for downloads or other non-sensitive files. This is mainly due to the fact that I only have 4 sata ports on my motherboard.

1 - Is it a bad idea to use a single SSD for all this? I could possibly change this option, I've got room in my case, but that would mean buying a new SSD and an M2 sata expansion card.

2 - I've understood the backup of VMs, to put them on one of my HDDs, but in the event that my 1TB SSD in my current configuration fails, I haven't really understood how I can simply restore the Proxmox OS part. Would I have to redo a fresh installation of proxmox, re-partition, and then restore the VMs on my HDD, and get back to my previous configuration? Is this also valid for hardware changes (CM, processor), or is it transparent for the VMs?

3- ZFS: From what I understand, it's a software raid thing, but some people use ZFS with a single disk in YouTube tutorials, so I haven't found the reason. In any case, I can't do this with my configuration if I've understood correctly, because I wouldn't be able to make a partition for my temporary data?

4- Some people merge the local-lvm partition with the main local partition. Do you recommend this?

Thanks for your help

 

[Dunuin]

1 - Is it a bad idea to use a single SSD for all this? I could possibly change this option, I've got room in my case, but that would mean buying a new SSD and an M2 sata expansion card.

Ideally you would use 4 disks. 2 SSD/HDDs in mirror as system/boot disks + 2 SSDs in a mirror for your VM/LXC storage. Keep in mind that SSDs and HDDs are consumables that will always fail sooner or later. Its not the question if they will fail, but when. So it's a great idea to have all disks mirrored or with parity data so you don't have to start from scratch and do all the work again, once a disk fails. Disks often fail when you don't have time to fix stuff and without a proper documentation you probably also can't remember how you set stuff up some years ago.
Benefit of dedicated system and VM storage disks is, that you are more versatile. You could remove the VM/LXC storage disks, replace them with bigger disks and create a new raid without needing to install PVE again, as the system disks will be untouched.

Next best option would be to use two SSDs in a mirror for shared system + VM storage.

Single SSD for system + VM storage will work, but then you shouldn't run anything you rely on and have a proper backups, documentation and a disaster recovery plan.

2 - I've understood the backup of VMs, to put them on one of my HDDs, but in the event that my 1TB SSD in my current configuration fails, I haven't really understood how I can simply restore the Proxmox OS part. Would I have to redo a fresh installation of proxmox, re-partition, and then restore the VMs on my HDD, and get back to my previous configuration?

There is no host backup yet. So yes, you will have to install and set up PVE again.

Is this also valid for hardware changes (CM, processor), or is it transparent for the VMs?

It's usually not a problem to move the disks into a new server unless you use HW raid. You will have to edit some stuff (like the network config because the interface names will probably change) but in general your existing PVE installation should work there.

3- ZFS: From what I understand, it's a software raid thing, but some people use ZFS with a single disk in YouTube tutorials, so I haven't found the reason. In any case, I can't do this with my configuration if I've understood correctly, because I wouldn't be able to make a partition for my temporary data?

Its not only SW raid. Its raid + volume management + filesystem. People like it for its features and data integrity even when using a single disk. Bit rot protection, trasparent blocklevel compression, replication for clustering, encryption, deduplication, ...

4- Some people merge the local-lvm partition with the main local partition. Do you recommend this?

If using the "local-lvm" wouldn't be recommended, it wouldn't be the default.
So no, not recommended unless you got dedicated system disks and don't plan to store any VMs/LXCs on those disks.

 

[RomainD2]

Ideally you would use 4 disks. 2 SSD/HDDs in mirror as system/boot disks + 2 SSDs in a mirror for your VM/LXC storage. Keep in mind that SSDs and HDDs are consumables that will always fail sooner or later. Its not the question if they will fail, but when. So it's a great idea to have all disks mirrored or with parity data so you don't have to start from scratch and do all the work again, once a disk fails. Disks often fail when you don't have time to fix stuff and without a proper documentation you probably also can't remember how you set stuff up some years ago.
Benefit of dedicated system and VM storage disks is, that you are more versatile. You could remove the VM/LXC storage disks, replace them with bigger disks and create a new raid without needing to install PVE again, as the system disks will be untouched.

Next best option would be to use two SSDs in a mirror for shared system + VM storage.

Single SSD for system + VM storage will work, but then you shouldn't run anything you rely on and have a proper backups, documentation and a disaster recovery plan.


There is no host backup yet. So yes, you will have to install and set up PVE again.


It's usually not a problem to move the disks into a new server unless you use HW raid. You will have to edit some stuff (like the network config because the interface names will probably change) but in general your existing PVE installation should work there.


Its not only SW raid. Its raid + volume management + filesystem. People like it for its features and data integrity even when using a single disk. Bit rot protection, trasparent blocklevel compression, replication for clustering, encryption, deduplication, ...


If using the "local-lvm" wouldn't be recommended, it wouldn't be the default.
So no, not recommended unless you got dedicated system disks and don't plan to store any VMs/LXCs on those disks.

Thanks for your clear answer !