Partitioning Setup Help

letmehost

New Member
Jul 22, 2021
5
0
1
39
Hello everyone, I'm relatively new to Proxmox and would like some assistance in the partitioning of my 3 drives for use with it. I'm not really sure what route I should take. I plan on running several web servers, some sandbox containers for testing, and a few more web servers that I have customers who pay me to create and manage for them. I will also be running my businesses website from this as well. I would like to contain all of these separately and be able to clone the clients vm's for when/if they should ever decide to seek management elsewhere. My hardware is as follows:
Intel Core i7-3930NIC 1 Gbit
Intel 82541PI
2x HDD SATA 3,0 TB
1x SSD SATA 120 GB
8x RAM 8192 MB DDR3

As I said I am pretty new and have been reading a lot about the different file systems but haven't been able to deduce the best for my situation. Any help would be appreciated.
 
For the VM storage you possibly want some kind of raid1 with two enterprise SSDs. I would suggest a ZFS mirror if you are not forced to use a raid controller.
You really want some kind of mirror if running buissness stuff on that. And HDDs really have problems with all the IOPS that multiple VMs create.
Consumer/prosumer SSDs are a bad idea because you want to run some DB if you are running web servers and these do small sync writes and SSDs without a powerloss protection (basically all consumer/prosumer SSD) got terrible sync write performance and high write amplification because they can't cache without it.

And I would recommend to install a proxmox backup server (best on a second host) so you can do incremental backups of your VMs. For that you want some dedicated disks (SSDs highly recommended because HDDs are very slow because everything will be stored in millions of small chucks because of the deduplication).

And if you want high availability (I would require that as a customer) you should consider getting atleast 3 servers and using ceph. So that if one server goes down because of maintaince or failures the running VMs will be automatically migrated to the other running hosts without any (or atleast only a short) downtime.
 
Last edited:
I will take all of that into consideration and thanks for the reply. Right now my customers sit comfortable on a Contabo server located in the US. The reason they chose me to host is simple though. They're my friends lol. But they (only 3 of them) get a good deal from me and they can actually come talk to me face-to-face about any issues. I would say more of a technical liaison for their small-business. They throw me a few $100 a month and I build their website, host it, update it, and maintain it. That being said, I will most definitely be looking into all of the prosumer options from hetzner (more than probably) in the near future, but, I still would like to know a good partitioning scheme for the moment using the hardware I have. partition /dev/sda --> lvm --> vg0 ? should I use the 2x3TB hdd's for iso images and storage of snapshots? Or should i install proxmox on one and use the ssd for VM's or for swap? Thanks in advance guys/gals!
 
I will take all of that into consideration and thanks for the reply. Right now my customers sit comfortable on a Contabo server located in the US. The reason they chose me to host is simple though. They're my friends lol. But they (only 3 of them) get a good deal from me and they can actually come talk to me face-to-face about any issues. I would say more of a technical liaison for their small-business. They throw me a few $100 a month and I build their website, host it, update it, and maintain it. That being said, I will most definitely be looking into all of the prosumer options from hetzner (more than probably) in the near future, but, I still would like to know a good partitioning scheme for the moment using the hardware I have. partition /dev/sda --> lvm --> vg0 ? should I use the 2x3TB hdd's for iso images and storage of snapshots? Or should i install proxmox on one and use the ssd for VM's or for swap? Thanks in advance guys/gals!
Like I already said, I would alteast use raid1 everywhere. If your system disk fails every configuration is lost, every VM is offline and you need to freshly install your OS. It takes hours or days to setup everything again with all the firewall rules, setup of the mail server, setup of monitoring tools, .... A buisness website that is down for hours or days is bad for the reputation and for the buisness loosing customers.
Same for the VM storage. It isn't that bad if you create daily backups but you still loose all data created that day and it takes some time so restore all VMs from backups. I don't know if you are running some webshops or something like that but loosing a day of data and so all webshop orders, is really bad. People paid stuff and you will never deliver it because you lost the orders.
Use raid1 for redundancy and any disk may fail without anyone being able to notice it (but you hopefully notice it because you use some kind of monitoring tool like zabbix, grafana or whatever).
And if you aren't willing to use raid1 I would think you are also not prepared and already have bought a spare disk that is laying around so you could directly replace it if it fails? In that case you need to buy one first so the replacement will take even longer...
And if you already got a spare one, put it in and use it as a raid1. Sooner or later a drive will fail and that way you atleast don't loose any data.

And even if none of your drives may totally fail your data still might silently degrade over time. Google for "bit rot" to understand what I mean. Write something to a SSD or HDD, wait 1 year and read it again. The data won't be 100% the same because bits will flip over time. Against that it is usefull to use a advanced filesystem like ZFS that is checksumming all data and will compare all data to the checksums on a monthly schedule. That way it can find rotten bits. If you now also got some parity data (like using a raid1) it can automatically repair those rotten bits and repair your data. If you don't got a raid1 it will tell you what files are corrupted but it can't do anything to repair them.
So again a good point why to use drive redundancy. And backups won't help you against bit rot, if you for example are using LVM, because you don't know if a file is corrupted or not when you are backing it up. So you might think everything is fine but you overwrite your old backups with new backups and your healthy file is replaced with a corrupted one.

And you can't store snapshots on another disk. Its always part of the same disk that you want to snapshot. So the HDD would only be usefull for backups/isos. And snapshots aren't really helping because if you loose a disk without raid1 you also loose all your snapshots so you only got your backups and these are not the newest because backups can take hours if you don't use PBS with SSDs so you aren'T creating them every hour or every several minutes.
 
Last edited:
Ok so, I will setup all 3 of the drives for RAID 1? I am not forced to use a raid controller, however, the data loss is obviously not an option. But I will lose 5 TB of data availability right? I thought you could only RAID the amount of your smallest drive? Or is it possible to RAID the amount of the smallest drive and use the leftover on the other 2 for whatever? I bet you can... I'm gonna google search that for a little while. Also, if I can allocate the rest to whatever I want, does that mean ZFS is an option?
 
With ZFS you can use partitions as mirrors so it theoretically would be possible to create three 120GB partitions and create a three-way-mirror with them and use the rest for something else...but that really isn't a good idea. Your SSD would be slowed down to HDD performance.
I don't see why you can't just buy another SSD. You get a new 120GB consumer SSD for just 21$ and they pay you each 100$ a month. If your drive dies and you loose everything you will be angry that you didn't just spend 21$ for a second drive.
But if you plan to use ZFS I would recommend to buy two identical enterprise grade SSDs. NEw they are expensive but second hand they are cheap too. I got a pair of second hand 100GB enterprise SSD that are 30 times more durable and way faster compared to a consumer SSD for just 24$.
So best would be to use two SSDs as a mirror for proxmox as system drives and as a VM storage (for example 2x 120GB where 108GB would be usable) and the both 3TB HDDs as another mirror for backups, as NAS and so on (where 2,7TB would be usable).
And if you don't got space for 4 drives...remove all 3 drives and just get two big SSDs. I wouldn't want to run a webserver on slow HDDs in 2021. Look in the Proxmox GUI at your nodes IO delay. If it over 5 or 10% your HDDs are slowing down everything and the CPU needs to wait for the HDDs to continue.
 
Last edited:

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!