Disk option recommendation

[domingosvarela]

Hello,

I'm installing the new version for the first time on an HP server with 4 300G disks, I want to know the recommended option for using the disks, keep proxmox installed on a single disk and use the rest in poom zsf mode for the vms?
What option do you recommend?
Thanks

Best Regards

 

[Dunuin]

I guess those are 2.5" HDDs? In that case I would try to stripe as much disks as possible to sqeeze out every bit of IOPS performance you can get. So I would create a striped mirror (aka raid10) with those 4 disks and use that for PVE system + VM/LXC storage.

But don't expect great performance when using HDDs. You will primarily need IOPS performance and HDDs are really terrible at that. Have a look at the IO delay. If it gets too high you might want to replace the HDDs with a mirror of proper enterprise grade SSDs with power-loss protection.

 

[domingosvarela]

I guess those are 2.5" HDDs? In that case I would try to stripe as much disks as possible to sqeeze out every bit of IOPS performance you can get. So I would create a striped mirror (aka raid10) with those 4 disks and use that for PVE system + VM/LXC storage.

But don't expect great performance when using HDDs. You will primarily need IOPS performance and HDDs are really terrible at that. Have a look at the IO delay. If it gets too high you might want to replace the HDDs with a mirror of proper enterprise grade SSDs with power-loss protection.

Hi Dunuin,
I have 6 not 4, sorry for mistake!
6x 300GB SAS 10K SFF SC DS HDD in a HPE DL380 Gen10.
I've been reading and found several recommendations that say to use the system and the vms on separate disks, as I don't have experience with proxmox I wanted to hear more opinions.

 

[Dunuin]

Can be useful in some situations, like for example destroying your ZFS pool storing the VMs to recreate it with more/bigger disk without needing to wipe the PVE system while doing that. But you usually want all IOPS performance you can get, when using HDDs and 6 disks in a raid10 wuold give you +50% better IOPS performance than 4 disks in raid10 for VMs and 2 disks in raid1 for the system.
~300 IOPS with a 4 10K HDD raid10
~450 IOPS with a 6 10K HDD raid10
1.5 Million IOPS with a raid1 of decent SSDs.
Just to give a comparison what performance levels you can expect. When your performance is that low, even 150 more IOPS can make a big difference.

 

[domingosvarela]

in short I gain more advantages in performance and redundancy, but I lose in space using the 6 disks in raid10 in relation to the separation of the system and the VMs in two different raids.
@Duniun Thank you very much for the explanation and teachings.

 

[Dunuin]

in short I gain more advantages in performance and redundancy, but I lose in space using the 6 disks in raid10 in relation to the separation of the system and the VMs in two different raids.

No, with 6 disks in raid10 you get 900GB that VMs, PVE system and file storage for ISOs and so on can share.
With a 4 disk raid10 you get 600GB for VMs + 2 disk mirror you get 300GB for PVE system + files + VMs.
So in both cases you get 900GB.

 

[domingosvarela]

I will follow your recommendations.
Raid10 for all disks
Thanks

 

[domingosvarela]

Dunuin,

In that case, if I set up hardware raid using the included raid card with ext4 versus ZFS, would I gain more performance? considering running vms on HDDs!
Thanks

 

[Dunuin]

If that raid card got a BBU and cache, probably yes.

 

[domingosvarela]

based on your experience which of these options is recommended ZFS or Card Raid?
sorry for asking too many questions.

 

[Dunuin]

Depends on your priorities. ZFS is better for data integrity, portability and features. HW raid with BBU+cache will perform better.

 

[domingosvarela]

Many thanks for your help and time!