BTRFS

Maher Khalil

Member
Jul 11, 2021
113
5
18
40
I have 2 NVMe disks, 4 SSD disks and 4 HDD disks
I want to use all of them in ZFS or BTRFS pool but I am afraid that NVMe and SSD speed will limit to match HDD speed
I need advice on that
 

LnxBil

Famous Member
Feb 21, 2015
5,621
651
133
Germany
Yeah, using all of those in one pool might NOT be the best choice.

A pool of spinning rust can always be enhanced by using SSD for metadata caching and SLOG (SLOG even on NVMe will be blazing fast, but only needs a few GB). So, you have a lot of other stuff laying around. What are the disk sizes of each?
 

LnxBil

Famous Member
Feb 21, 2015
5,621
651
133
Germany
2 x 250 GB NVMe I want to use for OS
PVE needs at most 10GB and no speed after it is booted, what about the rest?

One setup could be this setup that uses all block devices and creates two nvme-boosted pools:
Split each NVMes in two 5 GB partitions and in two 120 GB partitions, then build two pools with mirrored slog and mirrored special allocation class for metadata. The data itself is then on the SSDs (pool1) and on the stripped mirror SAS hdds (pool2). If you want to use the SSDs for the PVE install, you can just install on a mirrored pool on the two SSDs, partition the two NVMes and then add the partitions.

Is this clear what the idea is?
 

LnxBil

Famous Member
Feb 21, 2015
5,621
651
133
Germany
Please note that the HDD type is SAS 12 Gbps which I think very close to SAS SSD 12Gbps

In sequential IO it may be in the same ballpark, but still slower. In random IO performance, there is no comparison. A disk has at most 200 IOPS, a SSD over 100k.
 

Maher Khalil

Member
Jul 11, 2021
113
5
18
40
So, In this case it may be better to do RAID 0 FOR ssd and use HDD for backups
please note I plan for future expansion of ssd drives
 
Last edited:

djsami

Active Member
Aug 24, 2009
133
2
38
RAID0 has no right to exist in a production system.
sata sas you are right
but ssd no ssd risk dying is very difficult
My 5 year old device is 24x1 tb ssd available raid 0 still works
there was not the slightest problem, but of course we use proxmox backup, at the same time we have a total of 1200 vms.
 

Maher Khalil

Member
Jul 11, 2021
113
5
18
40
sata sas you are right
but ssd no ssd risk dying is very difficult
My 5 year old device is 24x1 tb ssd available raid 0 still works
there was not the slightest problem, but of course we use proxmox backup, at the same time we have a total of 1200 vms.
Which file system do you use?
 

djsami

Active Member
Aug 24, 2009
133
2
38
2 x 250 GB NVMe I want to use for OS Raid 0 + BTRFS
2 x 1.6 TB SSD ZFS 0
4 x 2.4 TB SAS HDD ZFS 0

Okey
 

Maher Khalil

Member
Jul 11, 2021
113
5
18
40
2 x 250 GB NVMe I want to use for OS Raid 0 + BTRFS
2 x 1.6 TB SSD ZFS 0
4 x 2.4 TB SAS HDD ZFS 0

Okey
So, I will use HDD as backups for SSD right?
also for2 x 250 GB NVMe , I will use raid controller then I can get benefits of it, right?
 
Last edited:
  • Like
Reactions: djsami

LnxBil

Famous Member
Feb 21, 2015
5,621
651
133
Germany
2 x 250 GB NVMe I want to use for OS Raid 0 + BTRFS
2 x 1.6 TB SSD ZFS 0
4 x 2.4 TB SAS HDD ZFS 0

Okey

Besides the RAID 0 stupidity (seriously!), why use RAID and then BRTFS? BTRFS has built-in raid functionality (mirroring, striping and more).
 
  • Like
Reactions: tjk

Maher Khalil

Member
Jul 11, 2021
113
5
18
40
Besides the RAID 0 stupidity (seriously!), why use RAID and then BRTFS? BTRFS has built-in raid functionality (mirroring, striping and more)
because of different sizes and types of disks, I am afraid of BTRFS lower speed to speed of HDD so what I am thinking is to use raid controller for only operating system with ext4. And then use SSD for BTRFS or zfs and then HDD for BTRFs.
Do you think using BTRFS on different disk types is ok?
 

LnxBil

Famous Member
Feb 21, 2015
5,621
651
133
Germany
Mixing different disk types for the same kind of data is always not good. You have to mix it with respect to the type of data stored on them - as I described in post #4.

We have not talked about what the goal of your machine is. Normally, goals come first, then you buy the hardware, so why did you buy three different kinds of storage? Nowadays you normally don't buy HDD if you already have NVMe in there.
 

Maher Khalil

Member
Jul 11, 2021
113
5
18
40
You are right, but I bought it new assembled with excellent price.
I want to for client to buy c/m and use it as they want
They may use it for storage or databases or stream …etc
Nowadays dell gives option to separate operating system on NVMe with raid they call it boss and other disks with or without raid
In my opinion because btrfs work on block levels it will not matter mixing disk types each block will have its own speed
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get your own in 60 seconds.

Buy now!