Support to create RAID1 by SW to 2x SATA SSD

[jader]

Hi
I've bought a new HP DL20 gen10+ server with 32GB RAM + 2x SATA SSD 1TB to OS+ISOs. VMs will be on another pair of disks.
I'd like to create a RAID1 by software on PVE host, so the PVE install is protected by RAID.
How can I do it ?
I imagine at boot time to do a CTRL+ALT+4 and run several commands to download extra piece of software, create RAID by hand running mdadm or something like that and returning to GUI interface to choose right disk (SW RAID) as target to install .
Just don't know what command I should to use or if I'm doing wrong assumptions!

Please enlight me so I can move forward and get my new server running ASAP!

Regards,

Jáder

 

[Dunuin]

PVE uses ZFS (stable) or btrfs (experimental) as software raid1.
If you want mdadm raid1 you could use the debian 11 installer and then turn that Debian into a PVE7: https://pve.proxmox.com/wiki/Install_Proxmox_VE_on_Debian_11_Bullseye

 

[jader]

PVE uses ZFS (stable) or btrfs (experimental) as software raid1.
If you want mdadm raid1 you could use the debian 11 installer and then turn that Debian into a PVE7: https://pve.proxmox.com/wiki/Install_Proxmox_VE_on_Debian_11_Bullseye

How I can use ZFS (or even BTRFS) for it. I started the installation and it show 2 SSDs to choose where to install.
I'm lost about how to use the internal software RAID solution.
BTW: I have 32GB RAM, and 1TB disk. How much will ZFS cost in RAM ?

 

[Dunuin]

How I can use ZFS (or even BTRFS) for it. I started the installation and it show 2 SSDs to choose where to install.

You select it in the installer (see 7, 8 and 9):

BTW: I have 32GB RAM, and 1TB disk. How much will ZFS cost in RAM ?

Depends. By default up to 50% so 16GB. But with just 2 1TB disks you can probably lower that so something like 2-4GB. But the more RAM you allow it to use, the better the performance.

 

[jader]

You select it in the installer (see 7, 8 and 9):

Depends. By default up to 50% so 16GB. But with just 2 1TB disks you can probably lower that so something like 2-4GB. But the more RAM you allow it to use, the better the performance.

I could sware I press options button and see nothing there... but thanks I'll check it again and update you.
fs
About ZFS and 8GB RAM, I think it's a lot so:
1) what are the problems with BTRFS ? I'm 4.000 km away from server, so I'd hate to have problems!
2) I imagine after choose it if I decide to go back and change it I've to start over and recreate RAID1 from zero loosing all data in devices, so it's a major decision.

3) can I start ZFS with 4GB RAM CACHE and later decide to go up or viceversa ? How to find out the correct amount of RAM to alocate to ZFS ?

 

[Dunuin]

what are the problems with BTRFS ?

It's only been added recently and is more like an experimental feature. For more see for example the discussion here: https://forum.proxmox.com/threads/btrfs-vs-zfs-on-raid1-root-partition.124281/

can I start ZFS with 4GB RAM CACHE and later decide to go up or viceversa ?

Yes, can be changed later. You just need to rebuild the initramfs and reboot in order to apply changes.

How to find out the correct amount of RAM to alocate to ZFS ?

I would start with a lot of ARC and then lower it step by step. While doing that have compare the output of arc_summary. If your "hit rates" drop too low or your dnode/metadata cache is running out of space, you know that you reduced the ARC too much.

 

[jader]

I have installed ZFS1 with two SSD SATA 960GB... thanks for your help.
I'm needing help to see ARC_summary and all other info about ZFS.
I don't know how to see even size of cache!

 

[Dunuin]

Just run arc_summary in the CLI. "ARC size (current)" is the current size of the ARC

 

[jader]

got it:

ARC size (current): 38.9 % 6.1 GiB
Target size (adaptive): 39.6 % 6.2 GiB
Min size (hard limit): 6.2 % 1000.3 MiB
Max size (high water): 16:1 15.6 GiB

With NO USAGE at all I'm using 6GB of max 16GB. How/where I can change that ?
Please send me detailed instructions, this server is not production but I've spent a lot of time to configure it till now!

 

[Dunuin]

got it:

ARC size (current): 38.9 % 6.1 GiB
Target size (adaptive): 39.6 % 6.2 GiB
Min size (hard limit): 6.2 % 1000.3 MiB
Max size (high water): 16:1 15.6 GiB

With NO USAGE at all I'm using 6GB of max 16GB. How/where I can change that ?
Please send me detailed instructions, this server is not production but I've spent a lot of time to configure it till now!

It will grow and shrink between 1000.3 MiB and 15.6 GiB depending on how much RAM is available. The more RAM you allow ZFS to use, the faster your pool will be. So limiting the RAM is usually not desired, unless the OOM killer kills your VMs because ZFS can't shrink the ARC fast enough. See here for how to lower it: https://pve.proxmox.com/wiki/ZFS_on_Linux#sysadmin_zfs_limit_memory_usage

 

[_gabriel]

and then final question :
what ssd model ?
because zfs blast ssd if are not datacenter
zfs require enterprise grade disks

edit : another way that i use on one system, is zfs miror for pve os itself (partitions) , then two lvmthin datastore on partition of each disk for VM data. if one ssd crash, restore from backup of daily pbs backup.

 

[jader]

I'm using Kingston KINGSTON_SEDC500R960G.
I'm a newbie on PVE so not sure about to change from ZFS to BTRFS (I'm 4.000km away from server) or start over (is not production yet).
On Smart of disk I see low values (I dont know how to copy it from GUI to show you ).

 

[_gabriel]

ok you've got not fired
I use Kingston DC500 too, but no ZFS with DC500R because I'm not sure in its endurance
Of course it depends your usage , you will need to monitor SMART which works great.
DC500R 960GB has 876 TBW (0.5 DWPD/5yrs)
DC500M 960GB has 2278 TBW (1.3 DWPD/5yrs)
ZFS mirror with DC500M for PVE OS but only 1 critical VM on the ZFS mirror. I have create another lvmthin datastore on same disks for other VMs where daily backups are sufficient.
(sorry for my english ...)