ZFS root fs and special device for separate hdd

[Gregor]

Hello,

having this hardware setup for Proxmox VE:

• 32GB RAM
• 2 x 500GB SSD
• 2 x 6TB HDD

I want PVE to be installed on a SSD RAID1 with ZFS. LXC system files shall be stored also on that SSD RAID1. LXC data shall be stored on a HDD RAID1 with ZFS. To make use of the benefits of special devices I want around 32GB (too much?) from the SSD RAID1 to be configured as the special device for the HDD RAID1.

Can you tell me how to manage this?

As I am installing a new system, it would be great to enable ZFS encryption as a mitigation of the risk that my harddware is stolen. Is it possible to configure both RAID1 in a way, that the password is only asked when the whole PVE server gets rebooted? Are there any concerns using PVE with ZFS encryption?

Thanks for you help.

Best regards
Gregor

 

[guletz]

I want PVE to be installed on a SSD RAID1 with ZFS. LXC system files shall be stored also on that SSD RAID1. LXC data shall be stored on a HDD RAID1 with ZFS. To make use of the benefits of special devices I want around 32GB (too much?) from the SSD RAID1 to be configured as the special device for the HDD RAID1.

Hi,

So, on the same SSDs, you want to:
- install PMX/zfs
- install some CT's
- use 32GB/SSD as SD

I do not recomend this(many eggs in the same basket for my taste). In the case when you lose both SSD, then you lose ALL of your data(zfs install, CTs, and also the HDD pool). If yours SSDs are not "enetrprise class", then is a very good probability to lose ALL of your valuable data.

If I will be in your case, I will do at minimum but more safe, like this:

- at least 3 SSDs(enterprise class)
- 1st SSD with only for PMX install
- 2nd and 3rd SSD as Special Devices, or only for some CTs(not both as the same time)
- one SSD(2 or 3) I will do 1-2 TB write before use, so not both SSD wil be broken in the same time(probabilistic)
- backup for any CTs to the HDD pool

Good luck / Bafta!

 

[Gregor]

I do not recomend this(many eggs in the same basket for my taste). In the case when you lose both SSD, then you lose ALL of your data(zfs install, CTs, and also the HDD pool). If yours SSDs are not "enetrprise class", then is a very good probability to lose ALL of your valuable data.

If I will be in your case, I will do at minimum but more safe, like this:

- at least 3 SSDs(enterprise class)
- 1st SSD with only for PMX install
- 2nd and 3rd SSD as Special Devices, or only for some CTs(not both as the same time)
- one SSD(2 or 3) I will do 1-2 TB write before use, so not both SSD wil be broken in the same time(probabilistic)
- backup for any CTs to the HDD pool

Good luck / Bafta!

Hi,
thanks for your reply. I am aware that if both ssd fail, my system fails. This is why I want them to be in a raid1. You rate the probability of both SSDs failing at the same time still too high?
The setup you proposed will not work. I only have 4 SATA ports. Also, the HDDs are used for LXC data, too, so I don't want backups to be on there. Last, I already own the mentioned hardware.
What would be a valid setup with the hardware mentioned?

Br
Gregor