System on SSD and only one rpool on HDD (ZFS Special Device)

[Proxmoxuser41]

The following components were chosen:
2x 250 GB for system (RAID1)
3x 5TB HDD for rpool (Raid-Z1)

During the installation, we chose HD Size: 80GB of the SSD. the remaining 170GB are later for the "ZFS Special Device".

After the installation we made a Raid-Z1 (rpool2) out of the 3x5TB HDD.

The problem is we now have two rpools, but we only want one. is it somehow possible to exchange the default rpool with the rpool from the HDD's (rpool2)?

 

[floh8]

what is the problem with 2 pools? one for the system and an other for the vms or data. if u build one pool out of different disks than u loose speed or/and disk space.

 

[Dunuin]

Jup, two pools make more sense. You could just use one pool when using special devices and special_small_blocks. Lets say you set special_small_blocks to 64k, then everything using a 64k+ volblocksize will be stored on the HDD and everything using a 32K- volblocksize will be stored on the SSD. But thats not really that practical as you will be limited to what volblocksize your guests can use.
More veratile and easier to manage if you got two pools and two ZFS storages and you then decide for each VM which virtual disk should be stored on the SSDs or HDDs by choosing the right storage.

 

[LnxBil]

Lets say you set special_small_blocks to 64k, then everything using a 64k+ volblocksize will be stored on the HDD and everything using a 32K- volblocksize will be stored on the SSD. But thats not really that practical as you will be limited to what volblocksize your guests can use.

The easiest method to do so is to have such a setup:

pool
 + PVE            (special_small_blocks 128K)
 + proxmox-ssd    (special_small_blocks 128K)
 + proxmox        (no special setting, this will go to the hdd)

And add a second ZFS storage proxmox-ssd with that name and set a low quota so that you won't write your special device full by mistake.

In order to install with the default installer, you need to install on the HDDs first and then add the mirrored special device. Afterwards you need to set special_small_blocks on PVE and send/receive your ROOT dateset in order to move it's data to the SSD (if you want). Best do to in a rescue environment.