Hello all,
I built actual a Proxmox Cluster in my private home with 3 MiniPCs which every node have two 256GB SSDs (ZFS Raid1) and a Single 4TB M.2 NVM SSD for data. I have a "old" 2bay Diskstation NAS which I let run with 2 HDDs Raid0 to get 5TB space. This I use over NFS for backup the container and the data if it is possible so I plan to use it as backup.
I want to use the local ZFS Raid1 for installing proxmox itself and the containers and then mount the storage for data. So I plan some containers and a Nextcloud container with all our documents and pictures over the years. So the data is important for me.
I planned to use Ceph with my 4TB SSDs of all 3 nodes with a replication factor of 3 but I have checked this Calculator: ceph-calculator and the notes (It's surprisingly easy to get into trouble. Mainly because the default safety mechanisms (nearfull and full ratios) assume that you are running a cluster with at least 7 nodes. For smaller clusters the defaults are too risky.) and the values which I get there seems that I need maybe a replication factor of 2 but I`m not so happy to have only 2 copies but if the cluster is self healing and running out of storage like described I´m maybe lost if backup not working ;-)
How would you setup your proxmox cluster?
Is it maybe a better way to make every 4TB SSD as Single ZFS Pool and then replicate between each other with some command? Is this possible that a container can then failover and use the local 4TB storage on which node it runs?
So I want 4TB or more useable hard disk. Better more lets say 6TB would be wonderful but 4TB is okay ;-)
Any idea? Some other protocol or keep with Ceph but what about replication factor? Only 2? I have the feeling that Ceph is still the best bet - maybe with https://docs.ceph.com/en/latest/rados/operations/erasure-code/ and LZ4 compression but the replication factor drives me crazy ;-)
Best,
Dominic
I built actual a Proxmox Cluster in my private home with 3 MiniPCs which every node have two 256GB SSDs (ZFS Raid1) and a Single 4TB M.2 NVM SSD for data. I have a "old" 2bay Diskstation NAS which I let run with 2 HDDs Raid0 to get 5TB space. This I use over NFS for backup the container and the data if it is possible so I plan to use it as backup.
I want to use the local ZFS Raid1 for installing proxmox itself and the containers and then mount the storage for data. So I plan some containers and a Nextcloud container with all our documents and pictures over the years. So the data is important for me.
I planned to use Ceph with my 4TB SSDs of all 3 nodes with a replication factor of 3 but I have checked this Calculator: ceph-calculator and the notes (It's surprisingly easy to get into trouble. Mainly because the default safety mechanisms (nearfull and full ratios) assume that you are running a cluster with at least 7 nodes. For smaller clusters the defaults are too risky.) and the values which I get there seems that I need maybe a replication factor of 2 but I`m not so happy to have only 2 copies but if the cluster is self healing and running out of storage like described I´m maybe lost if backup not working ;-)
How would you setup your proxmox cluster?
Is it maybe a better way to make every 4TB SSD as Single ZFS Pool and then replicate between each other with some command? Is this possible that a container can then failover and use the local 4TB storage on which node it runs?
So I want 4TB or more useable hard disk. Better more lets say 6TB would be wonderful but 4TB is okay ;-)
Any idea? Some other protocol or keep with Ceph but what about replication factor? Only 2? I have the feeling that Ceph is still the best bet - maybe with https://docs.ceph.com/en/latest/rados/operations/erasure-code/ and LZ4 compression but the replication factor drives me crazy ;-)
Best,
Dominic