[SOLVED] Upgrading a ZFS cluster

[LunarMagic]

Hello,

I have 3 basically identical systems I’m having put together in a cluster. 2 out of the 3 have the same amount of storage but 1 of them has only half the amount. I want to make everything zfs and I am using intel or micron enterprise ssds. With zfs would I be able to slowly upgrade the smaller server to be as big as the larger one over time or will I have to rebuild the zfs raid with all new bigger drives at once?

Thank you

 

[UdoB]

With zfs would I be able to slowly upgrade the smaller server to be as big as the larger one over time

Yes.
You can either change one old/small drive with a new/larger one or add new disks. Both approaches really need(!) to follow some rules as there are some pitfalls. Both handle "vdevs", not "disks".

Post the output of the current state (zpool status) and tell us what you want to change and someone will recommend the right way.

or will I have to rebuild the zfs raid with all new bigger drives at once?

No. The only situation where a "rebuild from scratch" is necessary is when you want to change the vdev topology. Example: go for RaidZ2 when you have Mirrors.

 

[LunarMagic]

Yes.
You can either change one old/small drive with a new/larger one or add new disks. Both approaches really need(!) to follow some rules as there are some pitfalls. Both handle "vdevs", not "disks".

Post the output of the current state (zpool status) and tell us what you want to change and someone will recommend the right way.


No. The only situation where a "rebuild from scratch" is necessary is when you want to change the vdev topology. Example: go for RaidZ2 when you have Mirrors.

Oh that’s awesome! So I have 16 1.6TB drives in RaidZ2 and I want to upgrade a few to 3.2TBs. I just can’t do a lot at once because drives are really expensive.

So when I switch out 1 drive will I just be able to air it to the vdev and it will sync it in? Also will the additional space add to the total vdev capacity or will that only happen after I upgrade X number of drives?

I’m used to hardware raids where it doesn’t change capacity until all the drives are changed so I just want to make sure I understand correctly . Thank you!

 

[UdoB]

Oh that’s awesome! So I have 16 1.6TB drives in RaidZ2 and I want to upgrade a few to 3.2TBs. I just can’t do a lot at once because drives are really expensive.

So when I switch out 1 drive will I just be able to air it to the vdev and it will sync it in? Also will the additional space add to the total vdev capacity or will that only happen after I upgrade X number of drives?

That's one of the rules: all disk in a vdev (not: in a pool) must (should) be equal. The smallest one decides the capacity of that vdev.

If you have three RaidZ2 vdevs you can just upgrade one of them. But all x drives in one vdev must be replaced to gain storage space.

And be careful: change one disk and wait for the resilvering to finish! This is crucial. Only after disk 1 is "finished" replace disk number two.

Best regards

 

[LunarMagic]

That's one of the rules: all disk in a vdev (not: in a pool) must (should) be equal. The smallest one decides the capacity of that vdev.

If you have three RaidZ2 vdevs you can just upgrade one of them. But all x drives in one vdev must be replaced to gain storage space.

And be careful: change one disk and wait for the resilvering to finish! This is crucial. Only after disk 1 is "finished" replace disk number two.

Best regards

Thank you so much for your help . So I have all 16 drives in a RAIDZ2 vdev so that means i can upgrade the drives slowly but i won't see any space increases until all of them are the higher drive size. When I do this as long as I do it slowly and make sure the vdev syncs I can upgrade and not lose any data as long as I give it time to resync the raid. Is this correct?

 

[UdoB]

So I have all 16 drives in a RAIDZ2 vdev so that means i can upgrade the drives slowly but i won't see any space increases until all of them are the higher drive size. When I do this as long as I do it slowly and make sure the vdev syncs I can upgrade and not lose any data as long as I give it time to resync the raid. Is this correct?

Yes, correct

 

[LunarMagic]

Yes, correct

Thank you for the help and the swift reply . I hope you have a great day!!!