Zfs over RAID10 HW... wrong idea?

S

Sebastian2000

Member
Oct 31, 2017
80
1
8
43
Hello

I know that the better way to use ZFS is with no HW RAID, but I have an C6220 with lsi 9265 raid card. It's a bit difficult to flash it to get IT-MODE so I have build and RAID10 (with 6 SSD kingston DC) and install proxmox in ZFS 0 over the virtual disk.
I was creating the nagio RAID monitoring, and to test it, I have put offline an disk in setup, and then online. When I put online again the disk, server run to rescue mode....

It's really better to go to ext4/lvm in my case? ZFS is not good idea in raid hardware?
 
Zfs strength is the fact that it monitors drives itself. You've just introduced another factor that could fail.

Also if the raid card would die you will need to buy the same card or your data is lost.

In pure zfs or zfs with an it card you could just got another card and you will be up and running.
 
belrpr said:
Zfs strength is the fact that it monitors drives itself. You've just introduced another factor that could fail.

Also if the raid card would die you will need to buy the same card or your data is lost.

In pure zfs or zfs with an it card you could just got another card and you will be up and running.
Click to expand...

I know it, but my situation is this one... I have this RAID card and have to use it to have access to disk, change card can't be an solution for now because need to use yet the server....
I have try the same test in an RAID10 with ZFS and ext4/lvm, perf seem a little better en ZFS, but if an harddrive failure can cause the filesystem corruption with ZFS over RAID HW, I have no other choice to go to ext4...
 
Well, really I do an mistake as I put offline an disk, and then, not do an rebuilt but an simple "online" when the correct option would be "rebuild". But this have brake our ZFS filesystem (it was an test server for now). It's also possible that can break ext4?
 
Sebastian2000 said:
Hello

I know that the better way to use ZFS is with no HW RAID, but I have an C6220 with lsi 9265 raid card. It's a bit difficult to flash it to get IT-MODE so I have build and RAID10 (with 6 SSD kingston DC) and install proxmox in ZFS 0 over the virtual disk.
I was creating the nagio RAID monitoring, and to test it, I have put offline an disk in setup, and then online. When I put online again the disk, server run to rescue mode....

It's really better to go to ext4/lvm in my case? ZFS is not good idea in raid hardware?
Click to expand...

If the silence corruption occurs in HW Raid it will be not detected, but ZFS will detect it and your VM disk will be corrupted with no rescue ability.

If I`m not wrong HW Raid system protect from HDD malfunction not others problems.
 
Sebastian2000 said:
I know it, but my situation is this one... I have this RAID card and have to use it to have access to disk, change card can't be an solution for now because need to use yet the server....
I have try the same test in an RAID10 with ZFS and ext4/lvm, perf seem a little better en ZFS, but if an harddrive failure can cause the filesystem corruption with ZFS over RAID HW, I have no other choice to go to ext4...
Click to expand...

The advice is not to do it, everyone will tell you that. If you are trying to convince yourself it's ok to do it, than go to it. When it breaks, you will own the pieces. Sorry to be blunt, but that's the way it is.
 
rpotter28 said:
The advice is not to do it, everyone will tell you that. If you are trying to convince yourself it's ok to do it, than go to it. When it breaks, you will own the pieces. Sorry to be blunt, but that's the way it is.
Click to expand...

I understand that you talk about RAID HW and ZFS...
Well, there is not only one voice about this, for this I wrote this post, I am no trying to convince myself about nothing, only post an question to recieve answers .

I finally decide to reinstall my nodes in ext4/LVM as I can't use RAID software and had to use my LSI raid card... It's probably that ZFS would be better way in general, but in my case with HW raid, I suppose that this seem the better setup.
 
Sebastian2000 said:
Hello

I know that the better way to use ZFS is with no HW RAID, but I have an C6220 with lsi 9265 raid card. It's a bit difficult to flash it to get IT-MODE so I have build and RAID10 (with 6 SSD kingston DC) and install proxmox in ZFS 0 over the virtual disk.
I was creating the nagio RAID monitoring, and to test it, I have put offline an disk in setup, and then online. When I put online again the disk, server run to rescue mode....

It's really better to go to ext4/lvm in my case? ZFS is not good idea in raid hardware?
Click to expand...

Create 1 disk RAID0, what basically the same as HBA mode.
 
belrpr said:
Zfs strength is the fact that it monitors drives itself. You've just introduced another factor that could fail.

Also if the raid card would die you will need to buy the same card or your data is lost.

In pure zfs or zfs with an it card you could just got another card and you will be up and running.
Click to expand...

You don't need to buy the same raid card, its enough to buy card from the same factory, or in this case, any card with LSI chipset. They will recognise raid configuration and load it from disks.
 
Hello,
I'm using some zfs with hw raid10 on Dell and HP servers with no problem. I made a big volume and install zfs on a single virtual disk (raid0)
If you have BBU to protect your cache, I think it's secure as all other filesystems.
 
dendi said:
Hello,
I'm using some zfs with hw raid10 on Dell and HP servers with no problem. I made a big volume and install zfs on a single virtual disk (raid0)
If you have BBU to protect your cache, I think it's secure as all other filesystems.
Click to expand...

It was what I do at first time, but when see the failure (with my bad action of offline/online in RAID setup, when it would be offline/rebuild action), I decide to go back with ZFS and still with LVM. As I see, the perf are a few worst, but still very good for us.
@janos, also read that RAID0 RAID HARDWARE is not an valid/good option in this case.
 
So we can resume it on :
- Or use ZFS RAID software (perhaps the better option if it's possible)
- Or use LVM if RAID hardware...

ZFS on RAID HW seem have some risk for data...
 
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom
Back