Proxmox and NAS - Possible?

B

bearhntr

Member
Sep 9, 2022
167
13
23
Atlanta, GA USA
I have an old Optiplex 7010 - which was my old Server 2016 Domain Controller. I have made a 2019 (may go to 2022) Proxmox VM on another host.

The old Optiplex 7010 has a MegaRAID 9260-8i controller with 6 HDDs attached in a RAID-5 with HSP. I moved everything that was there I wanted to keep off to a 4TB EXT HDD - and deleted the other stuff, then rebuild the array.

I played putting Proxmox 8.1.3 on the box last night (as it also has a 120GB SSD in it), and another 500GB HDD (which I will probably remove - has started that wonderful 'ticking before death' sound. All of them are seen (see image). If I wanted to put a NAS software on here and maybe 2-3 other VMs, how would I best dedicate the RAID Card and drives to the NAS?

1702393949248.png
 
i'd put the MegaRAID 9260-8i to HBA mode, if possible. otherwise create 6 single RAID0 arrays. Then make it a 6-drive zfs RAID.
 
h00bi said:
i'd put the MegaRAID 9260-8i to HBA mode, if possible. otherwise create 6 single RAID0 arrays. Then make it a 6-drive zfs RAID.
Click to expand...
HBA mode? I am looking through the User Guide for this card, and I see nothing related to this mode. A GOOGLE search sometimes calls HBA "IT Mode". I found this info:

IT mode stands for "initiator target". It presents each drive individually to the host. IR mode is in whatever raid formats the HBA supports.

I also cannot find anything about that in the user guide.

From what I have read, ZFS RAID is not super-reliable and eats RAM like Tic-Tacs. True?
 
bearhntr said:
HBA mode? I am looking through the User Guide for this card, and I see nothing related to this mode. A GOOGLE search sometimes calls HBA "IT Mode". I found this info:

IT mode stands for "initiator target". It presents each drive individually to the host. IR mode is in whatever raid formats the HBA supports.
Click to expand...
Yes, for ZFS you want a dumb HBAwithout any additional abstraction layer directly presenting the disks.
bearhntr said:
From what I have read, ZFS RAID is not super-reliable and eats RAM like Tic-Tacs. True?
Click to expand...
True. HW raid isn't checksumming nor auto-repairing corrupted data. So it can't detect or fix bit rot like ZFS could. Rule of thumb is 1GB RAM per 1TB of raw storage capacity (5x when using deduplication) but if you don'T care that much about performance even something like 0,25-0,5GB per 1TB would work.
 
Dunuin said:
Yes, for ZFS you want a dumb HBAwithout any additional abstraction layer directly presenting the disks.

True. HW raid isn't checksumming nor auto-repairing corrupted data. So it can't detect or fix bit rot like ZFS could. Rule of thumb is 1GB RAM per 1TB of raw storage capacity (5x when using deduplication) but if you don'T care that much about performance even something like 0,25-0,5GB per 1TB would work.
Click to expand...
This machine I want to use is a Desktop Class machine - an old DELL Optiplex 7010 MiniTower. It only has 16GB of RAM. Not a lot if I am going to setup (still cannot figure out which software to use for NAS ... had some strange luck with TrueNAS a couple years back). If I wanted to put more than one VM on here under Proxmox with the NAS (as the machine has a this CPU: 8 x Intel(R) Core(TM) i7-3770 CPU @ 3.40GHz (1 Socket) --- seems like all my RAM would go to the NAS (the computer has 6x2.5" HDDs in one of these: https://global.icydock.com/product_74.html [6-drive bay for 5.25" opening]. Worked great with the LSI 9260-8i controller and Server 2016 to give me 4.5TB HDD space (RAID 5 with a GHSP).

I doing some Googling - and looks like an HBA card which can be used (probably uses the same connectors that the 9260 does so I would not need new cables to split out the 2xSAS ports into 8xSATA connectors - is the LSI 9211-8i (flashed to IT mode). Many of these on eBay. Also appears that the DELL PERC H200 (primarily the same card - just different layout) can do it too, but has some limitations for NON-DELL Certified drives and disabling 'on-drive cache on SATA drives'.

Also - none of these cards seem to have battery backup - never really been an issue, as I have the computer on a UPS...but, I still liked the fact it was there.
 
Dunuin said:
See the ZFS manual why you shouldn't use ZFS with HW raid
Click to expand...
A single drive RAID0 array is basically the same as JBOD. Not to mention that RAID0 is actually not a RAID, it is "striping".
So yes, ZFS on many single drive RAID0 arrays is fine. Using this here. Each RAID0 Array is independant, ZFS is doing alle the magic.


bearhntr said:
IT mode stands for "initiator target". It presents each drive individually to the host.
Click to expand...
correct thats HBA mode.

Is it a retail MegaRAID 9260-8i or is it OEM branded?
If retail, you can flash it to IT mode.

If not, you can bypass by single drive RAID0 / JBOD

What NAS Software are you thinking about?
If NAS is the main use case and VMs are just for fun, maybe check out unraid instead of Proxmox.
 
Last edited:
Neobin said:
Source?
Click to expand...
Just random readings over the years about the problems people have had on a power loss, or if they tried to enlarge or expand a ZFS.

For example, if you have a RAID 5 with three drives and want to add another drive to expand it. It is not that hard to do and you gain added space. Also say you have a RAID 5 (4x1TB HDDs) and you want to replace them all with 4TB HDDs. You can do this (takes long time as you have to replace 1 drive, wait for re-striping, and then do another drive until all 4 have been replaced. Then you 'expand' the ARRAY.

Back when I had my TrueNAS fiasco - I was using it. The NAS box had 4x1TB HDDs and I wanted to do just what I mentioned above - and I could not do it. That box died due to a faulty system board and I could not replace the board (no longer made - and was going to trust a used one). This is why I went to the system I have now with the LSI 9260-8i.
 
h00bi said:
correct thats HBA mode.

Is it a retail MegaRAID 9260-8i or is it OEM branded?
If retail, you can flash it to IT mode.

If not, you can bypass by single drive RAID0 / JBOD
Click to expand...
It is a Retail Card I believe - The boot up screen for the computer that you can access by pressing CTRL+H -- I see this:

1702484231352.png
 
@h00bi

UPDATE:

I used the boot 'webpage' shown above. I deleted the array and GSHP which I had setup. I also went into the CONTROLLER PROPERTIES and set the "Controller BIOS" to [Disabled] and rebooted. I would assume this setting changes the card from a Hardware RAID card to an HBA type card. I saw no other setting

I now no longer see anything related to the drives on attached to that card in Proxmox {see image} --- should I not be seeing them as individual drives here?

1702569881834.png

** Also - what is this "Wearout" option I am seeing? First time noticing that.
 
Additional Update:

I went to the SHELL after setting Controller BIOS to [Disabled] and rebooting. Using 'fdisk -l' and 'lsblk' commands -- still no indication of these 6 HDDs.

I rebooted and [Enabled] the Controller BIOS - and performed same test - no disks are seen.

h00bi said:
Is it a retail MegaRAID 9260-8i or is it OEM branded?
If retail, you can flash it to IT mode.

If not, you can bypass by single drive RAID0 / JBOD

What NAS Software are you thinking about?
If NAS is the main use case and VMs are just for fun, maybe check out unraid instead of Proxmox.
Click to expand...

How and where do I find said 'FLASH' ?

I want to use Proxmox - if I can, and then create a VM to use these 6 drives as a NAS, but I also want to put other VMs on here (If I can). I do not know what "unraid" is - is that something like Proxmox? I guess I will need to GOOGLE it. I already have 1 Proxmox host - and had planned to 'cluster' that one with this one, once I got this working...so that I would able to use one webGUI to see both hosts and all VMs.
 
bearhntr said:
should I not be seeing them as individual drives here?
Click to expand...
No. if you deleted the array, and the controller is NOT in IT Mode (HBA mode), then the drives are just "sitting around" behind the raid controller.
The raid controller does only present virtual drives (arrays) to the system.

bearhntr said:
How and where do I find said 'FLASH' ?
Click to expand...
can't help you on that. i just know this option should exist if it is a non-OEM controller.
You should contact broadbom support on this.

If you cannot flash to controller to IT Mode, you have 3 options:

a) create 5 individual arrays in JBOD or stripe(RAID0) mode, so proxmox will see 5 individual drives.
b) recreate the HW RAID and use that, but make sure the file system you will choose is doing well on HW RAID.
c) replace your RAID Controller with a 6port SATA HBA
 
  • Like
Reactions: bearhntr
h00bi said:
a) create 5 individual arrays in JBOD or stripe(RAID0) mode, so proxmox will see 5 individual drives.
Click to expand...
No, this is not an adequate alternative to an HBA. That's not how an HBA works. With a RAID0 you still have the RAID controller in between.
 
  • Like
Reactions: bearhntr
h00bi said:
No. if you deleted the array, and the controller is NOT in IT Mode (HBA mode), then the drives are just "sitting around" behind the raid controller.
The raid controller does only present virtual drives (arrays) to the system.
Click to expand...

h00bi said:
can't help you on that. i just know this option should exist if it is a non-OEM controller.
You should contact broadbom support on this.
Click to expand...

I have purchased another controller - from other readings the LSI 9211-8i is basically the same controller that I have without the RAID stuff and no BBU. It is supposed to be here this week. I will replace the 9260-8i and see if this resolves everything.

Thanks again for the assist.
 
Thank You all for your assistance.

The LSI 9211-8i arrived today. Was a direct replacement/swap with the LSI 9260-8i - same slot, and SAS to SATA cables plugged right in. Booted up - card initialized. Went into Proxmox, enabled iommu and rebooted.

Created VM for TrueNAS...passed LSI card, and all 6 drives are seen just fine, RAID Z1 built and all good to go.
 
  • Like
Reactions: UdoB
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom