ZFS versus RAID+LVM

mir

mir

Famous Member
Apr 14, 2012
3,570
131
133
Copenhagen, Denmark
Hi all,

I am soon going to build a new shared storage solution and for this shared storage I am seriously considering trying ZFS instead of my usual Linux+RAID+LVM. If I choose the ZFS way I will use Freenas.

Anybody here which have done any kind of benchmarks between Linux+RAID+LVM and FreeBSD+ZFS?

Is there a third or fourth way to go instead of the above solutions?
 
I would prefer a ZFS solution.
1) It is a bit faster in some cases but only if you have enough RAM.
2) You can safe a lot of storage with the compression and deduplication features of ZFS. I have ~15 TB server backups on a 6 TB ZFS storage.
3) ZFS is safer than ext3/ext4 because of the checksum features.


Check openindiana + nappit as an alternative to FreeNAS. I do not know how the current Version of Freenas works but i had some performance/stability issues with 8.0 .

How much RAM do yo have and how much storage do you need?
 
JohnD said:
I would prefer a ZFS solution.
1) It is a bit faster in some cases but only if you have enough RAM.
2) You can safe a lot of storage with the compression and deduplication features of ZFS. I have ~15 TB server backups on a 6 TB ZFS storage.
3) ZFS is safer than ext3/ext4 because of the checksum features.


Check openindiana + nappit as an alternative to FreeNAS. I do not know how the current Version of Freenas works but i had some performance/stability issues with 8.0 .

How much RAM do yo have and how much storage do you need?
Click to expand...
I have done some preliminary tests with Freenas 8.3 and so far it works flawlessly. My total unscientific numbers shows so far that given the right configuration a ZFS mirror seems to give 10-15% gains in performance. When my equipment arrives I will make more accurate benchmarks.

The amount of RAM for the storage will be 8GB so I expect to see even greater numbers when prefetch kicks in.

A question for compression: I have read that this is only recommended if you have a powerful CPU and my storage is supposed to be a low power consumption solution so a completely other league than Xeon or Opteron:-).

Thanks for the suggestion of openindiana + nappit. I will look closer into this solution.
 
I saw some very interesting benchmarks here: http://catn.com/vcluster/openindiana-vs-nexentastor-vs-freenas/

The conclusion seems to be that ZFS rocks when used as a storage backend. Of course the system used for benchmarking is way over my budget but the numbers are conclusive:-)

Also in the benchmark the author explains that performance wise OpenIndiana + nappit is comparable to Freenas but Freenas lacked support for xattrs which was the cause the chose OpenIndiana. I wonder if this is still correct since Freenas 8.3 has updated ZFS to version 28 and the benchmark shows it is based on ZFS 15 which indicates Freenas 8.2. So in that case I seem to favor Freenas since I no FreeBSD and OpenBSD a lot but having little or no experience with Solaris. I will however give it a test run because I don't expect Solaris to be that much different than *BSD.
 
You will probably get a better performance with compression.
You should not take an atom cpu. Intel I3 or something like that should be ok.
 
It is hard to say the requirements of minimum CPU. If you have testing platform you can turn compression on and change compression algorithm and make some benchmarks. ZFS support lzjb ( wiki ) and gzip (1 to 9 level). As i know lzjb compress less than gzip so CPU requirements are less.
 
in my experience freenas (from v8.0 up to v8.3) has not been stable for me. random nfs/iscsi lockups etc... on two separate machines on separate switches, so that rules out hardware issues. i guess it's ok in a home setup, but not for 24/7 usage. you mileage may vary tho.

on a sidenote, with zfs you want to have as much ram as possible.
 
BiagioParuolo said:
I'm interested too in a shared solution for vps files and backups. Have you a solution with FreeNAS? have you tried Openfiler?
Click to expand...
I am waiting to get the hardware to do proper tests. I will post back with my results. I have tested OpenFiler, but the project is dead last release is actually a 3.0 beta which dates back to 2011-04-13.
 
Freenas is good but based on FreeBSD 8. nas4free is based on FreeBSD 9 which means support for a lot of newer hardware and the ZFS support seems to be more extensive. nas4free provides support for thin provisioning of volumes in the GUI and there are also some more nifty details. The GUI in Freenas seems more polished but I hate menus open in tabs nas4free avoids this "feature". Rumors on the net also seems to indicate problems with performance and/or stability with Freenas 8.x. But all in all I will wait with my final conclusion.

Ps there is also a Linux/Debian based solution called openmediavault.
 
mir said:
...
Ps there is also a Linux/Debian based solution called openmediavault.
Click to expand...
Hi,
IMHO is openmediavault more for homeuse. But an debian based solution is http://www.openattic.org - run's well (but I haven't tried zfs-stuff).
An advantage of an linux based nas is the good support for 10G-Nics - this was not so good with older freenas-versions. Perhaps it's ok now with the FreeBSD9?!

Udo
 
For Intel's 10 Gb device you can get the fixed version for FreeBSD 9 here: http://files.kramse.org/tmp/xpc-10g/ixgbe-2.4.4-stable.tgz

HTML: 
config:
/etc/sysctl.conf for begge:
hw.intr_storm_threshold=20000
net.inet.tcp.sendspace=1048576
net.inet.tcp.recvspace=1048576
kern.ipc.maxsockbuf=4194304
kern.ipc.nmbjumbop=262144 

/boot/loader.conf for begge:
net.link.ifqmaxlen=1024 
ixgbe_load="YES"
hw.ixgbe.rxd=4096
hw.ixgbe.txd=4096
kern.ipc.nmbclusters="262144"
kern.ipc.nmbjumbop="262144"

/etc/rc.conf:
ifconfig_ix0="inet 10.0.10.20 netmask 255.255.255.0 lro"


HTML: 
Test:
$ sudo iperf -t 60 -i 5 -c 10.0.10.30
------------------------------------------------------------
Client connecting to 10.0.10.30, TCP port 5001TCP window size: 1.00 MByte (default)
------------------------------------------------------------
[  3] local 10.0.10.20 port 33617 connected with 10.0.10.30 port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0- 5.0 sec  5.02 GBytes  8.62 Gbits/sec
[  3]  5.0-10.0 sec  5.47 GBytes  9.40 Gbits/sec
[  3] 10.0-15.0 sec  5.47 GBytes  9.40 Gbits/sec
[  3] 15.0-20.0 sec  5.47 GBytes  9.40 Gbits/sec
[  3] 20.0-25.0 sec  5.47 GBytes  9.40 Gbits/sec
[  3] 25.0-30.0 sec  5.47 GBytes  9.40 Gbits/sec
[  3] 30.0-35.0 sec  5.47 GBytes  9.40 Gbits/sec
[  3] 35.0-40.0 sec  5.47 GBytes  9.40 Gbits/sec
[  3] 40.0-45.0 sec  5.47 GBytes  9.40 Gbits/sec
[  3] 45.0-50.0 sec  5.47 GBytes  9.40 Gbits/sec
[  3] 50.0-55.0 sec  5.47 GBytes  9.40 Gbits/sec
[  3] 55.0-60.0 sec  5.47 GBytes  9.40 Gbits/sec
[  3]  0.0-60.0 sec  65.2 GBytes  9.33 Gbits/sec
 
Last edited:
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom