Omnios: zfs and iscsi

[mir]

Hi all,

FYI. Omnios (latest stable) seems to have fixed the zfs and iscsi problems found in freenas, nas4free and openindiana. Even with ZIL in sync mode I have not been able to provoke any iscsi errors yet. And since latest napp-it has official support for omnios (the developer is actively migrating all his current openindiana systems to omnios) we seem to have found a winner for NAS/SAN in the production server room

Official guide can be found here: http://www.napp-it.org/downloads/omnios_en.html

Storage migrating a 4 GB disk

Hardware:

Hardware via prtdiag, Memory via prtconf:

Memory size: 2038 Megabytes

System Configuration: 
BIOS Configuration: Intel Corp. LF94510J.86A.0278.2010.0414.2000 04/14/2010

==== Processor Sockets ====================================
Version                             Location Tag
-------------------------------- --------------------------
Intel(R) Atom(TM) CPU  330   @ 1.60GHz U1PR

==== Memory Device Sockets ================================

Type       Status Set Device Locator  Bank Locator
----------- ------    ---  ------------------- ----------------
DDR2      in use  0    J1MY                 CHAN A DIMM 0

==== On-Board Devices =====================================
Intel(R) Extreme Graphics 3 Controller
Realtek RTL8102E Ethernet Device
Intel(R) High Definition Audio Device

Disks: SATA 2 but MB only supports SATA 1
nic: gigabit

time ./migrate.pl -d omv_lvm:vm-124-disk-2 -s omnios_lvm
Storage migrate type: lvm2lvm
Create target...
Logical volume "vm-124-disk-2" changed
Logical volume "vm-124-disk-2" created
Migrate source...
(100.00/100%)
Update vm config...
Remove source...
Logical volume "vm-124-disk-2" changed
Logical volume "vm-124-disk-2" successfully removed


real 1m31.568s
user 0m0.432s
sys 0m11.324s

 

[mo_]

I assume this thing isnt built on solaris 11 and therefor doesnt have a recent zpool version (>28)?

 

[mir]

It's build on illumos (http://wiki.illumos.org/display/illumos/illumos+Home).
Current version of zpool is 28. zpool > 28 is closed sourced by Oracle which means any solaris > 10 is incompatible with zfs in any OSS os.

What does zpool > 28 provide that you need?

 

[mo_]

mainly the included encryption. the 1MB blocksize is also a nice bonus.
problem is that you really cant just stick any encryption on top of zfs since it conflicts with dedup, checksums in general, maybe even CoW.

 

[mir]

well, basically I see no need for encryption on a day to day basis at the file system level in a SAN/NAS. This must be handled else where. The 1MB block is nice but does it actually matter performance wise? Is there any benchmarks showing real gains using 1MB blocks?

 

[mo_]

tbh I havent bothered checking for benchmarks - like I said, its a bonus, nothing I would base my choice of NAS OS on. The actual key aspect is the encryption in my book for two reasons:
a) its generally a good idea to encrypt anything and everything (maybe apart from linux/open source download mirrors)
b) easier disk switching in case of failure. now, the disks in your storage WILL die, its only a matter of when and thus will have to be replaced. Erasing data on a faulty/dead disk can range from annoying via difficult to impossible. Having encrypted storage means you can just give your faulty disks away for replacement without having to worry about giving away actual data. This is a huge time saver (compared to tiny cpu overhead that encryption introduces)

 

[mir]

ad a) if you a paranoid, then maybe. but bare in mind though that if your data is worth something to someone else then they will get the data encrypted or not.
ad b) This is not an issue at my work since we are by law forced to destroy hardware containing data so all our disks are exposed to magnetism before throwing it in the bin.

 

[acw-flo]

Hi all,

FYI. Omnios (latest stable) seems to have fixed the zfs and iscsi problems found in freenas, nas4free and openindiana. Even with ZIL in sync mode I have not been able to provoke any iscsi errors yet.

Hi,

we use OI with iSCSI, but we do not have any problems with this! Never heard of such problems! What problems did you have?

@all,
Oracle bought Sun and created some new features for zfs with versions over 28. Unfortunatly, oracle will not make the source code for versions over 28 public.
To overcome those version-problems, Illumos invented a new version scheme. It starts with version 5000 and all new features are released as feature-flags. All feature-flags could be individually enabled. Encryption will come! The newest feature-flag is a new compression algorithm with 50% speed improvement and it could be enabled for l2arc, too.

 

[mo_]

re b) wish we had that. over here you can buy second hand office-class printers off ebay and have ALL the documents ever printed/scanned/faxed with it still reside on its internal disk...