What open source solutions are available to use "ZFS over iSCSI with Proxmox"?

A

al.semenenko88

Member
Dec 25, 2017
23
0
21
36
What open source solutions are available to use "ZFS over iSCSI with Proxmox"?
Will FreeNas 9 or 10 work for "ZFS over iSCSI"?
What solutions are on Linux?
 
These patches added support for native Linux ZFS over ISCSI support. And what about FreeNAS support?
I've got problems with running OmniOS on my hardware. What would be a better choice for NAS/SAN storage in near future? Linux server with LIO/targetcli or FreeNAS?
 
Bump

I've managed to apply ILO patches on my testing virtual environment. Looks like it works - I can create, delete and migrate offline KVM VMs. Great work of Udo Rader !!!
Can't test it on running VMs yet, as Virtualbox doesn't allow to run nested virtualization. Right now I'm preparing to move my tests to bare metal hardware.
Is there any ETA to include patches to official Proxmox release?
 
Cool. Now I need to choose between ZFS On Linux and FreeBSD :)
I think, FreeBSD would be a better choice.

Are these patches going to be included in Proxmox in near future?
 
I like the way how the connection from proxmox to freenas is set up. It's done over the api, not over ssh with config file manipulation.
I'd appreciate too, when this would be included into proxmox mainstream.
 
stefanzman said:
Raku - What kind of problems did you experience with OmniOS? This was just recommended to for an iSCSI shared storage solution for a PVE cluster. Is FreeNAS a better choice in your opinion?
Click to expand...

Typical hardware compatibility problems:
1) OmniOS doesn't recognize my SATA drives connected to LSI based RAID controller. Only SAS drives are visible
2) OmniOS doesn't work with my Mellanox 10Gbit network adapter

Is FreeNAS better? I don't know. I've patched my cluster with freenas-proxmox from https://github.com/TheGrandWazoo/freenas-proxmox. I was able to connect the cluster to my FreeNAS storage (16 x 4 TB SAS). I can create, run, migrate, destroy KVM machines. I've tested sequential writes with dd and got transfers about 1GB/s on empty cluster. Everything works good so far.
What I miss is https access from proxmox to FreeNAS API. I'm working to fix that and when it's ready, I'll create pull request to include my patches to freenas-proxmox project.
 
  • Like
Reactions: liptech
raku said:
2) OmniOS doesn't work with my Mellanox 10Gbit network adapter
Click to expand...
This is strange since the drivers for Mellanox 10Gbit in FreeNAS/FreeBSD is the ported drivers from OmniosCE. I have no problems with either Mellanox 10 or 20 Gbit in both Omnios and FreeNAS.
 
Thanks for the clarification, Raku - please keep me posted on your progress within this topic?

We are trying to establish a standard / preferred shared cluster storage to recommend to our Proxmox customers and prospects, and I would like to have assurances that people have had consistent success with specific configuration(s).

Thanks very much.
 
Progress report:
I've configured FreeNAS storage and found it faulting while cloning VMs and booting from virtio SCSI hard drive. So I moved to Ubuntu 18.04 and iSCSI via LIO. First look after patching Proxmox and configuring ZFS pool and iSCSI target - all works smoothly. But closer look revealed huge immature of ZoL and iSCSI soluton.

I've tested storage performance inside KVM VM:
FreeNAS: about 450-700 MB/s, 3500 IOPS
Ubuntu: about 80-120 MB/s, 700 IOPS

So I switched back to FreeNAS and just a couple minutes ago solved my issues:
https://github.com/TheGrandWazoo/freenas-proxmox/issues/9
https://github.com/TheGrandWazoo/freenas-proxmox/issues/10

I think, I'm going to stay with the FreeNAS solution. Zvol over iSCSI on linux sucks.
 
Last edited:
All features I need work OK. But so far, I run only 1-2 VMs simultaneously. I need to do some more tests running about 50-100 VMs. If that works, I'll say it is production ready on my site and start to migrate from the old XenServer cluster.
 
It would be simply amazing to use zvol over ISCSI with FreeNAS. I'm a bit nervous to put it into production without an official release though. Is there any thoughts on this from Proxmox devs? I am going to give it a shot on some test hardware.
 
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom
Back