ZFS on Linux is production ready - any plans to incorporate it into Proxmox?

[gkovacs]

One of the most advanced filesystems (with volume management built in), the Sun Microsystems developed Zettabyte File System (ZFS) was open sourced in 2005, and it's Linux port has recently reached production-ready status. It is built for data security, providing much higher levels of reliability than hardware RAID arrays - and does it all in software. It uses a variety of techniques to achieve that, like redundancy, checksums, distributed superblocks, rollbackable changes and fast snapshots thanks to copy-on-write and others. More info here:
http://zfsonlinux.org/faq.html

According to this article (and many others), it is especially useful for todays extremely large filesystems:
http://www.theregister.co.uk/2013/03/30/zfs_on_linux_production_ready/

Proxmox badly needs a reliable and fast filesystem, as ext3 is not suitable to large deployments (our 1 TB arrays routinely take 30-60 minutes to fsck while the server is down), and ext4 has been causing problems with LVM backups as of the latest PVE kernel:
http://forum.proxmox.com/threads/13...everal-servers-since-upgrading-to-Proxmox-2-3

I know it's not a small project, but would glady help testing it on several different hardware.
Are the any plans to incorporate it?

 

[dietmar]

Are the any plans to incorporate it?

no, license does not allow that.

- - - Updated - - -

I guess btrfs will be the future. For now, xfs is also very good.

 

[tom]

As long as you want to use OpenVZ, ext3 is the recommended setup.

Just to note ZFS in NOT always fast. ext3/ext4/xfs is considerable faster for single servers- ZFS is designed for large file-systems with many spindles. Also btrfs is not known to be the fastest.

But you can always use the best suitable file system for your needs, in our wiki there is howto for ZFS.

 

[dipe]

no, license does not allow that.

- - - Updated - - -

I guess btrfs will be the future. For now, xfs is also very good.

-------------------------------------------------
Actually, the licensing issue has always been discussed in the context of ZFS getting integrated in into the kernel itself. ZFS on Linux is deployed as a Kernel module which is a perfectly legal way to deploy this software.

http://zfsonlinux.org/faq.html#WhatAboutTheLicensingIssue
"In the case of the kernel, this prevents us from distributing ZFS as part of the kernel binary. However, there is nothing in either license that prevents distributing it in the form of a binary module or in the form of source code."

I would agree that BTRFS will take over at some point in the future. However, we will probably not see any Enterprise deployments in the next 2 years. (actually people said it will be "another year" since 2009 and some of us got grey hair in the meantime)
However BTRFS does not have a caching feature that is as elaborate as ZFS. We might have bcache or something like that at some point in the future but today there is nothing compared to the ARC/L2ARC that ZFS provides/

ZFS is actually quite slow as a file system (because of all its features) but it scales much better on high end hardware than XFS or other Linux file systems. Today you can buy a server with 500GB RAM for little money and if you add a reasonably fast SSD to it there is nothing that is currently faster than ZFS ....and who wants to use Proxmox on a pizza box with 4GB RAM?

Official Proxmox support for ZFS would be great and quite a few more people would get a support contract if you did that. After all Promox is a still a well kept secret for most people. A proxmox box with zfs would essentially be a similar thing to what Joyent does with http://smartos.org/ .....except Proxmox has more features

 

[demoletor]

Any news?

 

[dietmar]

Any news?

I will upload test packages in a few days to pvetest repository.

 

[demoletor]

I will upload test packages in a few days to pvetest repository.

Great. We use nas4free with zfs in own storage server and I hope native zfs in proxmox. the idea is use "zfs send" and "zfs receive" for backup the proxmox system.

 

[proxtest]

I will upload test packages in a few days to pvetest repository.

Will it be possible to move from drbd to zfs in a 2 node setup?

Drbd stops working sometimes and i dont can't find any reason, so i want to move to a more stable system.

I only get:
Jan 12 10:49:34 node1 kernel: block drbd0: Remote failed to finish a request within ko-count * timeout

I post it to the mailinglist already but get no mails and can't find it there

http://www.gossamer-threads.com/lists/drbd/users/

also. so i want to move, is zfs the best idea to have 2 nodes in sync?

 

[acidrop]

so i want to move, is zfs the best idea to have 2 nodes in sync?

ZFS cannot do that. I mean it cannot do real time syncing of data like DRBD does.
You can use zfs send/receive to backup your primary server to a secondary server but you cannot work in dual primary mode like in DRBD, so no live migration is possible.
DRBD works fine for me (no HA), no issues at all.

 

[e100]

Drbd stops working sometimes and i dont can't find any reason, so i want to move to a more stable system.

I only get:
Jan 12 10:49:34 node1 kernel: block drbd0: Remote failed to finish a request within ko-count * timeout

Start a new thread and I will try to help you, include your drbd config files and your server specifications.

 

[proxtest]

Start a new thread and I will try to help you, include your drbd config files and your server specifications.

I only want to inform, i havn't changed anything and with pri/sec it runs without problems since 2 weeks!
On the drbd mailinglist eric's mention looks right, there is a problem with pri/pri only!

U want me to start a new thread?

 

[Nemesiz]

ZFS on linux can work great but have some disadvantage/problems.

ZFS pool as separate file system problems:

* inside OpenVZ no 'live' disk usage (can be updated by 'vzctl quotainit <ctid>' then container is offline)
* inside OpenVZ attr does not work.

ZFS as native file system (I use it for my desktop ubuntu)

* same problems as list before.
* need special GRUB version to support ZFS. After upgrade ZFS from 0.6.2 to 0.6.3 had big problems with GRUB. After searching working deb I stay with 2.01-22debian1+zfs3-0.6.3-2~wheezy but on startup I have to export and import manually ZFS pool to continue boot.

 

[fireon]

is still really what is here: https://pve.proxmox.com/wiki/Pci_passthrough#Verify_Operation
Is this only when i have NAS or is this too when i use Proxmox with there own zpool on local harddisk?

Editorial Note: Using PCI passthrough to present drives direct to a ZFS (FreeNAS, Openfiler, OmniOS) virtual machine is dangerous on many levels and is not recommended for production use. Specific FreeNAS warnings can be found here: http://forums.freenas.org/threads/a...ide-to-not-completely-losing-your-data.12714/

Thanks