Proxmox btrfs support roadmap, as fallback for possible licensing issues ZFS on linux

[EuroDomenii]

FYI, RedHat seems to be abandoning btrfs, as it was removed from the latest RHEL 7.4.

This might be serious, because as we know, RedHat has quite strong voice in Linux. Remember, it came with systemd, and despite strong oppostion, it pressed it forward to became de-facto standard init system...

Most relevant Josef Bacik (ex btrfs maintainer at Red Hat) opinion about this step: https://news.ycombinator.com/item?id=14907771

So is rather the limitations of the RedHat releasing pattern, they don't keep up with BTRFS.

This story reminds me of https://docs.docker.com/engine/userguide/storagedriver/device-mapper-driver/, they prefer to implement their own driver, to match their criteria, even from density and Paas point of view, doesn't shine.

Is true that RedHat was a pioneer in many ways, but in this case they over simplify, leaving their clients behind, missing the feature of a modern filesystem.

Developing their own filesystem? It takes at least 10 years for a filesystem to be mature.

On the other hand, it could be also a conflict with Oracle.

RedHat leaves something was never really into, but Suse, Facebook, Fujitsu, Oracle are powerful companies, backing BTRFS

 

[Rhinox]

There is not real alternative to btrfs.

And do we want to have alternative to btrfs? (OT: check bcachefs, seems very promising)

...ZFS in linux is poorly integrated

It is in mainstream kernel, so what you'd like to improve in its integration?

...competes successfully against server services by eating all memory resources

Every modern OS (and filesystem) does the same, following the rule "any utilization of ram is better then not utilizing it at all". The only difference is ZFS is using its own (much better) ARC, instead of standard linux disk-cache.

...is cumberstone..

Explain...

...And all the time is that licensing risk liability...

Guess who started btrfs development? Yes, Oracle! No need to say more...

We are debian/ubuntu fans. Red Hat is too risky because they are main (unique?) contributors of Fedora IMHO..

RH is risky because it is top linux commercial player, with dominant position on corporate market. It can not be taken lightly. Just remember systemd-story...

 

[guletz]

This is like saying: "never use xfs, ext4."

You missunderstend me(my fault) I want to say that are many situation, when is not a big problem if you lose some or all data(a backup server for example). But at the same time, are cases when if you lose data, it is a disaster!

This does mean you don't get CRC protection for data, though. Since most databases do this internally, that is probably no great loss

CRC ptotection has only one advantage - it tell you if that the data it is corupted or not. But when you know that you have some data coruption, CRC can not recover your data. And for this reason, the guy tell "that is probably no great loss". "No great loss" is not the same thing with NO DATA LOSS. If in your case " no great loss" is OK/acceptable, then it is OK.

Have a nice day!

 

[Rhinox]

One more interesting link (some might have missed). It explains RH plans in storage, which basically is "stratis+xfs".
BTW, "stratisd", "systemd"... deja-vu? I would not be surprised...

 

[EuroDomenii]

One more interesting link (some might have missed). It explains RH plans in storage, which basically is "stratis+xfs".
BTW, "stratisd", "systemd"... deja-vu? I would not be surprised...

Thanks for the useful links, but I couldn't help but notice the "d" suffix, all over the post, sounds like solid numerology arguments! The list could go on container -> containerd ...

 

[alexskysilk]

The point, I think, is mostly moot. btrfs and zfs overlap but dont really serve the same purpose.

btrfs is/was designed to replace ext4 is a mainstream file system, offering CoW features such as snapshots, yadda yadda. Its biggest advantage vs zfs, afaict, is that btrfs is designed to be rebalance-able where zfs is not. this is a major design advantage if/when they get it actually working.

The thing is, zfs has such a major head start that looking at the btrfs roadmap (https://btrfs.wiki.kernel.org/index...elopment_or_Planned_for_Future_Implementation), it doesnt even have features on the roadmap zfs is already shipping.

Consider the current release for zfs:
https://github.com/zfsonlinux/zfs/releases/tag/zfs-0.7.0

resumable zfs send- check.
zfs send native compression- check
zvol/filesystem delegation- check
compressed ARC- check

coming down the pike, THIS should interest anyone with large pools:
https://github.com/zfsonlinux/zfs/wiki/dRAID-HOWTO

All things being equal, I dont see btrfs a compelling solution for the timebeing.

 

[Pablo Alcaraz]

And do we want to have alternative to btrfs? (OT: check bcachefs, seems very promising)
It is in mainstream kernel, so what you'd like to improve in its integration?

I understand is not part of the kernel. Please check: https://wiki.archlinux.org/index.php/ZFS
In special:

Note: Due to potential legal incompatibilities between CDDL license of ZFS code and GPL of the Linux kernel ([2],CDDL-GPL,ZFS in Linux) - ZFS development is not supported by the kernel.

As a result:

• ZFS still resides in Arch User Repository and unofficial archzfs repository.
• ZFSonLinux project must keep up with Linux kernel versions. After making stable ZFSonLinux release - Arch ZFS maintainers release them.
• This situation sometimes locks down the normal rolling update process by unsatisfied dependencies because the new kernel version, proposed by update, is unsupported by ZFSonLinux.

So we have "cuts" of kernel+ZFS4L in several flavors: ProxMox's cut, zfsonlinux.org's cut, etc and I do not feel comfortable using patches out of mainline. I try to avoid that kind of things.

Every modern OS (and filesystem) does the same, following the rule "any utilization of ram is better then not utilizing it at all". The only difference is ZFS is using its own (much better) ARC, instead of standard linux disk-cache.

This difference is big. I have not objections if ZFS is used in a filesystem server. In that scenario I am good if they allocate all the memory and use it at its pleasure.

But this is not the scenario I am using ProxMox with. In my case ProxMox hosts are VM and container host offering some hardware resources (like GPU) and storage.
In those servers I have containers running databases. They will compete for the memory with ZFS. With btrfs since it is implemented using kernel VFS the kernel can do better memory management. In special postgresql loves using OS caches and buffers. Kernel could give it all the memory postgresql wants while btrfs is not using it and viceversa. That is not possible with ZFS where I must define how much memory it will use.

Explain...

BTRFS is more flexible IMO. It should not be usual, but I migrate volumes from JBOD to DUP or RAID1 and then back to JBOD more than I would like. I need support to do it using so few command lines as possible so I can do it fast. It is not a strong point though. I could adapt to more command lines if it would offer me same flexibility.

Guess who started btrfs development? Yes, Oracle! No need to say more...

But the license is different:

• ZFS is CDDL: There is debate (aka liability risk) about if it is GPL compatible. I am too busy and it could generate lot of problems that do not compensate ZFS technical advantages IMHO.
• BTRFS is GPL: Impossible to beat by corporations. Court tested.

RH is risky because it is top linux commercial player, with dominant position on corporate market. It can not be taken lightly. Just remember systemd-story...

This is under hot debate in both sides. I prefer to stay away of corporate drama as far as I can by using only technologies with low legal liability risk.

 

[alexskysilk]

ZFS is CDDL: There is debate (aka liability risk) about if it is GPL compatible. I am too busy and it could generate lot of problems that do not compensate ZFS technical advantages IMHO.

There is no liability risk, please stop spreading fud. Zfs is cddl which means it cannot comingle in the linux kernel- which it isn't. Zfs ships as dkms.

BTRFS is GPL: Impossible to beat by corporations. Court tested.

I don't have any clue what you are trying to say. What court tested the gpl, and in what context would this be relevant to the conversation? The license conflict between cddl and gpl is that gpl requires publishing the source code and cddl doesn't.

 

[EuroDomenii]

There is no liability risk, please stop spreading fud.

Fud is saying that there is no problem with ZFS licence on Linux

You are right and you are wrong. There are 3 cases:

1. The licence incompatibility comes with linux distributions: EG: Linux distribution Canonical, Ltd. announced their plans to commercially distribute, in Ubuntu 16.04, a binary distribution of ZFS as a Linux kernel module, which adapts ZFS natively for Linux.

2. You are right https://sfconservancy.org/blog/2016/feb/25/zfs-and-linux/

Pure distribution of source with no binaries is undeniably different. When distributing source code and no binaries, requirements in those sections of GPLv2 and CDDLv1 that cover modification and/or binary (or “Executable”, as CDDLv1 calls it) distribution do not activate. Therefore, the analysis is simpler, and we find no specific clause in either license that prohibits source-only redistribution of Linux and ZFS, even on the same distribution media.

But not completely:

Nevertheless, there may be arguments for contributory and/or indirect copyright infringement in many jurisdictions. We present no specific analysis ourselves on the efficacy of a contributory infringement claim regarding source-only distributions of ZFS and Linux. However, in our GPL litigation experience, we have noticed that judges are savvy at sniffing out attempts to circumvent legal requirements, and they are skeptical about attempts to exploit loopholes.

We note that Debian's decision to place source-only ZFS in a relegated area of their archive called contrib, is an innovative solution.

If you are compile the ZFS yourself and use it at home, you are 100% safe. Thanks @guletz for pointing this out for me!

3. You are wrong

But, even if you compile it yourself but you are planning to release a public product based on Proxmox ( eg PaaS as we plan to do), you fall back to distribution use case ( usage of distribution of your custom version of Proxmox with ZFS, for your clients).

ZFS and BTRFS are the most featured filesystem.

So, no thanks, I am not planning to setup a business online on ZFS moving sands, in order to migrate my clients to BTRFS, in the ZFS licence worst case scenario.

 

[alexskysilk]

So, no thanks, I am not planning to setup a business online on ZFS moving sands, in order to migrate my clients to BTRFS, in the ZFS licence worst case scenario.

That is your decision to make, but using that logic you should also avoid using motor vehicles in your business because there is no guarantee that you won't get in an accident- which is far more likely then you running into legal trouble with zfs. Even the articles you mention only suggest this possibility as a far fetched hypothetical.

Zfs has been shipping in open source for 12 years. During that time no one has expressed any interest in stopping it. It is used in production in many places including government contracts. If anything, the only thing that will kill zfs is irrelevance, not law; zfs isn't clusterable. As the feature overlap between ceph and zfs grows it may simply become unimportant.

 

[EuroDomenii]

That is your decision to make, but using that logic you should also avoid using motor vehicles in your business because there is no guarantee that you won't get in an accident- which is far more likely then you running into legal trouble with zfs.

"It is estimated that motor vehicle collisions caused the deaths of around 60 million people during the 20th century,[6] around the same as the number of World War II casualties. https://en.wikipedia.org/wiki/Epidemiology_of_motor_vehicle_collisions. " Your comparison is inappropriate and morbid. So, anything you do is far less likely, than a car accident.

It is used in production in many places including government contracts.

Let's separate the Solaris and BSD derivatives, without any licensing issues, from ZFS on Linux.

On the other hand, why Facebook would hire Chris Mason, the lead developer of BTRFS, https://www.linux.com/news/learn/in...ok-uses-linux-and-btrfs-interview-chris-mason, instead of using ready made ZFS on Linux?

Lawrence Livermore National Laboratory has done the initial port, with all the disclaimers http://zfsonlinux.org/zfs-disclaimer.html.

Please give me more big production installations of ZFS on Linux, it could be useful.

 

[guletz]

Some kind of companys who use zol:

http://open-zfs.org/wiki/Companies

 

[mir]

Some kind of companys who use zol:

http://open-zfs.org/wiki/Companies

You should write OpenZFS. Most of the listed companies does not use zol but Solaris/Illumos or FreeBSD.

 

[guletz]

You should write OpenZFS

Sorry, my mistake.