btrfs as a guest file system

[VGusev2007]

Dear all, I want to have urbackup VM and I want to use btrfs killer feature (cross link, offline dedup and so on) with urbackup.

What do you think if I will have zfs (zvol) as my host fs and btrfs as a guest system?

Best regards,

 

[LnxBil]

What do you think if I will have zfs (zvol) as my host fs and btrfs as a guest system?

Technically, it'll work but it will not be fast.

Btrfs has also killer features like not-production-readieness and the king-of-data-lossiness (if used with any other raid level than mirroring).

 

[VGusev2007]

Btrfs has also killer features like not-production-readieness and the king-of-data-lossiness (if used with any other raid level than mirroring).

I understand for non best performance. I have read about btrfs still improve from time to time in current linux kernel. Do u have real production use btrfs or just read about bad for use it?

 

[LnxBil]

Do u have real production use btrfs or just read about bad for use it?

Talked to people that develop it, they even not recommend raid-level higher than 1 for production use. They even put it on Wikipedia:

https://en.wikipedia.org/wiki/Btrfs#Implemented_but_not_recommended_for_production_use

 

[RMM]

I also want to use UrBackup (But I'm more interested in the endless incremental style backups, which is possible to do on zfs and btrfs). I first tried to get it working in a LXC container, but it wasn't that easy to manage to allow it to directly make snapshots etc. out of the container.
So I tried it inside of a VM using btrfs with kernel 4.19 (default of debian buster).
The first test result with doing a simple dd from urandom was pretty bad.
Around 45MB/s.
Directly in the host system I get 150-160MB/s.

I've tried out to deactivate cow in btrfs (nodatcow) => no improvement (maybe that would show some improvement when changing files)

Deactivating compression in zfs (btrfs already compresses transparently) => no improvement

Changing volblocksize in zfs to 16Kb since Archwiki claims that the default blocksize of btrfs is 16Kb. I couldn't find any other source for it, and neither could I find out how to get the blocksize of a btrfs filesystem. I've tried it anyway. => seems like it improved for maybe 10-15% but that could also be measurement errors, the dd "benchmark" is neither very accurate nor reliable nor does it give any real life results ;-).

deactivating sync on the zvol => That made a real improvement. it flactuates a lot. but it seems to improve the writing speed by 80% up to almost 200% (=140MB/s). In Most measurements the improvement was around 100% (so the double).
So with sync=disabled it seems to be usable. I'm too lazy to do some proper benchmarks, but I'll try out to get the backups in like that for a while and see what the performance is like. Maybe also changing the blocksize of the zvol helps, but if it does. It doesn't seem to make a huge impact.

 

[VGusev2007]

deactivating sync on the zvol => That made a real improvement. it flactuates a lot. but it seems to improve the writing speed by 80% up to almost 200% (=140MB/s). In Most measurements the improvement was around 100% (so the double).
So with sync=disabled it seems to be usable. I'm too lazy to do some proper benchmarks, but I'll try out to get the backups in like that for a while and see what the performance is like. Maybe also changing the blocksize of the zvol helps, but if it does. It doesn't seem to make a huge impact.

Thank a lot for you answer! I have a doubt about disable sync=disabled... What will with your data in "power cut" situation?

 

[Nemesiz]

Just don`t forget sync=disabled in the machine crash you can lose some data.

 

[VGusev2007]

Just don`t forget sync=disabled in the machine crash you can lose some data.

Yep, I can confirm that, because I have this bad situation...

 

[RMM]

I've read at several places in this forum that people consider sync=disabled to be save.

I've found out that the block size of 16kb on btrfs is only for the metadata. The block size for the real data seems to be 4kb. So increasing the zvol blocksize shouldn't really help (i guess? ;-)).

 

[VGusev2007]

I've read at several places in this forum that people consider sync=disabled to be save.

No. I have lose my data. That is not safe! Don't use it in production.

 

[RMM]

I guess it depends what you use it for. If your server is backed up by a ups, then the only way you could loose data would be if the host OS crashes or your hardware utterly fails. The changes are relatively low. Anyway you would loose around 5seconds of data (The data will be written back every 5 seconds anyway, so you'd loose up to 5 seconds + the time needed to write the data back). With bad luck it probably could lead to some inconsistencies in the btrfs file system, since it relies on that his sync writes are really sync (with sync=disabled we lie to the guest host and tell him the sync write has finished even though it didn't). I'm not sure how the difference would be to a normal black out when using btrfs directly on a disk. As long as zfs doesn't reorder the writes it should be pretty much the same (maybe it does it for performance reasons for non-synced writes?)?
Anyway you could still force a sync like twice per day and take a snapshot of the zvol. If you get a data lose you can go back to the snapshot. For my backup server and the data I backup there that would be ok for me.

I've tried out some real backups with sync=standard and sync=off backups were coming in with up to 200MBit/s. There is no client which could deliver more. Most of the backup clients even deliver quite less. So for the moment I'll stay with sync enabled.
If sync=disabled improves your speed a lot and you need that extra speed but can't risk any data loss at all, then consider a slog (separate device to write the zil to. Let's call it a write "cache" ;-) ) . For my purpose that would be too expensive...

 

[VGusev2007]

slog (separate device to write the zil to. Let's call it a write "cache" ;-) ) . For my purpose that would be too expensive...

Did you understand for zlog at all? zlog works only with a database, the zlog may tell you db engine I'm write your data when a real data is on RAM and SSD cache, after that zfs will write your data to disk from RAM. zlog SSD will have only if power cut situation. zlog is very good for exmaple in 'netavis observer', when you need to db for store events and you have HDD for store video data. If you don't have any DB, and just copy regular data you don't need an any SSD cache.

 

[VGusev2007]

The first test result with doing a simple dd from urandom was pretty bad.
Around 45MB/s.
Directly in the host system I get 150-160MB/s.

There is another moment: https://access.redhat.com/documenta..._configuration-random_number_generator_device

I'm not deep in this, but I suppose if you use program urandom in VM it is have some overhead in compare with real hardware. Thank for your banchmark. It is very good point to start use btrfs for me.

 

[Pablo Alcaraz]

I understand for non best performance. I have read about btrfs still improve from time to time in current linux kernel. Do u have real production use btrfs or just read about bad for use it?

I have all the guest hosts using btrfs. I lost nothing. I use it in JBOD and RAID 1 configurations. The dangerous configurations are software btrfs RAID 5 and software btrfs RAID 6 configurations. They are not strictly RAID 6 or 5 and in any case, they are not stable.

btrfs is safe and uses FAR less resources than ZFS. Just do not use software raid 5 or 6 and you will be fine.

 

[VGusev2007]

I have all the guest hosts using btrfs. I lost nothing. I use it in JBOD and RAID 1 configurations. The dangerous configurations are software btrfs RAID 5 and software btrfs RAID 6 configurations. They are not strictly RAID 6 or 5 and in any case, they are not stable.

btrfs is safe and uses FAR less resources than ZFS. Just do not use software raid 5 or 6 and you will be fine.

Thank a lot for you! I'll use it!