how btrfs works

[powersupport]

Hi,
I would like to understand more on how btrfs work. In the proxmox server, there are 4 x 1.92TB diskspace, if combined together, there should be about 8TB diskspace, however in the server, there is only about 4TB diskspace available for use.

Question:-

1. May I know if this is 1:1 ratio? How will it impact if 1 SSD fails?
2. Will the system still work if 3 SSDs fail?
3. Can we replace an SSD if SSD fails and it will rebalance it automatically like OSDs?

 

[Deleted member 116138]

Btrfs is a file system but no RAID level.
The file system (BTRFS, EXT4, etc.) is placed „on top“ of your RAID.

If you want to use all available disk space you’ll have to use RAID-0 which has no protection if one drive fails.

Overview of standard RAID levels:

https://en.m.wikipedia.org/wiki/Standard_RAID_levels

Within PVE you can create ZFS RAID of different levels. ZFS RAID-Z are parity RAID levels with different speed and protection. The more drives you use for parity the more space you’ll loose.

https://www.diskinternals.com/raid-recovery/what-is-raidz/

 

[mvs]

Question:-

1. May I know if this is 1:1 ratio? How will it impact if 1 SSD fails?
2. Will the system still work if 3 SSDs fail?

1. run btrfs filesystem df /path command to see redundancy level. Most probably you have a RAID10 configuration (striped 2-way mirror).
2. you need a 4-way mirror to be able to withstand 3 drive loss. I doubt that you have this configuration.

 

[powersupport]

May I know an update for the below?

1. if one of the SSD fails, May I know whether it will rebalance it automatically like OSDs?

 

[mvs]

if one of the SSD fails, May I know whether it will rebalance it automatically like OSDs?

To my knowledge BTRFS should not start a rebalance automatically. At least if you use default configuration without hot-spare mods, etc.

 

[Neobin]

Btrfs is a file system but no RAID level.

BTRFS also provides raid functionality. [1]

[1] https://btrfs.wiki.kernel.org/index.php/Using_Btrfs_with_Multiple_Devices

 

[powersupport]

I am not an expert on btrfs file system,
when I check the df -h I get the result below

/dev/sda3                                            3.5T  469G  3.1T  14% /

So, while running the command get the result below
btrfs filesystem df /dev/sda3

ERROR: not a directory: /dev/sda3

Also, there 4 disks in the server each has a capacity of 8 TB, but after the created the btrfs file system, only shows 4 TB, may I know is there any reason for the same?


Thank you

 

[mvs]

You need to specify a mount point as path. Your /dev/sda3 partition is mounted as root, so just
btrfs filesystem df /

You will get info separately for data, system and metadata. Single means no redundancy, dup means duplicated/mirrored.

 

[powersupport]

Please see the result below
# btrfs filesystem df /
Data, RAID10: total=576.00GiB, used=575.13GiB
System, RAID10: total=16.00MiB, used=80.00KiB
Metadata, RAID10: total=2.50GiB, used=822.44MiB
GlobalReserve, single: total=512.00MiB, used=0.00B

Also, could you please advise on the query
There 4 disks in the server each have a capacity of 8 TB, but after the created the btrfs file system, only shows 4 TB, may I know is there any reason for the same?

Thank you

 

[LnxBil]

There 4 disks in the server each have a capacity of 8 TB, but after the created the btrfs file system, only shows 4 TB, may I know is there any reason for the same?

As other already said: Redundancy. You have RAID10 according to the output which uses 2 out of the 4 disks as copies, so that every bit is stored twice.

 

[powersupport]

May I know how to configure to sync 4 disks? any documentation?
Also, is this the reason for the disk only showing 4 TB instead of 8?

 

[tom]

May I know how to configure to sync 4 disks? any documentation?
Also, is this the reason for the disk only showing 4 TB instead of 8?

you will find links to docs in this thread already, read all the answers again.

 

[powersupport]

HI,

Thank you for the information, I understood that there is two pair of disk for mirroring and stripping

May I check another thing, is the disks in Raid10 btrfs possible to replace? , after replicating the disk, will it rebalance automatically like OSDs?

Thank you.

 

[mvs]

May I check another thing, is the disks in Raid10 btrfs possible to replace? , after replicating the disk, will it rebalance automatically like OSDs?

Yes, you can replace RAID10 btrfs disks using standard procedures:
https://btrfs.wiki.kernel.org/index.php/Using_Btrfs_with_Multiple_Devices#Replacing_failed_devices

Use btrfs replace method. It is faster (rebalance would occure only once) and easier then old method (btrfs device add + btrfs device delete missing).