how btrfs works

powersupport

Active Member
Jan 18, 2020
244
2
38
29
Hi,
I would like to understand more on how btrfs work. In the proxmox server, there are 4 x 1.92TB diskspace, if combined together, there should be about 8TB diskspace, however in the server, there is only about 4TB diskspace available for use.

Question:-
  1. May I know if this is 1:1 ratio? How will it impact if 1 SSD fails?
  2. Will the system still work if 3 SSDs fail?
  3. Can we replace an SSD if SSD fails and it will rebalance it automatically like OSDs?
 
Btrfs is a file system but no RAID level.
The file system (BTRFS, EXT4, etc.) is placed „on top“ of your RAID.

If you want to use all available disk space you’ll have to use RAID-0 which has no protection if one drive fails.

Overview of standard RAID levels:

https://en.m.wikipedia.org/wiki/Standard_RAID_levels

Within PVE you can create ZFS RAID of different levels. ZFS RAID-Z are parity RAID levels with different speed and protection. The more drives you use for parity the more space you’ll loose.

https://www.diskinternals.com/raid-recovery/what-is-raidz/
 
Last edited by a moderator:
Question:-
  1. May I know if this is 1:1 ratio? How will it impact if 1 SSD fails?
  2. Will the system still work if 3 SSDs fail?

1. run btrfs filesystem df /path command to see redundancy level. Most probably you have a RAID10 configuration (striped 2-way mirror).
2. you need a 4-way mirror to be able to withstand 3 drive loss. I doubt that you have this configuration.
 
Last edited:
May I know an update for the below?
  1. if one of the SSD fails, May I know whether it will rebalance it automatically like OSDs?
 
I am not an expert on btrfs file system,
when I check the df -h I get the result below
Code:
/dev/sda3                                            3.5T  469G  3.1T  14% /
So, while running the command get the result below
btrfs filesystem df /dev/sda3
Code:
ERROR: not a directory: /dev/sda3

Also, there 4 disks in the server each has a capacity of 8 TB, but after the created the btrfs file system, only shows 4 TB, may I know is there any reason for the same?


Thank you
 
You need to specify a mount point as path. Your /dev/sda3 partition is mounted as root, so just
btrfs filesystem df /

You will get info separately for data, system and metadata. Single means no redundancy, dup means duplicated/mirrored.
 
Please see the result below
# btrfs filesystem df /
Data, RAID10: total=576.00GiB, used=575.13GiB
System, RAID10: total=16.00MiB, used=80.00KiB
Metadata, RAID10: total=2.50GiB, used=822.44MiB
GlobalReserve, single: total=512.00MiB, used=0.00B

Also, could you please advise on the query
There 4 disks in the server each have a capacity of 8 TB, but after the created the btrfs file system, only shows 4 TB, may I know is there any reason for the same?

Thank you
 
There 4 disks in the server each have a capacity of 8 TB, but after the created the btrfs file system, only shows 4 TB, may I know is there any reason for the same?
As other already said: Redundancy. You have RAID10 according to the output which uses 2 out of the 4 disks as copies, so that every bit is stored twice.
 
May I know how to configure to sync 4 disks? any documentation?
Also, is this the reason for the disk only showing 4 TB instead of 8?
 
May I know how to configure to sync 4 disks? any documentation?
Also, is this the reason for the disk only showing 4 TB instead of 8?
you will find links to docs in this thread already, read all the answers again.
 
  • Like
Reactions: LnxBil
HI,

Thank you for the information, I understood that there is two pair of disk for mirroring and stripping

May I check another thing, is the disks in Raid10 btrfs possible to replace? , after replicating the disk, will it rebalance automatically like OSDs?

Thank you.
 
May I check another thing, is the disks in Raid10 btrfs possible to replace? , after replicating the disk, will it rebalance automatically like OSDs?
Yes, you can replace RAID10 btrfs disks using standard procedures:
https://btrfs.wiki.kernel.org/index.php/Using_Btrfs_with_Multiple_Devices#Replacing_failed_devices

Use btrfs replace method. It is faster (rebalance would occure only once) and easier then old method (btrfs device add + btrfs device delete missing).
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!