how btrfs works

Jan 18, 2020
167
1
23
27
Hi,
I would like to understand more on how btrfs work. In the proxmox server, there are 4 x 1.92TB diskspace, if combined together, there should be about 8TB diskspace, however in the server, there is only about 4TB diskspace available for use.

Question:-
  1. May I know if this is 1:1 ratio? How will it impact if 1 SSD fails?
  2. Will the system still work if 3 SSDs fail?
  3. Can we replace an SSD if SSD fails and it will rebalance it automatically like OSDs?
 

Deleted member 116138

Guest
Btrfs is a file system but no RAID level.
The file system (BTRFS, EXT4, etc.) is placed „on top“ of your RAID.

If you want to use all available disk space you’ll have to use RAID-0 which has no protection if one drive fails.

Overview of standard RAID levels:

https://en.m.wikipedia.org/wiki/Standard_RAID_levels

Within PVE you can create ZFS RAID of different levels. ZFS RAID-Z are parity RAID levels with different speed and protection. The more drives you use for parity the more space you’ll loose.

https://www.diskinternals.com/raid-recovery/what-is-raidz/
 
Last edited by a moderator:
Mar 25, 2022
74
19
8
Question:-
  1. May I know if this is 1:1 ratio? How will it impact if 1 SSD fails?
  2. Will the system still work if 3 SSDs fail?

1. run btrfs filesystem df /path command to see redundancy level. Most probably you have a RAID10 configuration (striped 2-way mirror).
2. you need a 4-way mirror to be able to withstand 3 drive loss. I doubt that you have this configuration.
 
Last edited:
Jan 18, 2020
167
1
23
27
I am not an expert on btrfs file system,
when I check the df -h I get the result below
Code:
/dev/sda3                                            3.5T  469G  3.1T  14% /
So, while running the command get the result below
btrfs filesystem df /dev/sda3
Code:
ERROR: not a directory: /dev/sda3

Also, there 4 disks in the server each has a capacity of 8 TB, but after the created the btrfs file system, only shows 4 TB, may I know is there any reason for the same?


Thank you
 
Mar 25, 2022
74
19
8
You need to specify a mount point as path. Your /dev/sda3 partition is mounted as root, so just
btrfs filesystem df /

You will get info separately for data, system and metadata. Single means no redundancy, dup means duplicated/mirrored.
 
Jan 18, 2020
167
1
23
27
Please see the result below
# btrfs filesystem df /
Data, RAID10: total=576.00GiB, used=575.13GiB
System, RAID10: total=16.00MiB, used=80.00KiB
Metadata, RAID10: total=2.50GiB, used=822.44MiB
GlobalReserve, single: total=512.00MiB, used=0.00B

Also, could you please advise on the query
There 4 disks in the server each have a capacity of 8 TB, but after the created the btrfs file system, only shows 4 TB, may I know is there any reason for the same?

Thank you
 

LnxBil

Famous Member
Feb 21, 2015
6,059
740
133
Germany
There 4 disks in the server each have a capacity of 8 TB, but after the created the btrfs file system, only shows 4 TB, may I know is there any reason for the same?
As other already said: Redundancy. You have RAID10 according to the output which uses 2 out of the 4 disks as copies, so that every bit is stored twice.
 

tom

Proxmox Staff Member
Staff member
Aug 29, 2006
15,525
911
163
May I know how to configure to sync 4 disks? any documentation?
Also, is this the reason for the disk only showing 4 TB instead of 8?
you will find links to docs in this thread already, read all the answers again.
 
  • Like
Reactions: LnxBil
Jan 18, 2020
167
1
23
27
HI,

Thank you for the information, I understood that there is two pair of disk for mirroring and stripping

May I check another thing, is the disks in Raid10 btrfs possible to replace? , after replicating the disk, will it rebalance automatically like OSDs?

Thank you.
 
Mar 25, 2022
74
19
8
May I check another thing, is the disks in Raid10 btrfs possible to replace? , after replicating the disk, will it rebalance automatically like OSDs?
Yes, you can replace RAID10 btrfs disks using standard procedures:
https://btrfs.wiki.kernel.org/index.php/Using_Btrfs_with_Multiple_Devices#Replacing_failed_devices

Use btrfs replace method. It is faster (rebalance would occure only once) and easier then old method (btrfs device add + btrfs device delete missing).
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get your own in 60 seconds.

Buy now!