2 drive LVM pool formatted with BTRFS passed to LXC container setup

[alishaded]

I've spent countless hours on this and I'm reaching a dead end so thought I'd turn here for some experienced and practical advice.

I've just setup Proxmox 8.1.3 on my homelab server running an Intel N100, 16gb RAM, a 256gb SATA ssd for OS and VM/LXC use and 3 3TG SATA spinning hard drives.

I'm trying to setup 2 of the drives in an LVM pool and then pass those combined drives as a single 6TB "drive" to an LXC container running docker for media storage use formatted with btrfs to take advantage of compression. I don't need redundancy at all and I don't need the pool to be backed up.

I used this guide: https://help.nodespace.com/knowledgebase.php?article=307 to group /dev/sdb & /dev/sdc (the two drives I want to "combine" and then have added a mount point to this storage under /media in the LXC config using the GUI.

What I'm unclear on is how to handle the storage either inside the container as it's mounted to /media and outside the container on the Proxmox level where it appears to be available as /dev/media/vm-100-disk-0 ...

I previously used "defaults,noatime,space_cache=v2,compress=zstd:3" as mount options when I had this all set up on Ubuntu but without using an LVM pool as various guides said that compression can help performance and obviously somewhat reduce storage use although thats not a priority.

My question is, what's the best or correct way to setup what I've described?

The storage pool "Format" in Proxmox "media" is specified as "raw" which is then I assume passed directly to the LXC container. I've successfully managed to format the LVM pool with btrfs in the Proxmox command line (it defaulted to ext4) but I'm not sure how to mount it inside the LXC container with the mount options mentioned above? fstab doesn't seem to apply either inside or outside the container since it's passed into the LXC "raw" and it isn't really mounted outside the container as far as I can tell?

What's the correct why to set this up? Would btrfs compression or other options be handled inside or outside the LXC container? How are they set?

 

[lildadou]

Proxmox "does not accept" btrfs natively. When the document claims to be able to use btrfs, it means that it is able to create a file in a btrfs and that this file will be the block device containing another fs, mounted in your container. (This was a big disappointment for me; I was expecting elegant use of subvolumes).

To answer your question: everything "btrfs" must be on the hypervisor side.

1. Create your btrfs fs in your hypervisor,
2. mount the fs,
3. create a sub-volume for your container,
4. mount the sub-volume,
5. edit the container configuration (no GUI available; you have to get your hands into /etc/pve/lxc/xxx.conf) and add a "bind mount" (https://pve.proxmox.com/wiki/Linux_Container#_bind_mount_points).

Note that it is not possible to have a btrfs-formatted rootfs directly on top of a device.

 

[alishaded]

Proxmox "does not accept" btrfs natively. When the document claims to be able to use btrfs, it means that it is able to create a file in a btrfs and that this file will be the block device containing another fs, mounted in your container. (This was a big disappointment for me; I was expecting elegant use of subvolumes).

To answer your question: everything "btrfs" must be on the hypervisor side.

1. Create your btrfs fs in your hypervisor,
2. mount the fs,
3. create a sub-volume for your container,
4. mount the sub-volume,
5. edit the container configuration (no GUI available; you have to get your hands into /etc/pve/lxc/xxx.conf) and add a "bind mount" (https://pve.proxmox.com/wiki/Linux_Container#_bind_mount_points).

Note that it is not possible to have a btrfs-formatted rootfs directly on top of a device.

Thanks for the reply. To clarify, I used makefs to format the LVM pool: mkfs.btrfs -f /dev/media/vm-100-disk-0

So the pool is formatted with btrfs and I confirmed this inside the LXC container:

root@docker:~# btrfs filesystem df /media
Data, single: total=8.00MiB, used=0.00B
System, DUP: total=8.00MiB, used=16.00KiB
Metadata, DUP: total=1.00GiB, used=128.00KiB
GlobalReserve, single: total=5.50MiB, used=0.00B
root@docker:~#

I think I understand your reply but since I don't need subvolumes can't I just specify/set the btrs options somehow (defaults,noatime,space_cache=v2,compress=zstd:3) or does that require subvolumes? It seems I'm almost there. I mean I can currently use the btrfs storage it looks like. I just now need to set those options. Are they set inside the container or outside?

 

[lildadou]

I'd forgotten that you could prepare rootfs before creating the container and therefore have rootfs in btrfs. Well done.

No, of course you don't have to use subvolumes. It's just that it seemed obvious to use them in the scenario I presented (the hypervisor "owns" the btrfs and slices it up between the different containers). The downside is that you can no longer decide on mounting options except lazytime, noatime and nosuid.