[SOLVED] BTFRS Swap-Partition

itNGO

Renowned Member
Jun 12, 2020
794
181
68
46
Germany
it-ngo.com
I have 3 PVE-Nodes (8.02).
2 of them where installed some weeks ago. I am pretty sure that during Setup, BTFRS Raid-1 I had the option to create SWAP which I did, as I have 32GB SWAP-Partition on both nodes.

Now I am in Progress to Install 3rd node and nowhere in BTRFS-RAID-1 Setup during the installer I get the Option to define SWAP-Size?
What I am missing? ISO for setup is still the same.
 
Never mind.... Problem in front of Screen....
 
  • Like
Reactions: GastonJ
I'm still looking for a way to mirror my swap as ZFS and mdraid shouldn't ne used for that. Could you have a look how PVE is setting up the swap when using btrfs? Is that swap mirrored too?
No it does not. And indeed it was my mistake... the other two nodes have m2-Raid-1 and the third one has just 2 SATA-Disks which I put in Mirror now.

I left some space on both SSDs and created 2 SWAP-Partitions with same Prio in FSTAB.
 
  • Like
Reactions: Dunuin
I left some space on both SSDs and created 2 SWAP-Partitions with same Prio in FSTAB.
Yes, this is how I do it right now. But defeats a bit the benefits of a raid1 if the server will still crash in case any of the two disks fails, as then the swapped out data will be lost, that is needed for operation.
 
Last edited:
Yes, this is how I do it right now. But defeats a bit the benefits of a raid1 if the server will still crash in case any of the two disks fails, as then the swapped out data will be lost, that is needed for operation.
Our idea is to have at least some sort of "hook" when a disk is in pre-fail condition, like SMART-Failure or ZFS-Write-Error/CRC-Counter increases to "disable" swap by remote-shell-command on that partition. We still have to figure out if BTRFS has the same "information" available about write errors. We use Zabbix to catch all that data and let it "react" with script.

However, this will not help if a disk "suddenly" fails complete. But at least some failures can be "detected" early enough to have no data-loss from SWAP.

The alternative would be, to use a "SWAPFILE" on the Mirror, but I guess that will make "other" problems....
 
please consider the real pupose of swap. it is not for extra ram, it is not as emergency ram (even tough it works as such so system doesnt lock up and kernel oom has time to clean up to prevent a freeze)

But main purpose is memory organisation and its incredible important for it.
without it we cant page out anonymous pages as one of many examples where we need it.

what happens if swap is dead?
well not that much actually. systemcrashes are highly unlikely. However it crashes anything that reads and writes to it.

This is a problem on a proxmox host, with huge memory chunks from the VMs.
and i agree swap on zfs is bad, swap on non fault tollerant disks is bad too.

it may not crash the entire host but it may as well corrupt an entire vm.

another thing is vm.swappiness - this is not what people think it is and not what proxmox documented.
it does not reduce or increase swapping

it decides the ratio between file_prio and anon_prio
file_prio: reclaim file pages
anon_prio: reclaim anonymous pages
Default is 200 file_prio minus swappiness for anony.prio

example: vm.swappiness=10 equals 190 file_prio and 10 anon.prio

so its just a cost calculation. if you have nvmes as swap a swappiness of 100 can be incredible beneficial if you also have a lot of unlcaimed pages.
yea i know contrary to the proxmox documentation



now the issue is fault tollerance. we all know zfs is a nogo, mdraid seems to be an issue too.
btrfs i wouldnt trust it.

i see here 3 possible options:


#1 zram with enough ram, run at least 8gb or more
#2 hardware raid for pure swap disks
#3 a combination of both

iam currently also exploring zswap as its a combination of classic swap and zram, but i have my doubts about fault tollerance.
 
  • Like
Reactions: itNGO and Dunuin

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!