swap on proxmox server ?
- Thread starter RolandK
- Start date
I use ZRAM myself: https://pve.proxmox.com/wiki/Zram#Alternative_Setup_using_zram-tools
You can also leave some space unallocated on the disk and create a SWAP partition or a SWAP file on a ext4 file system created on it.
You can also leave some space unallocated on the disk and create a SWAP partition or a SWAP file on a ext4 file system created on it.
you have no redundancy with simple ext4, if your disk crashes and pages are swapped out, your server crashes. that's why i wrote "rundandant disk".
nowadays, typically zfs is being used and using mdraid is discouraged with proxmox
so i wonder what are the options, i.e. what's the right way....
nowadays, typically zfs is being used and using mdraid is discouraged with proxmox
so i wonder what are the options, i.e. what's the right way....
Last edited:
that's at least better then nothing, but is no real redundancy but more a comfortability switch, as you won't need maintenance on the machine on bootup when one swap device fails.
what about zram for swap alternative?
it is discouraged here https://pve.proxmox.com/wiki/Zram
but in zfs community it is currently recommended as alternative to swap on zvol : https://github.com/openzfs/zfs/issues/7734#issuecomment-3723591151
it is discouraged here https://pve.proxmox.com/wiki/Zram
but in zfs community it is currently recommended as alternative to swap on zvol : https://github.com/openzfs/zfs/issues/7734#issuecomment-3723591151
zram is useful if you have ~16GB+ RAM, then you can give it a couple of gigs and it will compress.
For rpool servers I reserve a couple of gigs at the end of the disk(s) and make swap partitions manually postinstall with gdisk.
Don't forget if you have spare slots, you can have a dedicated disk for swap. Even USB3 external SSD will work since the whole point of a server is to have max RAM usage and minimal swapping.
For rpool servers I reserve a couple of gigs at the end of the disk(s) and make swap partitions manually postinstall with gdisk.
Don't forget if you have spare slots, you can have a dedicated disk for swap. Even USB3 external SSD will work since the whole point of a server is to have max RAM usage and minimal swapping.
But then you don't have checksums or redundancy, which was the main point of the first post.
Maybe swap on a BTRFS mirror? But the tutorials mention that the swap would be NODATACOW which also disables checksums (NODATASUM), but at least some redundancy. Then you cannot tell which copy is correct but you can probably survive a missing or fully broken drive (but not a silently corrupted or partially broken one?).
Maybe swap on a BTRFS mirror? But the tutorials mention that the swap would be NODATACOW which also disables checksums (NODATASUM), but at least some redundancy. Then you cannot tell which copy is correct but you can probably survive a missing or fully broken drive (but not a silently corrupted or partially broken one?).
Last edited:
> what is the recommended proxmoy (sic) way to go to have swap on servers with requirement for redundant disk (i.e. raid storage) ?
You don't need "checksums or redundancy" for swap, that makes little sense. If the server has a requirement for RAID and comes with a RAID card, you can just set up ext4/lvm rootfs with hardware RAID6 or mirror, and use the installer to reserve some disk space for swap.
If using ZFS on HBA, then you (like I said) can reserve a bit of disk space at the end and make swap postinstall, or put swap on separate dedicated disk that aren't part of the ZFS pool.
You don't need "checksums or redundancy" for swap, that makes little sense. If the server has a requirement for RAID and comes with a RAID card, you can just set up ext4/lvm rootfs with hardware RAID6 or mirror, and use the installer to reserve some disk space for swap.
If using ZFS on HBA, then you (like I said) can reserve a bit of disk space at the end and make swap postinstall, or put swap on separate dedicated disk that aren't part of the ZFS pool.
>You don't need "checksums or redundancy" for swap, that makes little sense.
huh?
>If using ZFS on HBA, then you (like I said) can reserve a bit of disk space at the end and make swap postinstall,
>or put swap on separate dedicated disk that aren't part of the ZFS pool.
and if you loose the disk where the swap is on , if writes or reads will fail or read will return garbage - your system will crash
huh?
>If using ZFS on HBA, then you (like I said) can reserve a bit of disk space at the end and make swap postinstall,
>or put swap on separate dedicated disk that aren't part of the ZFS pool.
and if you loose the disk where the swap is on , if writes or reads will fail or read will return garbage - your system will crash
Last edited:
I don't really see your point, that has been the case since swap was invented (probably back in the 60's.) Any Linux system can have multiple swap partitions in addition to swap files. Or NO swap - although for Linux at least, a minimal swap amount is recommended for housekeeping.
Putting swap on hardware RAID may help to a certain extent, but I've never heard of a server failing because "the disk where the swap was housed failed" - it's usually the root filesystem getting corrupted.
Haven't tested it, but if you have more than 1 swap defined and take out a swap space entirely (fail the disk) you MAY still be able to recover the system if it hasn't kernel panic'ed by simply doing a swapoff on that extent. Part of it depends on how much swap was actually in use; modern sysadmins go for 0% to minimal usage. RAM was fairly cheap until last November, and 64GB+ systems aren't uncommon anymore.
EDIT: Do a search on " does linux kernel swap itself out " - kernel prioritizes swapping out USER space stuff and tries to keep running-the-system critical stuff in RAM.
Putting swap on hardware RAID may help to a certain extent, but I've never heard of a server failing because "the disk where the swap was housed failed" - it's usually the root filesystem getting corrupted.
Haven't tested it, but if you have more than 1 swap defined and take out a swap space entirely (fail the disk) you MAY still be able to recover the system if it hasn't kernel panic'ed by simply doing a swapoff on that extent. Part of it depends on how much swap was actually in use; modern sysadmins go for 0% to minimal usage. RAM was fairly cheap until last November, and 64GB+ systems aren't uncommon anymore.
EDIT: Do a search on " does linux kernel swap itself out " - kernel prioritizes swapping out USER space stuff and tries to keep running-the-system critical stuff in RAM.
Last edited:
Turns out that a swapfile on BTRFS is possible but only when using single data copy without checksums, which is too fragile IMHO.
Looks like the only (supported) way to put swap on a mirrored and check-summed storage is to create a VM (with fixed and pinned memory) on a mirrored and check-summed storage, create a file share, expose that to the Proxmox host (using NFS and loop) and put a swapfile on that?
Looks like the only (supported) way to put swap on a mirrored and check-summed storage is to create a VM (with fixed and pinned memory) on a mirrored and check-summed storage, create a file share, expose that to the Proxmox host (using NFS and loop) and put a swapfile on that?
Isn't this the same as putting a swapfile on btrfs or zfs directly but with extrasteps so not supported? And way more fragile due to the NFS layer between?I wouldn't see some hack from a raspberry forum as "supported".
I'm not too worried about the virtual network but I agree that it feels flimsy. mkswap and swapon don't accept files on BTRFS for good reason and the system might need additional memory to write to swap (causing a deadlock on ZFS also). By using a VM with pinned memory (like PCI passthrough) you reserve memory to do this safely. Not sure about CoW of the underlying storage, though. And if I passthroug the drives to the VM, I do indeed end up with the same issue but nested... maybe I'm overthinking this.
It just does not feel right to me to have ECC RAM and put swap on a drive without check-sums (and preferably redundancy to fix detected errors). mdadm can give redundancy (but not check-sums, so how would it know which copy is correct?) and is also not supported.
Good question, it made me do a short googling. Following older stackoverflow thread from the results has some interesting points:
https://unix.stackexchange.com/questions/269098/silent-disk-errors-and-reliability-of-linux-swap
The basic argument of the top answer is, that discs have also internal checks which should be sufficient for short-lifetime-data like swap compared to long-time-storage on regular filesystems. The actual answer is longer and more nuanced though so best to read for your self and decide whether you agree or not
https://unix.stackexchange.com/questions/269098/silent-disk-errors-and-reliability-of-linux-swap
The basic argument of the top answer is, that discs have also internal checks which should be sufficient for short-lifetime-data like swap compared to long-time-storage on regular filesystems. The actual answer is longer and more nuanced though so best to read for your self and decide whether you agree or not
Yes, it's probably not necessary in practice but it remains ironic: that which makes check-summing redundant filesystems perfect for swap is exactly what makes them impossible to use for swap.
I guess a md-raid1 of multiple md-integrity (without crash resistance) with pre-allocated memory would work. But then a small zram (which I have now) would be much simpler, as you should not over-commit memory on Proxmox anyway. EDIT: And at least with zram, you don't have to worry about vm.swappiness.
I guess a md-raid1 of multiple md-integrity (without crash resistance) with pre-allocated memory would work. But then a small zram (which I have now) would be much simpler, as you should not over-commit memory on Proxmox anyway. EDIT: And at least with zram, you don't have to worry about vm.swappiness.
Last edited:
That's going straight to crazy town, do not pass Go, do not collect $200. Trying to do the above a) introduces a S L O W - and needlessly complex - path for swap, and b) puts you at risk if/when the VM gets powered down, nicely or not, without doing a swapoff first. Rube Goldberg says Hi and sends thanks for the idea, btw.
If you want swap on RAID, one easy way is to buy a hardware-raid external disk enclosure and mirror 2 disks / small SSDs.
(amzn) " CENMATE Aluminum 2 Bay Hard Drive RAID Enclosure with Cooling Fan for 2.5“/3.5" SATA HDD/SSD with USB A/C 3.0, Tool-Free HDD Enclosure, 4 Modes " - under $60 at time of writing; not a recommendation, just an example
Last edited: