Which would be the best way to configure SWAP memory with HD disks ZFS file root system?

BrandonN

Member
Mar 2, 2018
28
5
23
32
Colombia
Hello for everyone, I have an standalone node with 380 GB RAM and CPU usage is normal, 20 CT and 15 VM with Promox 6.3.6, I don't have IO delay. I just now I'm trying to configure a new CT, but I found that the container doesn't have SWAP memory (I did that setup with 2 Gb) so when I start to watch summary node information I found with RAM usage over 320 Gb and my SWAP usage as N/A.
I understand SWAP errors with the ZFS root system as mine has this filesystem, but how can I enable SWAP memory on my CTs? I did not present this error, additionally I think that the consumption of RAM is exaggerated, would someone have any recommendations please?

Thanks in advance.
 
Thanks @chrcoluk , do you know if is there another way? and what do you think about the RAM usage? as I can see, I did a setup with almost 150 Gb but RAM usage 320 - 330 Gb of 380 total.
 
Last edited:
Thanks @chrcoluk:

ARC status: HEALTHY
Memory throttle count: 0

ARC size (current): 95.5 % 180.4 GiB
Target size (adaptive): 100.0 % 188.9 GiB
Min size (hard limit): 6.2 % 11.8 GiB
Max size (high water): 16:1 188.9 GiB
Most Frequently Used (MFU) cache size: 93.9 % 161.2 GiB
Most Recently Used (MRU) cache size: 6.1 % 10.5 GiB
Metadata cache size (hard limit): 75.0 % 141.7 GiB
Metadata cache size (current): 8.9 % 12.6 GiB
Dnode cache size (hard limit): 10.0 % 14.2 GiB
Dnode cache size (current): 20.1 % 2.9 GiB

ARC hash breakdown:
Elements max: 24.2M
Elements current: 48.9 % 11.8M
Collisions: 228.7M
Chain max: 7
Chains: 926.4k

ARC misc:
Deleted: 92.7M
Mutex misses: 2.2k
Eviction skips: 56.6k

Additional, I was watching and reread the documentation and my zfs_arc_max and min aren't configured, I understood the default config uses 50% of available RAM as cache (this is useful to ZFS).
For more than 2 years I didn't have problems with ZFS just I want to be right with my whole system.
Thanks in advance for your help.
 
I dont know what to suggest for swap, I have read posts on here and proxmox documentation, as I understand it for zfs on root configurations (default), a swap is no longer created by default due to issues been reported using zfs based swap volumes.

What I personally did was add an extra partition on the end of both of my disks (I always consider good practice to leave some unpartitioned space in case needed in future), and set them up as a swap device via mdadm. It could also be done without mdadm, just adding them as individual swap devices.

If you have no unpartitioned space and you want swap, then you can either use zram (I am experimenting with this locally, I know this somehow prevents memory fragmentation based OOM without using disk based swap), or add a new disk, add a partition on that disk, and format that partition for swap using mkswap, and then finally adding it to fstab and enabling it via swapon.

The #1 on this page seems a reasonable guide. But I would mount using the label rather than the raw device, so it will still work if disk ordering is changed.

https://www.thegeekdiary.com/how-to-add-swap-space-in-linux/
 
Last edited:
  • Like
Reactions: BrandonN
I dont know what to suggest for swap, I have read posts on here and proxmox documentation, as I understand it for zfs on root configurations (default), a swap is no longer created by default due to issues been reported using zfs based swap volumes.

What I personally did was add an extra partition on the end of both of my disks (I always consider good practice to leave some unpartitioned space in case needed in future), and set them up as a swap device via mdadm. It could also be done without mdadm, just adding them as individual swap devices.

If you have no unpartitioned space and you want swap, then you can either use zram (I am experimenting with this locally, I know this somehow prevents memory fragmentation based OOM without using disk based swap), or add a new disk, add a partition on that disk, and format that partition for swap using mkswap, and then finally adding it to fstab and enabling it via swapon.

The #1 on this page seems a reasonable guide. But I would mount using the label rather than the raw device, so it will still work if disk ordering is changed.

https://www.thegeekdiary.com/how-to-add-swap-space-in-linux/
Thanks a lot, I will do this change the next week, I don't have any disk or unpartitioned space available right now so I will need to move some VM's and CT's disk and be able to use one as swap with zram.
 
I'm torn on this issue. One the one hand, I especially want checksums and redundancy (mirror of 2 or more) for my swap because I don't want the system to go down when a disk fails or there is an error (yes, I also use ECC memory). On the other hand, I don't know how to achieve this without ZFS, which can cause slowdowns (because of sync writes) or other problems with swap (and much write amplification). Are there alternatives that can give similar guarantees about data protection on disk (with swap)?
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!