Which would be the best way to configure SWAP memory with HD disks ZFS file root system?

[BrandonN]

Hello for everyone, I have an standalone node with 380 GB RAM and CPU usage is normal, 20 CT and 15 VM with Promox 6.3.6, I don't have IO delay. I just now I'm trying to configure a new CT, but I found that the container doesn't have SWAP memory (I did that setup with 2 Gb) so when I start to watch summary node information I found with RAM usage over 320 Gb and my SWAP usage as N/A.
I understand SWAP errors with the ZFS root system as mine has this filesystem, but how can I enable SWAP memory on my CTs? I did not present this error, additionally I think that the consumption of RAM is exaggerated, would someone have any recommendations please?

Thanks in advance.

 

[chrcoluk]

I set one up two days using 2 hdd's each with a native (not zfs) swap partition.

 

[BrandonN]

Thanks @chrcoluk , do you know if is there another way? and what do you think about the RAM usage? as I can see, I did a setup with almost 150 Gb but RAM usage 320 - 330 Gb of 380 total.

 

[chrcoluk]

in shell type

arc_summary -s arc

and post the output please.

 

[BrandonN]

Thanks @chrcoluk:

ARC status: HEALTHY
Memory throttle count: 0

ARC size (current): 95.5 % 180.4 GiB
Target size (adaptive): 100.0 % 188.9 GiB
Min size (hard limit): 6.2 % 11.8 GiB
Max size (high water): 16:1 188.9 GiB
Most Frequently Used (MFU) cache size: 93.9 % 161.2 GiB
Most Recently Used (MRU) cache size: 6.1 % 10.5 GiB
Metadata cache size (hard limit): 75.0 % 141.7 GiB
Metadata cache size (current): 8.9 % 12.6 GiB
Dnode cache size (hard limit): 10.0 % 14.2 GiB
Dnode cache size (current): 20.1 % 2.9 GiB

ARC hash breakdown:
Elements max: 24.2M
Elements current: 48.9 % 11.8M
Collisions: 228.7M
Chain max: 7
Chains: 926.4k

ARC misc:
Deleted: 92.7M
Mutex misses: 2.2k
Eviction skips: 56.6k

Additional, I was watching and reread the documentation and my zfs_arc_max and min aren't configured, I understood the default config uses 50% of available RAM as cache (this is useful to ZFS).
For more than 2 years I didn't have problems with ZFS just I want to be right with my whole system.
Thanks in advance for your help.

 

[chrcoluk]

Seems to add up reasonably well, your extra ram usage is on the ZFS ARC, which is the read cache for the ZFS filesystem. You are well within your actual ram constraints, but if it does bother you, you can reduce the maximum size of the ARC.

https://pve.proxmox.com/wiki/ZFS:_T...onfigurations_for_running_Proxmox_VE_with_ZFS

 

[BrandonN]

Seems to add up reasonably well, your extra ram usage is on the ZFS ARC, which is the read cache for the ZFS filesystem. You are well within your actual ram constraints, but if it does bother you, you can reduce the maximum size of the ARC.

https://pve.proxmox.com/wiki/ZFS:_T...onfigurations_for_running_Proxmox_VE_with_ZFS

Sure, thank you so much, and what about SWAP memory, I need to enable this for new CT installations. Add another normal disk?

 

[chrcoluk]

I dont know what to suggest for swap, I have read posts on here and proxmox documentation, as I understand it for zfs on root configurations (default), a swap is no longer created by default due to issues been reported using zfs based swap volumes.

What I personally did was add an extra partition on the end of both of my disks (I always consider good practice to leave some unpartitioned space in case needed in future), and set them up as a swap device via mdadm. It could also be done without mdadm, just adding them as individual swap devices.

If you have no unpartitioned space and you want swap, then you can either use zram (I am experimenting with this locally, I know this somehow prevents memory fragmentation based OOM without using disk based swap), or add a new disk, add a partition on that disk, and format that partition for swap using mkswap, and then finally adding it to fstab and enabling it via swapon.

The #1 on this page seems a reasonable guide. But I would mount using the label rather than the raw device, so it will still work if disk ordering is changed.

https://www.thegeekdiary.com/how-to-add-swap-space-in-linux/

 

[BrandonN]

I dont know what to suggest for swap, I have read posts on here and proxmox documentation, as I understand it for zfs on root configurations (default), a swap is no longer created by default due to issues been reported using zfs based swap volumes.

What I personally did was add an extra partition on the end of both of my disks (I always consider good practice to leave some unpartitioned space in case needed in future), and set them up as a swap device via mdadm. It could also be done without mdadm, just adding them as individual swap devices.

If you have no unpartitioned space and you want swap, then you can either use zram (I am experimenting with this locally, I know this somehow prevents memory fragmentation based OOM without using disk based swap), or add a new disk, add a partition on that disk, and format that partition for swap using mkswap, and then finally adding it to fstab and enabling it via swapon.

The #1 on this page seems a reasonable guide. But I would mount using the label rather than the raw device, so it will still work if disk ordering is changed.

https://www.thegeekdiary.com/how-to-add-swap-space-in-linux/

Thanks a lot, I will do this change the next week, I don't have any disk or unpartitioned space available right now so I will need to move some VM's and CT's disk and be able to use one as swap with zram.

 

[leesteken]

I'm torn on this issue. One the one hand, I especially want checksums and redundancy (mirror of 2 or more) for my swap because I don't want the system to go down when a disk fails or there is an error (yes, I also use ECC memory). On the other hand, I don't know how to achieve this without ZFS, which can cause slowdowns (because of sync writes) or other problems with swap (and much write amplification). Are there alternatives that can give similar guarantees about data protection on disk (with swap)?

 

[BrandonN]

Hello for everyone.

I make the upgrade and all setup to run and use PM 7.x. Right now this is the result.