rsync and ZFS crash hypervisor

AndreasR

New Member
Dec 20, 2020
2
0
1
46
Greetings
I am experimenting with proxmox to see if it can be an alternative/replacement for vmware and so far I am impressed with the features and have one thing that is weird.
When I rsync a lot of files to a (ubuntu) vm with a disk that is formatted ZFS (single disk) the memory runs full in 5 minutes and the hypervisor reboots.

When the disk is formatted+added as LVM it works - I have now rsync'd 2 TB of data and no problems.
( I played around with rsync parameters, used SCP and CIFS (just to eliminate SSH) but either crashes proxmox when using rsync+zfs disk)

The problem seems either related to ZFS or ubuntu or rsync. (in that combination)
searching in the forums this indicates a cache overflow / not enough IOPS but I wonder if I can help trace this. Is this expected behavior?

So far I am not able to confirm the awesomeness of ZFS but then I am using a single disk in ZFS... no cache, just 8 GB of RAM (should be enough, no?)

If I can provide any traces / reproduce the problem le me know.

regards


Andreas


Dec 19 22:01:38 plex kernel: Key type cifs.idmap registered Dec 19 22:01:38 plex kernel: CIFS: Attempting to mount //192.168.42.206/plex Dec 19 22:01:38 plex kernel: No dialect specified on mount. Default has changed to a more secure dialect, SMB2.1 or> >>> start rsync job <<< Dec 19 22:02:50 plex Plex Media Server[768]: Connection to 82.94.168.38 closed by remote host. Dec 19 22:05:34 plex PackageKit[1179]: daemon quit Dec 19 22:05:34 plex systemd[1]: packagekit.service: Succeeded. -- Reboot -- Dec 19 22:08:07 plex kernel: Linux version 5.4.0-58-generic (buildd@lcy01-amd64-004) (gcc version 9.3.0 (Ubuntu 9.3> pported cpus:
same happens with rsync over SSH - both directions (server-client (push) and client-server (pull))
 
Hello Andreas,
Have you limited your ZFS RAM-usage?
By default it is allowed to use up to 50℅ of your memory.
That could lead to memory exhaustion and trigger a host restart.
If not: check for "ZFS arc limit configuration" here in the forum. There are plenty of topics about it.
 
thanks for reaching out. well.. I have 16G of RAM and allocated 8 to this guest.

is there a way I can monitor/check this? the reload is instant and no warning appears
 
Search for "zfs arc monitor" or "zfs arc usage" here in the forum or web. There have been plenty of discussions about that. Don't know of the top of my head and writing from mobile ;)
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!