server crashes with no logs when copying bigger files

[PiotrDev]

When I try to copy 5-10GB of files, in the middle server crashes. I logged in to remote console but don't see anything on screen
When I copy files with bandwith limit set to 15000kb/s then no problem even copying 20GB of data
any suggestion what can I check/change?

configuration:
hpe microserver gen10 plus
2 hdd + 2 ssd drives
Intel(R) Pentium(R) Gold G5420 CPU @ 3.80GHz
180W psu
8GB memory (may this be that big of a problem? since filesystem is zfs? )

root@pve:~# pveversion
pve-manager/7.0-8/b1dbf562 (running kernel: 5.11.22-1-pve)

 

[Dunuin]

By default ZFS will use up to 50% of your hosts RAM. And rule of thumb for ZFS is 4GB + 1GB per 1TB of raw storage for its ARC. And PVE needs 1-2GB RAM too. So yes, that low RAM can really be the problem. So you actually only got like 2GB for guests,

 

[PiotrDev]

By default ZFS will use up to 50% of your hosts RAM. And rule of thumb for ZFS is 4GB + 1GB per 1TB of raw storage for its ARC. And PVE needs 1-2GB RAM too. So yes, that low RAM can really be the problem. So you actually only got like 2GB for guests,

I have 2 x 4TB hdd + 2 x 2TB SSD
any way to confirm if it's RAM issie before upgrading it?
disconnect hdds and just lave 2x 2TB SSD that proxmox is installed on?

So just for host I need 4GB + 4GB (4TB hdds) + 2GB (2TB ssds) + 1-2GB for PVE = 11-12GB just to run system, so at least 16 assuming I want 1-2 small VMs.

 

[PiotrDev]

so with only SSD disks (hdd I have directly attached to vm, so can easily shutdown that vm and disconnect them) this memory should be enough: 4GB + 2GB + 1-2GB. I will test disconnecting hdds and test copying some files.

 

[Dunuin]

I have 2 x 4TB hdd + 2 x 2TB SSD
any way to confirm if it's RAM issie before upgrading it?
disconnect hdds and just lave 2x 2TB SSD that proxmox is installed on?

So just for host I need 4GB + 4GB (4TB hdds) + 2GB (2TB ssds) + 1-2GB for PVE = 11-12GB just to run system, so at least 16 assuming I want 1-2 small VMs.

No, its raw storage. So 2GB for PVE + 16GB RAM for ZFS (4GB + 8GB for the HDDs + 4GB for the SSDs) so 18GB RAM would be good for the host. ZFS will run with less RAM, but 16GB would be the recommended ARC size for your setup for good performance. By default your ARC is set to 50% so right now ZFS shouldn't use more than 4GB RAM.

Are you using ZFS deduplication? In that case rule of thumb is 4GB + 5GB per 1TB of raw storage so 74GB RAM for ZFS for your setup instead of 16GB.

You should check the syslog for OOM logs. If your host is running out of RAM you should see killed processes there.

 

[PiotrDev]

No, its raw storage.

Oh, then I can't even test with 2x2TB ssd to be sure it's memory.
I ordered server with 16GB, shop sent me with 8GB and I agreed to get partial refund so now need to look for additional ram. Hopefully I can buy 16GB and install it with current 8GB stick to work together

 

[Dunuin]

By the way...did you also check syslog.1 for logs?

You might also want to monitor RAM and ARC by using arc_summary and free -h.