Proxmox 6 on home NAS - SLOOOWWWW

[SamTzu]

I decided to test Proxmox 6 on my home PC where I installed 4 HD (8Tb) that I setup with RAIDZ1.
From the start the speed of the raid set was just terrible but I managed to speed it up somewhat by using "zfs set sync=disabled rpool".

Now I'm testing other methods to speed up the setup. Maybe adding a SDD drive to helpout ZFS?

Any ideas?

 

[joshin]

How much memory on the computer? For a read-mostly workload on spinning-rust drives, a good sized ARC is necessary.

 

[SamTzu]

I gave 8Gb of the 16 to ZFS.

 

[joshin]

Depending on its write performance, that SSD might be okay for the Intent Log (more or less write cache). But you probably need to reset your expectations because, you're not going to see good performance with that system.

Do some IOPS calculations and you'll see why. Especially look at write performance.

If you want decent performance, try mirrored vdevs with the SSD (if decent write performance).

On the plus side, it will be a good learning experience for tuning ZFS and Linux.

 

[SamTzu]

It's strange. After reboot the data transfer speeds to ZFS storage is acceptable 80Mbs but after couple of hours it drops to 2Mbs. I'm not quite sure if it's a storage or network issue.

 

[joshin]

I'd guess memory contention.

Turn off ksmd or at least do a:
echo 2 > /sys/kernel/mm/ksm/run

 

[SamTzu]

Stopping ksmdtuned service running echo and "zfs set sync=disabled rpool" don't seem to help any after the system has been up couple of hours.
==> Me scratching my head!

 

[SamTzu]

The problem appears to be zfs_arc_max.
Every boot seems to reset the amount of memory I have reserved to ZFS to 0.
This command sets available memory for ZFS back to 8Gb and speeds up the storage considerably.

echo 8000000000 > /sys/module/zfs/parameters/zfs_arc_max

 

[SamTzu]

Hmm. Speed is accaptable now but I think I somehow managed to ruin systemd-timesyncd because I can't log in anymore. How do I "reset" Proxmox 6 date without installing NTP?

 

[BobhWasatch]

date -s '05 Nov 2019 11:09:00 AM'

 

[joshin]

Check out this section of the wiki -> Limit ZFS Memory Usage

https://pve.proxmox.com/wiki/ZFS_on_Linux

Hmm. Speed is accaptable now but I think I somehow managed to ruin systemd-timesyncd because I can't log in anymore. How do I "reset" Proxmox 6 date without installing NTP?

 

[SamTzu]

Turns out this had nothing to do with ZFS. It's IRQ/ACPI problem on old motherboard BIOS that causes "kernel: irq 18: nobody cared (try booting with the "irqpoll" option)" error message. After that the HD's just slow to crawl. Since there are no updates to the BIOS my attempt to turn this old HardWare to home NAS system seems doomed.