Hello there,
I have a Z440 workstation that I've been using as a Proxmox server for over a year. It has 192 GiBs of ECC RAM and an Intel Xeon CPU E5-2687W v4. Proxmox is installed on an Intel DC P3700 PCI-e SSD. The main storage pool is a raid z1 with 5 identical 4 TB Iron Wolf HDDs. The pool is named planb and I've made three virtual disks sized 3TB, 3TB and 1 TB with proxmox and passed them to an Open Media Vault (OMV) virtual machine. The pool also has an Intel Optane ZIL SLOG device (16GB) and a L2ARC samsung EVO 850 SSD.
The virtual disks are used to host (Windows) backups via Samba and the data is saved elsewhere, so no worries. The Proxmox is always on and well ventilated.
During the past year I've gotten excellent speeds from the pool. We have a 10 Gbe ethernet LAN network at home. Yesterday I updated proxmox from 7.1-5 to 7.2-3 via browser GUI and everything went well. Only after a reboot, the planb pool write speeds dropped from >1Gbps to between 0.5 and 10 Mbps. Read speeds are still consistent and good at >120 MB/s with something that wasn't in the L2ARC and close to 10 Gbps speeds with something that is in the ARC.
So far I've tried
so I'm able to see the IO wait times and such stuff from the pool. Usually there's nothing going on even if there's a write event, but every five seconds or so the stats indicate that something is written when there's a transfer going on in the background.
Here's a picture of the output when I'm transfering a big file from planb pool via Samba to my Windows 10 desktop.
Here's a picture of me copying an ISO file to test write speeds
Here's a picture of when I pause the transfer and resume it some 7 minutes later. It goes all the way to the end, but doesn't finish unless I wait for the original estimated time - then the file is written and usable.
Here's the same thing with sync disabled
And here we are with the log device removed:
And again with the log device re-added:
I have a Z440 workstation that I've been using as a Proxmox server for over a year. It has 192 GiBs of ECC RAM and an Intel Xeon CPU E5-2687W v4. Proxmox is installed on an Intel DC P3700 PCI-e SSD. The main storage pool is a raid z1 with 5 identical 4 TB Iron Wolf HDDs. The pool is named planb and I've made three virtual disks sized 3TB, 3TB and 1 TB with proxmox and passed them to an Open Media Vault (OMV) virtual machine. The pool also has an Intel Optane ZIL SLOG device (16GB) and a L2ARC samsung EVO 850 SSD.
The virtual disks are used to host (Windows) backups via Samba and the data is saved elsewhere, so no worries. The Proxmox is always on and well ventilated.
During the past year I've gotten excellent speeds from the pool. We have a 10 Gbe ethernet LAN network at home. Yesterday I updated proxmox from 7.1-5 to 7.2-3 via browser GUI and everything went well. Only after a reboot, the planb pool write speeds dropped from >1Gbps to between 0.5 and 10 Mbps. Read speeds are still consistent and good at >120 MB/s with something that wasn't in the L2ARC and close to 10 Gbps speeds with something that is in the ARC.
So far I've tried
- using a search engine to find similar issues and possible fixes
- disabling sync from the pool planb (no change)
- creating another VM with a samba share in planb pool to test if this is an issue with OMV (the write speed remains the same)
- removing the Optane ZIL SLOG (no noticeable change in write speeds)
- a SMART test for all the disks (around 8000 of power on hours, so just under a year as it should be)
- a scrub (no change)
- zpool trim planb (no change)
watch -n 1 zpool iostat -vly 1 1so I'm able to see the IO wait times and such stuff from the pool. Usually there's nothing going on even if there's a write event, but every five seconds or so the stats indicate that something is written when there's a transfer going on in the background.
Here's a picture of the output when I'm transfering a big file from planb pool via Samba to my Windows 10 desktop.
Here's a picture of me copying an ISO file to test write speeds
Here's a picture of when I pause the transfer and resume it some 7 minutes later. It goes all the way to the end, but doesn't finish unless I wait for the original estimated time - then the file is written and usable.
Here's the same thing with sync disabled
And here we are with the log device removed:
And again with the log device re-added: