I have 3 different nodes that I'm messing with. All 3 nodes are the same (servers, CPU, RAM, HBAs, etc),
They all use ZFS. Was actually upgrading them to PVE6. And I was doing some testing and can't understand whats going on with the pveperf FSYNC/SEC results. My understanding is that FSYNC with ZFS is handled by the ZIL, for which you can dedicate a SLOG(s) to handle if needed. So if you have a SLOG, the backing pool shouldn't matter. Without a SLOG the sync writes are written to the ZIL on the pool.
With that in mind, I'm seeing HALF the performance between Node 1 and Node 3. they both using the same SLOG device though, an Intel DC S3710 400GB. The only differences between these 2 nodes are;
Condensed view of results;
Code:
Intel R2224GZ4GC4 (barebones)
Intel S2600GZ (motherboard)
Intel E5-2670 x 2 (CPUs)
Hynix HMT31GR7CFR4A-H9 (8GB x 16) (RAM)
LSI 9340-8i (SAS3008 flashed to IT mode)They all use ZFS. Was actually upgrading them to PVE6. And I was doing some testing and can't understand whats going on with the pveperf FSYNC/SEC results. My understanding is that FSYNC with ZFS is handled by the ZIL, for which you can dedicate a SLOG(s) to handle if needed. So if you have a SLOG, the backing pool shouldn't matter. Without a SLOG the sync writes are written to the ZIL on the pool.
With that in mind, I'm seeing HALF the performance between Node 1 and Node 3. they both using the same SLOG device though, an Intel DC S3710 400GB. The only differences between these 2 nodes are;
- the backing disks (node 1 has more of them) but my understanding is this doesn't matter
- Node 1 I provisioned as 2 pools (rpool and tank) instead of the default manner on Node 3 (just rpool)
- Node 1 is running PVE 5.4 and Node 3 is running 6
- Node 1 is currently not hosting any VM's while Node 3 is hosting 2 small VMs
Condensed view of results;
| Node ID | PVE | ZFS version | Disk Layout | Notes | FSYNC/SEC |
| 1 | 5.4-13/aee6f0ec (running kernel: 4.13.4-1-pve) | 0.7.13-pve1~bpo2 | Intel DC 3710 SLOG + 7 mirrored vdevs (10K SAS) | 2 pools (rpool & tank) No running VMs | 4272.08 |
| 2 | 6.0-7/28984024 (running kernel: 5.0.21-1-pve) | 0.8.1-pve2 | 3 mirrored vdevs (Samsung PM853T) | 1 pool (rpool) running VMs | 2213.35 |
| 3 | 6.0-7/28984024 (running kernel: 5.0.21-1-pve) | 0.8.1-pve2 | Intel DC 3710 SLOG + 4 mirroed vdevs (10K SAS) | 1 pool (rpool) running VMs | 1816.79 |
Yuck