Random read slowdown of NVMe disk with kernel version 6.2.16-19-pve vs 6.2.16-3-pve

[Tabalugach]

I am noticing poor performance with NVMe disk :

Random read slowdown of NVMe disk with kernel version 6.2.16-19-pve vs 6.2.16-3-pve

fio --ioengine=io_uring --filename=/dev/nvme0n1 --direct=1 --rw=randread --bs=4K -numjobs=1 --iodepth=8 --runtime=10 --time_based --name=fio
6.2.16-19-pve :
Run status group 0 (all jobs): READ: bw=996MiB/s (1044MB/s), 996MiB/s-996MiB/s (1044MB/s-1044MB/s), io=9960MiB (10.4GB), run=10001-10001msec
6.2.16-3-pve :
Run status group 0 (all jobs): READ: bw=1573MiB/s (1650MB/s), 1573MiB/s-1573MiB/s (1650MB/s-1650MB/s), io=15.4GiB (16.5GB), run=10001-10001msec

NVMe disk : Samsung SSD 970 EVO Plus 500GB
CPU : AMD Ryzen 5700G
MB : ROG STRIX B550-F GAMING

No VM up during testing.

 

[fiona]

Hi,
10 seconds might be a bit short. How do the results look like when you let it run for a minute? Is the regression still there when you try the 6.5 kernel: https://forum.proxmox.com/threads/o...-2-for-proxmox-ve-8-available-on-test.135635/ ?
Otherwise, please try to narrow it down further by testing other kernel versions between 6.2.16-19-pve and 6.2.16-3-pve?