That did not make much difference but "pci=pcie_bus_perf" this. For future reference to others:
1- CPU Type: Host and NUMA enabled
2- Check the nvme afinity and put it in CPU Affinity under CPU settings (---for d in /sys/bus/pci/devices/*/nvme...
It is on q35 otherwise it dos not let me do pcie option on the passthrough. this last change did it.
Read Test:
Jobs: 32 (f=32): [R(32)][2.5%][r=9684MiB/s][r=2479k IOPS][eta 01m:57s]
Jobs: 32 (f=32): [R(32)][4.2%][r=9630MiB/s][r=2465k IOPS][eta...
Here is the ouput for the affinity:
root@test1:~# for d in /sys/bus/pci/devices/*/nvme; do echo "$(dirname $d): NUMA $(cat $(dirname $d)/numa_node)"; done
/sys/bus/pci/devices/0000:c1:00.0: NUMA 0
/sys/bus/pci/devices/0000:c3:00.0: NUMA 0
I am...
DId you ever solve this problem? I am in the same boat, I have a PM1753 NVME disk with 600K write and 3.3M read IOPS, if I test it within proxmox itself with fio I get those IOPS, either as a rad device or mounted as XFS but if I do a passthrough...