Hi everyone，lm testing a new PVE cluster(8.4.13,4 hosts) now and facing a problem. l've already create ceph with 14 osd (6ssd+8hdd).After creating a new vm(rocky linux 8,2 sockets 4 cores HOST CPU,8Gi Memory 500GHARD DISK from ceph_ssd),then l used fio to test IOPS:
fio --name=TEST-randwrite --filename=/data/testfile --size=10G --bs=64k --rw=randwrite --ioengine=libaio --iodepth=32 --numjobs=16 --runtime=120 --time_based --direct=1 --group_reporting
The result doesn't well so l trying to using fio on one host of my pve cluster,using same fio setting,find out that even using HHD IOPS is more higher than l using ssd on vm.
So l think thers is nothing wrong with my hardware,so l'm trying to test IOPS on another pve cluster(8.3.3) and also create a vm,500g disk from ceph(37 osd,all hdd) using the same setting fio .
it seems that ceph with hhd goes well in this case.So l testing again but this time l find out vm in my new cluster cpu usage is only 20% while fio testing,but 83% cpu usage on my old cluster and on new host while fio testing.
So l wonder is something wrong with my configure in ceph?l've check two cluster ceph configuration ,both is using rbd and pg_num is setting 128.The network environment is also the same.
l dont know what's wrong with my setting ,pls help if you have some ideas about this case ,thanks a lot.
fio --name=TEST-randwrite --filename=/data/testfile --size=10G --bs=64k --rw=randwrite --ioengine=libaio --iodepth=32 --numjobs=16 --runtime=120 --time_based --direct=1 --group_reporting
The result doesn't well so l trying to using fio on one host of my pve cluster,using same fio setting,find out that even using HHD IOPS is more higher than l using ssd on vm.
So l think thers is nothing wrong with my hardware,so l'm trying to test IOPS on another pve cluster(8.3.3) and also create a vm,500g disk from ceph(37 osd,all hdd) using the same setting fio .
it seems that ceph with hhd goes well in this case.So l testing again but this time l find out vm in my new cluster cpu usage is only 20% while fio testing,but 83% cpu usage on my old cluster and on new host while fio testing.
So l wonder is something wrong with my configure in ceph?l've check two cluster ceph configuration ,both is using rbd and pg_num is setting 128.The network environment is also the same.
l dont know what's wrong with my setting ,pls help if you have some ideas about this case ,thanks a lot.