Yes it does. It's a new version.
A rude way to make that observation, but I dont disagree. Nonetheless, directory is far less convenient that having the integration.
Glusterfs is still maintained. Please don't drop support!
- Thread starter kayson
- Start date
A new version or "another attempt at keeping life support on"? This doesn't matter for qemus gluster support anyhow. Debian packaging new gluster versions wom't help in maintaining the Code for qemus support of gluster as storage backend.
You are missing the point: For continuing to support glusterfs Proxmox developers would have to maintain their own fork of qemu at some point. Since this will cost developer times ( which could be used for other things ) this makes only sense businesswise if there is demand from paying customers. Homelabbers usually don't pay developers salaries and can still use glusterfs via the directory workaround. I might be wrong but my impression is that the developers prefer to concentrate on features people miss from vmware/esxi
Last edited:
You are the one missing the point. Qemu dropped the glusterfs storage backend claiming it's not maintained, which just isn't true. Sure, you could make an argument that it lacks sufficient support for enterprise deployments or even that it's poorly or slowly maintained. But you can't call something unmaintained if people are meeting weekly to discuss it.
Regardless, qemu not having a glusterfs storage backend is orthogonal to Proxmox VE supporting glusterfs as a storage plugin. They're completely independent things. Having the storage plugin means its usable as PVE "storage" in the GUI or CLI (the former requiring the interface elements to be patched back in). While this includes qemu VM images, it also includes isos, CT images, backups, etc. QEMU is just a small part. PVE also handles the mounts for you. There's a lot of convenience to gain over "directory." And really, a qemu storage backend for a file-level storage system doesn't amount to much anyways.
Not sure what the purpose of that statement serves on the proxmox forums. Probably should direct it to the QEMU devs.
Proxmox says they removed the storage plugin because it's dead because QEMU dropped the backend support (which QEMU did because they also say its dead). See post above for why the QEMU backend is only marginally important.
right; so logically the body responsible for the decision is elsewhere (QEMU.)
directory support still works and requires no support from PVE anyway. If the QEMU storage backend serves no additional utility, then what is the actual request for?
Last edited:
no... proxmox is the responsible body for removing the storage plugin, which is (essentially) what this whole thread is about....
see post above, like i already mentioned
I already re-created the storage plugin, compatible with PVE 9 here: https://forum.proxmox.com/threads/w...h-pve9-i-made-a-custom-storage-plugin.184107/
Adding the UI stuff back is a different story (I have patches for that too - if anyone's interested, they're just fragile as they depend on the compiled JS output).
Adding the UI stuff back is a different story (I have patches for that too - if anyone's interested, they're just fragile as they depend on the compiled JS output).
Yes. dev and support have time and cost associated. In a world where both are at a premium, responsible devs direct those scarce resources to where they have the most value. I would have made the same decision, but good on you for shouldering the slack.
You own comments disagrees with your statement that the data I ask about is irrelevant.
(It is highly relevant, pursuant to your own statements. If they were irrelevant, then you wouldn't have written what you wrote.)
Says the person who states:
According to you, if it wasn't relevant, then you wouldn't have written/mentioned it. But you did. Therefore; even according to you, it is relevant, otherwise you wouldn't have mentioned it in the first place.
If it isn't relevant, then why would you even bother to mention any of the points that you've mentioned?
If your own points weren't relevant, then why mention it in the first place?
You complain about my testing methodology which uses an actual VM load to generate the load rather than using
fio to generate a synthetic load whereby you can't tell me the circumstances under which any VM would be able to impart this kind of a synthetic load and then you talk about real systems in real deployments, but then use a synthetic load (rather than a real) load to run your tests.How are you so internally inconsistent with yourself?
If you're going to talk so much about how bad gluster sucks, in the real world, in real servers, running real applications, then why on Earth would you run a [ib]synthetic[/b] load?
That makes absolutely no sense, given what you've been complaining about.
You'd think that if you were complaining about how bad gluster sucks, in the real world, in real servers, running real applications, that you'd be running real workloads to prove your point.
But instead of doing that (which would be at least consistent with everything you've been complaining about up to this point), you choose to run a synthetic workload instead, and quite possibly your most internally inconsistent and one of your most baffling choice that you've made thus far.
But hey - you do you.
If you need a synthetic load, and one where you can't even explain nor describe how you would even actually get/put a VM into such a state -- more power to you.
What this actually proves/demonstrates is the extremely low probability of your benchmarking results = reality due to the extremely low probability that you'd even be able to put a VM into such a state, in order to get the synthetic load that you were using for your benchmarking.
And the fact that you can even explain/describe the commands that you'd have to run to even get/put a VM into such a state tells me how improbable the situation/scenario you've manufactured really is.
In other words, you had to make up a scenario to generate the data/numbers that shows gluster performing somehwere 75% of cephfs and ~80% of ceph_rbd, and even then, it wasn't like ceph was stomping all over gluster, despite your synthetic workload running on hardware (and software) that you're too embarrassed to disclose what you're using/running your tests on/with (as you complain about the fact that I'm running on 11-13 year old hardware).
Therefore; what this also means is that in the absence of this contrived, synthetic workload, gluster would perform probably just fine, if not slightly above (or slightly below) cephfs and/or ceph_rbd.
This is what your data and your methodology actually shows, given that you can't tell me the commands that I would need to run to put a VM into this kind of a state where it has a queue depth of 32, and trying to read 128 files simultaneously (across 4 VMs at that, where the probability of all 4 VMs running this exact same workload, in any actual real, practical deployment would be virtually impossible).
This is what your data actually proves/shows.
Again, what you're really saying here is that the ceph devs are wrong.
I mean the ceph devs explicitly tell you "Current code rads all data chunks" (and the picture shows you that it reads all of said data chunks from all OSDs.
You can argue against the ceph devs and say that said ceph devs are wrong all you want, but at the end of the day, if when it comes down to whom I would trust to know more about ceph if it is between you and the ceph devs, I'd pick the ceph devs all day, any day of the week and twice on Sundays.
(I find it hilariously laughable that you seem to think that you know more about ceph and how ceph works than the ceph devs. lol....)
Again, you only need to spend a few hours with ceph to realise that that EC ceph has a synchronisation problem. It's so obvious. And the fact that the ceph devs tells you. (cf. https://docs.ceph.com/en/latest/dev...nhancements/#current-overwrite-implementation, https://docs.ceph.com/en/latest/dev/osd_internals/erasure_coding/enhancements/#direct-read-i-o).
(It is quite evident that you haven't read the ceph erasure coding enhancements document that the devs have already put together. You should probably read that. A fair of what I've said and written about has already been documented by said ceph devs and is a known performance issue due to how the functons/operations are implemented. Will it stop or prevent you from arguing with the ceph devs? Probably not (on account of you still think that you know how ceph works better than said ceph devs).)
As the ceph devs have explicitly documented, this is true for ceph OSDs as well. See the ceph erasure coding enhancements document linked above.
Again, gluster realised and figured this out quite some time ago, when they advised/recommended people make gluster bricks out of ZFS pools because your ZFS pools can be whatever size/how ever number of disks you want there to be (in said ZFS pool) but then you present said ZFS pool (consisting of x number of disks) to gluster, as one brick.
That way, gluster only has to manage the one monolithic brick, rather than the individual disks=bricks and let ZFS deal with the brick-to-disk "translation" of sorts.
That is literally how hybrid OpenMP/MPI works for LS-DYNA. Intranodal communication is handled via OpenMP (e.g. ZFS in this case) and internodal communication is handled via MPI (gluster in this case).
Ceph hasn't quite gotten there yet.
But I've already written about and proposed basically the same idea/strategy so that ceph isn't dealing with 300+ OSDs. You make a ZFS pool out of disks and then you export that to ceph as a monolithic OSD so that ceph only has to deal with the OSD at the ZFS pool level, rather than at the individual disk level.
I came upon this very same idea when I found out that cache tiering has been deprecated in ceph, and therefore; if you use a ZFS pool as your OSD, then you'd be able to get at least some form of cache tiering by way of ARC, to help speed up reads, at minimum.
This is such an obvious solution to such an obvious problem -- a solution that gluster has recommended presumably years ago when they advised in favour of using ZFS pools as bricks.
You could've literally just downloaded CrystalDiskMark from within the Windows VM and click "run".
It's not that hard. Since when did downloading CrystalDiskMark and clicking "run" become rocket science for some?
You state/argue that it hasn't been fixed "in over 2 years", and yet, if you actually read the thread (which I can only surmise that you didn't), Ravishankar N at Redhat literally wrote:
"There was a case of AFR reporting spurious split-brain errors but that
was fixed long back (http://review.gluster.org/16362) and seems to be
present in glusterf-4.1.6."
Furthermore, i you read Erik Jacobson's reply to him, Erik writes:
"The strange thing, as I mentioned, is it only happened under the job
launch workload. The nfs boot workload, which is also very stressful,
ran clean with one brick down."
...
"> When you say accessing the file from the compute nodes afterwards works
> fine, it is still with that one server (brick) down?
I can no longer check this system personally but as I recall when we
fixed the ethernet problem, all seemed well. I don't have a better
answer for this one than that."
Thus, based on what Erik wrote, sounds like it was an ethernet problem, and not a problem with gluster, as you like to claim.
Which again, I can only surmise that you thought that it was a gluster problem because you didn't bother to read the rest of the message thread. (Since Erik tells you what the fix was, explicitly, from said message thread.)
But you're never going to admit that you didn't read the actual message thread, are you?
You got your confirmation bias and then skipped reading the rest of the thread, where Erik explicitly tells you how they fixed it/what their root cause of the problem was. (Hint: it wasn't gluster)
But, for you -- nevermind that. You got your confirmation bias and stopped reading after that.
Stupid question though - if Proxmox already has the glusterfs storage plugin that allows it to work from the PVE GUI, then why would that need to change?
I'm not sure that I follow.
I mean, you guys already have it.
It already works.
And as far as I can tell, when you create a VM, epsecially from within the PVE GUI, it asks you where you want to put the disks, and so long as the gluster storage plugin already exists, then why can't you just continue to use that, "as-is"?
I mean, if we translate "gluster is dead" as "gluster is stable" (hence lack of movement on the project), then why would you need to do anything with the gluster plugin?
I'd have to imagine that
nfs-common is probably pretty stable with very few commits as well (why break the NFS common client when it works?) and the PVE plugin for that still exists, no?I'm not sure if this is the nfs source code that's for the
nfs-common package, but there are 0 commits over the past year.https://github.com/linux-nfs/nfsd/graphs/commit-activity
Again, if the gluster plugin exists already and it works, why change/remove it? As I recall, VM creation with Proxmox just asks where you want to point the vm disk to and qemu/qm , as far as I can tell (and maybe I'm wrong), it just wants to know where to put the disk (on a volume backend). qm doesn't manage the storage backend.
pvesm is responsible for that, which is separate from qm and thus, QEMU right?So....if
pvesm sets up the volume backend, and qm only consumes or utilises said volume backend, this would mean that QEMU isn't managing the storage/volume backend anymore, right?And given that the gluster plugin already exists and works, if
qm and/or QEMU doesn't manage the storage/volume backend, or that QEMU, the way that it is implemented in Proxmox, doesn't need to talk directly to gluster, therefore; gluster being deprecated/removed from QEMU would be irrelevant to how pvesm works, right?Or am I missing something?
Therefore; if gluster is handed by
pvesm rather than by QEMU, and the gluster plugin already works, then why would Proxmox need to remove the independent gluster plugin, if pvesm and qm/QEMU function/operate separately and independently of each other?I mean, yes,
qm create still expects a place to put the VM disk on a volume backend somewhere but as far as qm is concerned, it doesn't really care what type/class the storage/volume backend is no? So long that you point it to a storage/volume backend, right?From qm(1);
Code:
qm create
...
--sata[n] [file=]<volume> [,aio=<native|threads|io_uring>] [,backup=<1|0>] [,bps=<bps>] [,bps_max_length=<seconds>] [,bps_rd=<bps>] [,bps_rd_max_length=<seconds>] [,bps_wr=<bps>] [,bps_wr_max_length=<seconds>] [,cache=<enum>] [,detect_zeroes=<1|0>] [,discard=<ignore|on>] [,format=<enum>] [,import-from=<source volume>] [,iops=<iops>] [,iops_max=<iops>] [,iops_max_length=<seconds>] [,iops_rd=<iops>] [,iops_rd_max=<iops>] [,iops_rd_max_length=<seconds>] [,iops_wr=<iops>] [,iops_wr_max=<iops>] [,iops_wr_max_length=<seconds>] [,mbps=<mbps>] [,mbps_max=<mbps>] [,mbps_rd=<mbps>] [,mbps_rd_max=<mbps>] [,mbps_wr=<mbps>] [,mbps_wr_max=<mbps>] [,media=<cdrom|disk>] [,replicate=<1|0>] [,rerror=<ignore|report|stop>] [,serial=<serial>] [,shared=<1|0>] [,size=<DiskSize>] [,snapshot=<1|0>] [,ssd=<1|0>] [,werror=<enum>] [,wwn=<wwn>]
Use volume as SATA hard disk or CD-ROM (n is 0 to 5). Use the special syntax STORAGE_ID:SIZE_IN_GiB to allocate a new volume. Use STORAGE_ID:0 and the [I]import-from[/I] parameter to import from an existing volume.cf. pvesm(1).
Last edited:
This will change as soon as qemu remove it's gluster support. Which will happen at some point otherwise they wouldn't have deprecated it. And maintaining a qemu fork is a whole different beast than continueing to ship a storage plugin.
A cluster file system should remain stable, even in case of hardware failure. I did read that thread and the point is that there was data unavailability that simply allows the OS to continue in a corrupted state. They didn’t know why only that it happens under load. The point of a cluster file system is that it continues working flawlessly even if you pull a network cable or an entire node down.
Gluster is being removed from the QEMU project. This is primarily because there are no developers able or willing to work on it, not because it is no longer a functional system for today’s world, there are other things in qemu that are ancient, like e1000 and even netware era NIC, as long as it works or someone maintains it.
It was a product of its time, as I said, I worked with it for a few years back in the day. Plugins don’t just drop into the system, you seem very confused about how these things work in general but no, the plugin is not just writing files to an NFS/FUSE mount. It leverages Gluster Block support. As to the rest of your comment, you’re reading things that aren’t there, it’s called the Dunning-Kruger effect. You should understand what the Ceph developer writes when it comes to CRUSH or ask ChatGPT to explain how it scales vs AFR/DHT-based systems.
My point on Windows is that you cannot reliably benchmark it if you don’t understand the underlying architecture. It doesn’t matter what your benchmark shows because you’re not testing the storage layer, but the CPU and RAM. I have controlled for those in my test, to do so on Windows would require significant work.
Gluster is being removed from the QEMU project. This is primarily because there are no developers able or willing to work on it, not because it is no longer a functional system for today’s world, there are other things in qemu that are ancient, like e1000 and even netware era NIC, as long as it works or someone maintains it.
It was a product of its time, as I said, I worked with it for a few years back in the day. Plugins don’t just drop into the system, you seem very confused about how these things work in general but no, the plugin is not just writing files to an NFS/FUSE mount. It leverages Gluster Block support. As to the rest of your comment, you’re reading things that aren’t there, it’s called the Dunning-Kruger effect. You should understand what the Ceph developer writes when it comes to CRUSH or ask ChatGPT to explain how it scales vs AFR/DHT-based systems.
My point on Windows is that you cannot reliably benchmark it if you don’t understand the underlying architecture. It doesn’t matter what your benchmark shows because you’re not testing the storage layer, but the CPU and RAM. I have controlled for those in my test, to do so on Windows would require significant work.
Last edited:
Again, unless I'm missing something here - is the
pvesm a part of QEMU???But as I understand it,
qm that PVE uses, doesn't set up nor configure nor "talk" to the storage backend directly, right?Again, unless I'm missing something here.
All
qm expects is for you to point to a location where it can dump/write the vm disk file and as far as I can tell, it doesn't particularly care what the storage backend that you're using, for the volume is right?I mean, this is why you have all of the various different storage options, that
pvesm manages/configures for qm to use, correct?Therefore; so long as the management and configuration of the storage backend is separate from qm(1), who cares is QEMU is dropping support for gluster?
As far as I can tell, PVE's qm(1) isn't and doesn't need to manage/configure/"talk" to the storage backend directly (because even using the gluster plugin, in the current state right now, will just mount it to
/mnt/pve/<<name_of_gluster (which you can either define/add it via the GUI and/or by modifying /etc/pve/storage.conf), right?Like I can understand it if qm(1) is likewise, used to manage/configure/talk to "gluster" (as a gluster client) directly, then sure.
But given that you have
pvesm which handles the management and configuration of the storage backends, which means that qm doesn't do that, then who cares if QEMU is dropping support for gluster?qm isn't using it anyways.pvesm is what deals with the different types of storage backends anyways.Again, per the qm(1) man page,
volume is just the where you're putting the vm disk, not how (you're putting said vm disk).
Code:
Use volume as SATA hard disk or CD-ROM (n is 0 to 5). Use the special syntax STORAGE_ID:SIZE_IN_GiB to allocate a new volume. Use STORAGE_ID:0 and the import-from parameter to import from an existing volume.So long as the two are separate (
qm creates/manages VMs and pvesm manages and configures the different tyles of storage backends, then it would be irrelevant that QEMU is dropping gluster support because pvesm mounts the storageID to /mnt/pve/<<storageID>> anyways, which means that effectively, nearly all storage backend types are all mounted (and exposed) to qm as a directory/mount point.Again, unless I'm missing something - this is how it works, no? That
pvesm and qm are separate, therefore; QEMU dropping support is irrelvent because pvesm exists?How do the heck do you get a split-brain with
It is a Distributed-Replicate volume 3x9. gvol???It's a (distributed) replicate gvol.
Therefore; copies of the data are replicated across the bricks, so how would you even get split-brain with a replicated volume?
I don't understand how that's possible.
Furthermore, Erik wrote that it's a distributed-replicate volume 3x9, but from his
gluster volume info, the output of that shows:
Code:
Volume Name: cm_shared
Type: Distributed-Replicate
Volume ID: e7f2796b-7a94-41ab-a07d-bdce4900c731
Status: Started
Snapshot Count: 0
Number of Bricks: 3 x 3 = 9lol.....you're clearly not on the ceph mailing list, are you?
People have been reporting all sorts of issues with ceph.
ceph isn't immune to issues, just because it's ceph.
I've been on the ceph mailing list since March 9th, and in just the last three months, that folder, in my email, has 221 emails in there, where the issues are quite diverse.
You seem to present ceph as though issues like these only happen with gluster and that issues doesn't happen with ceph, when, as the mailing list shows, is far from the realities of ceph deployments where ceph isn't immune to data unavailability as well. This isn't just a gluster issue. It can happen with ceph as well.
(cf. here, here, and here as well.)
If you read the thread, then you would've read that Erik stated that they fixed the problem that had nothing to do with gluster itself anymore than if you have a bad drive capable, which causes data corruption, would be the cause of whatever HW RAID system you were using, at the time.
Again, as I've cited, ZFS will commit corrupted data to stable storage if you have bad drive cables and/or issues with the IMC where ZFS won't know that it is commiting corrupted data to disk.
That'd like you blaiming gluster = bad because of a bad network cable, where if you fixed said bad network cable, then your gluster problem goes away.
Cuz that's what Erik from HPE wrote, which if you read that, then you would've known this rather than saying, quote;
"Similar split brain scenarios under heavy (note that for today's standards, this is no longer heavy, Ceph can hit 1M 64kB IOPS across 12 nodes) load was reported before on mailing lists in 2019 with Gluster 4 https://lists.gluster.org/pipermail//gluster-users/2019-September/037083.html It is a very similar problem to what I started seeing 15 years ago as our cluster grew, split brains started occurring. Yes, you can throw more CPU, hardware and do optimizations, but at some point, you get to a point that it is untenable, a file system that throws up its hands in I/O error is BAD."
Erik, in his reply to Ravishankar N at Redhat, literally tells you this.
But that's not what you wrote in the quote from you above.
If you read the thread and knew it, then that's knowingly making fraudulent claims/statements about gluster that you [i[know[/i] were fraudulent misrepresentations.
Therefore; given what you wrote, I would have to give you the benefit of the doubt where you didn't read said thread.
So which is it? You read the thread and then made knowingly fraudulent misrepresentations anyways, or you didn't read the thread, which is why you wrote what you wrote?
So which is it?
From the commentary from others, that's not the claimed reason why.
The claimed reason why is that they were looking at the relative lack of activity, but contrary to people's claims (that are then promulgated and repeated elsewhere like here), gluster has active developers even if the pace of commits isn't as active as a project that has a lot of problems that i needs to patch.
I mean, that's the easiest way to boost the number of commits per month. Have (or introduce) a lot of bugs that you have to fix. You can push the commit/month metric through the roof when you have a lot of bugs to fix.
(cf. Nvidia drivers)
Just out of curiosity - how many commits have there been for the e1000 NIC driver within any timeframe?
Again, as it has been noted, you can mount the glustefs with the
glusterfs-client package (in the case of Proxmox, from the Debian repo). Therefore; so long as the GUI/plugin is to provide a graphical front end for said glusterfs-client package, which already exists on Debian, and as @kayson mentioned, Debian's newest package supports gluster 11.2, therefore; so long as Debian supports it, the plugin can just be the graphical front end that sets up or passes the user input arguments to the glusterfs-client to mount a gluster volume.Said gluster block support is already provided via
glusterfs-client.You think that I'm confused and yet, you haven't been able to provide any citations that shows that I'm wrong (about how the
glusterfs-client works).More often than not, the people who cite Dunning-Kruger completely fail to realise that there's an entire right-side of the Dunning-Kruger effect plot that they often times, generally and convenient ignore.
(Ironically enough, and since you mention it -- you're still on the left side of your citation of the Dunning-Kruger effect where you think you know more about ceph than the ceph devs. This is a literal and practical demonstration of your reference of said Dunning-Kruger effect where you don't even know what you don't know.)
It's literally right there.
LOL...LMAO....
I can't help if you're unable to read (or too lazy to read -- can't really tell at this point whether it's a question of ability or whether it's a question of laziness and/or both.)
Or...you can just read what I've drawn the red box around from the ceph documentation that the ceph devs have authored.
1) Translation: you're too embarrassed to disclose what hardware/software you're using. Which is fine, because after all your complaining about it, what this means is that the hardware that you're running in your homelab would be susceptible to your own complaints about said hardware which is why you're embarrassed to disclose what hardware/software you're using.
2) You're embarrassed to have someone else replicate your tests because if someone else dares tries and am able to generate data that contradict your claims -- well, you just can't have that.
3) You will critique what WIndows/NTFS does all day long, but then do nothing about "fixing" it (because in reality, the vast majority of Windows users doesn't GAF. And by market share, Windows is still the dominant desktop OS, by far, which also means that if the issues that you've raised were really as serious as you make them sound, they would've been fixed a long time ago, because too many people would have lost their data because of how Windows/NTFS behaves. And yet, that's clearly not the case here.
Millions/billions of people have been using Windows just fine just as millions/billions of users don't care about the points that you've brought up (about how Windows/NTFS quote "lies" to you given that it works "well enough" for said millions/billions of users.
So, it's a whole lot of nothing burger.
4) You can't prescribe/produce a script where you'd be able to put a VM in such a state to generate the workload that you're using
fio to test with/generate, even though you claim that Windows can do it.So, if Wndows is able to generate/produce that kind of a workload, then why bother using
fio???Heck, use Linux for all I care (given your complaining about Windows).
According to you, any modern OS would be able to generate that kind of a workload, so then why don't you just use quote "any modern OS" for the load generation rather than
fio if you claim that any modern OS can generate that kind of a load???If it can do it, then why aren't you testing it with it then??? Seems so simple.
And for the record, I re-ran CrystalDiskMark 24 hours and 40 minutes later and this is what I got;
You argue that it can't be tested reliably and yet, the maximum stdev (in MB/s) was 4.13657 MB/s and the min stdev was 0 MB/s. As a percentage of the average stdev/mean was 2.28% for gluster and 3.94% for ceph (because it was so slow, that small differences in the results yielded a higher stdev, and therefore; as a percent of the average, the slow results of ceph made it more sensitive to variations.
Either way, I could run it over and over again to obtain the population statistics, but the relatively low stdev already shows that it will pass a gauge R&R.
What you see on the host side isn't necessarily what you're going to see, from inside a VM.
You argue that you had to cap the network at 400 Mbps meanwhile I installed 3 Win11 VMs, on the gluster cluster, without any issues. (System was RAM limited such that I couldn't spawn a 4th gluster node, and then spawn a 4th Win11 VM.)
You said that it would split-brain under quote "a heavy workload" (which is why you tested with
fio --iodepth=32 --numjobs=4 --nrfiles=128) and yet running three Win11 installs ran without any hitches and I didn't bother neutering the virtio-nic so that it will be able to run at full speed.In other words, your synthetic workload is precisely that - synthetic. It isn't a "real" workload of Windows (or Linux since complainers have to always complain about something) that would be more realistic and more representative of what an actual client would be running or can run.
Again, I don't need any tricks nor do anything special/extra.
Just install and go let the post-install Win11 updates do Win11 update things.
You could automate the install and testing with DISM.
I ran my testing/benchmarking the way that I have because it is designed to be so simple that anybody could repeat the tests since there's hardly anything to it.
Point your VM to the Win11 ISO and run through the process of creating said VM and then once it is all set up, just let Win11 updates to Win11 update things.
This is performed this way, because this is what Win11 installs are like. Nobody bothers customising anything (in terms of how NTFS reports drive statistics back to the user because no one GAF).
It's what people do.
And by purposely running the tests this way, it means that any can repeat the test, just as I didn't bother tuning ceph/gluster neither.
Install and go.
This is what the experience is when you just deploy ceph/gluster/Win11 VM "out of the box". This is what you get.
[B]alpha754293[/B] Did you test shard volume setup for vm images too ? Would be even nice to see crystaldiskmark results for 200GB testfiles size if possible (gv0 replicated+shard, ceph replicated as maybe a ec setup) to you.
I did not.
I'll have to look up a) what a shard volume is and b) how to set it up either in the context of an EC ceph pool and/or in the context of a replicated ceph pool.
re: 200GB testfiles size
I only set up my Win11 VM with 100 GB VM/OS disks just to i) make the disk big enough for the OS to fit and ii) make it so that when PVE does the default preallocation=metadata, it doesn't take that long.
As noted though, my test methodology is centered around the principle/idea of an "out of the box" experience as much as possible, and therefore; has little to no customisations/optimisations.
If people don't like the results that the "default" options gets you, then it is up to the ceph devs to upstream the changes to said defaults given that usually, the defaults are selected as the "best" "middle-of-the-road" choices for the various tunable parameters that isn't biased towards one workload type nor another.
As such, anybody who deploys ceph is able to test it how they want to.
No but pvesm is not a magic Wand which will support something not posdible anymore in qemu