I think you're trying to split hairs here. Lvm has more features than RAW but not as many as ZFS. LVM allows to have several VM's on single storage device, and device passthrough should be one per VM. From my (and few other admins) experience LVM seems to be more reliable than RAW disk passthrough but has a tiny penalty of speed.
[SOLVED] Performance comparison between ZFS and LVM
- Thread starter casalicomputers
- Start date
You are right, I stand corrected. I just read through it filtering out the noise and kept reading between you and staff. This has absolutely nothing to do with MDRAID. I apologise I added on to it recently, the original diversion happened in #37, anyhow I should have just started a new thread and reference to it.
But now you got me interested, I am looking at #36 where the mitigation was suggested - basically default to scsi-hd.
But that also means that this is still true:
So the old behaviour is still opt-in. And there has been no NO FIX whatsover:
https://qemu-devel.nongnu.narkive.com/7NrZkgBQ/data-corruption-in-qemu-2-7-1#post12
Do you mind I ask in the original thread what happened with this?
Well, story was pretty basic, I was one of first to actually perform the update of proxmox because it was during the christmas, and about 24 * 8TB drives worth of data went to hell (as far as I remember the size of the shelf). I was pretty "displeased" (and maybe was to harsh in the thread), however I was trying to find the culprit, and spent to much time trying to figure out what was going wrong. After I've noticed more and more people coming in with similar issues, I begun to be very doubtful that this is my misconfiguration and that problem was systemic. Then I think I've noticed a similar issue on non proxmox forum relating to qemu and RAW passthrough to the VM - at that point I said F* that and shifted whole storage to ceph for that specific VM. Few years back (duno the exact year but it was before pandemic) in one corp one admin floated to me idea of passing RAW disks to VM for performance - I gave him a stern warning. He came back to me after some time saying that he created test VM with that sort of setup and it ate his data, so I would say it was not resolved back then. After that I've not used RAW myself or for business aplications. However with LVM I had no major issues.
Another trick that I use is: if I really really need that sweet raw performance goodness, and my stuff can run in a container, I will just create what ever array I need and pass it to the container through the mount point.
But hell - ask away, maybe they will tell us all is good in raw passthrough land. I'm not going to test it.
Last edited:
As you describe it that's frustrating enough.
I understand, but based on the mitigation back then, this should not be occuring (even at the time) if scsi-hd is used (as opposed to scsi-block). So it's not really storage related, it's QEMU param related.
So this is raising eyebrows because I suppose he would have used defaults, which by then would have been scsi-hd.
I get that takeout, just I like to know the core of the issue. Because I could not find any further changes on qemu side in respect to scsi-block and with the anectodal evidence you mention, it also makes me wonder about scsi-hd, but that should not be a an issue, logically speaking, because it's not full passtrhrough.
Maybe I test it first, then ask. There's a test case in the mailing list even.
Thanks!
Yeah, mitigation was suggested few month later, I guess you can imaging what business will say if you suggest "just sit on our hands until somebody graciously will give us a way of fixing that". I'm not knocking of proxmox chap - they have their product that is based on open source projects -> they are not 100% in control of other people timetable as well.
Well, no. The whole discussion with him started from "hey do you know how to pass the devices raw ?" so he would not use defaults. Let's not kid our selfs, raw passthrough is and was far from default
Or maybe you could test what I've reported
Just use disk connections I listed in the original posts, and start dumping data into it. Duno, maybe you've got collection of pr0n that you can hash and later you can read through the passed disks and check hashes ? Just be warned that that corruption would not happen instantaneously. Btrfs would fail spectacularly because it doesn't have some safety features of zfs, but I had problems with ext4 as well. My luck was that this specific VM's was picking up footage from multiple sources and dumping it into a massive array so writes were pretty random and large (this was also the reason to use RAW passthrough since anything else would collapse on amount of random IO to spinning rust).BUT:
maybe let's not hijack this thread as well - dude was asking about ZFS vs LVM.
Last edited: