Failed Fail Over

[esi_y]

BTW Is this some RAIDZ or single vdev / (striped) mirror only?

 

[ballybob]

Thanks, I mostly wanted to know if zvol_use_blk_mq was not set (it's not), but may find something more later on. I just experienced more weird behaviour with ZVOLs over time. Please let us know later in case the "fix" was not real, but I will just assume that thick provisioning the volume made it for you.

I need to fail it more times to know for sure, I only did it once, but it worked immediately which it hasn't done before.

 

[LnxBil]

@ballybob FWIW I personally avoid ZVOLs for VMs, rather than anectodal evidence, if I pull quick search results this is still in the top:

https://jrs-s.net/2018/03/13/zvol-vs-qcow2-with-kvm/

Interesting. I like QCOW2 on $anything because of its tree-like snapshot structure, yet the argument I always get when I say that I use QCOW2 for some machines on ZFS (datasets) is that you would have COW on COW and with snapshots, the performance gets unpredicable and slow. I have to concur with that. All VMs feel much more sluggish, slower and its backup takes much more time. If I clone the VM to a zvol, it is noticebly faster, including the backup. I did not run artificial tasks, I ran real world tasks like creating oracle databases per script so that you can "see" the runtime, install OSes. The article does not mention this at all, just non-snapshot-states and nothing about backup speed, which would be - as much as I love fio - a real world perfectly comparable test.

QCOW2 is also not trimmable as easy as a ZVOL, so that you will waste much more space on your machines. You need to offline compact the file in order to get back space.

I run a server with 40+ machines on 6 enterprise ssds in a RAID10 on ZVOL and it worked great for all machines with only linear snapshots, for the tree-like ones, I used a dataset.

 

[esi_y]

Interesting. I like QCOW2 on $anything because of its tree-like snapshot structure, yet the argument I always get when I say that I use QCOW2 for some machines on ZFS (datasets) is that you would have COW on COW and with snapshots, the performance gets unpredicable and slow. I have to concur with that. All VMs feel much more sluggish, slower and its backup takes much more time.

Yes, but another way of looking at is that ZFS (or any COW) is simply unsuitable for it, not the other way around.

If I clone the VM to a zvol, it is noticebly faster, including the backup.

Yes, but again, it's simply removing one layer of the two, you picked the QCOW2.

I did not run artificial tasks, I ran real world tasks like creating oracle databases per script so that you can "see" the runtime, install OSes. The article does not mention this at all, just non-snapshot-states and nothing about backup speed, which would be - as much as I love fio - a real world perfectly comparable test.

It's not my post there, the second link came up with opposite results even, I included both on purpose.

QCOW2 is also not trimmable as easy as a ZVOL, so that you will waste much more space on your machines. You need to offline compact the file in order to get back space.

Yet if ZVOL fails and is literally not recommended to be run thin...

I run a server with 40+ machines on 6 enterprise ssds in a RAID10 on ZVOL and it worked great for all machines with only linear snapshots, for the tree-like ones, I used a dataset.

That's anectodal evidence, but yes, it's possible. My concern is for the quality of ZVOL implementation. E.g. I do not convince anyone to prefer BTRFS as much as it never failed me personally.

The question here really is ... why thick provisioning that ZVOL with efidisk on mostly empty dataset solved something at all.

 

[LnxBil]

First: good discussion ...

Yet if ZVOL fails and is literally not recommended to be run thin...

You mentioned that already, yet I don't know why. Where do you get this? I've never stumbled upton this. I run everything in ZFS thin for almost a decade.

Yes, but another way of looking at is that ZFS (or any COW) is simply unsuitable for it, not the other way around.

Which type of storage is on feature-par with this? I don't know of any.

That's anectodal evidence, but yes, it's possible. My concern is for the quality of ZVOL implementation.

I don't have any other information, so of course it is anectodal. It would be great if one could reproduce the problem.

The question here really is ... why thick provisioning that ZVOL with efidisk on mostly empty dataset solved something at all.

That I don't know.

 

[esi_y]

You mentioned that already, yet I don't know why. Where do you get this? I've never stumbled upton this. I run everything in ZFS thin for almost a decade.

ZVOLs have been riddled with issues for a long time, e.g.:

https://github.com/openzfs/zfs/issues/7631
https://github.com/openzfs/zfs/issues/10095

There are others, but you might say they are were not default for PVE:

https://github.com/openzfs/zfs/issues/15351

If you ask me about thin specifically, they are literally "not recommended" (without further reasoning):
https://openzfs.github.io/openzfs-docs/man/master/7/zfsprops.7.html

Obviously, I can't easily argue when a problem arises due to ZVOL being sparse and when it was universally a ZVOL issue, but anecdotal evidence supports the logical conclusion that there's more to break apart when a ZVOL is sparse.

Which type of storage is on feature-par with this? I don't know of any.

I just run it RAW, if I need a VM. I would then get my "features" within the VM.

I don't have any other information, so of course it is anectodal. It would be great if one could reproduce the problem.

Can you reproduce e.g. this:

https://forum.proxmox.com/threads/w...reported-avail-and-refreserv-on-zvols.151874/

That I don't know.

I find this case of OP extreme - it's a tiny volume, almost nothing changing there.