How to reclaim space in ZFS sparse volume?
- Thread starter carsten2
- Start date
What do you mean by "client-side-mirror"? Windows dynamic dynamic disk mirroring? This does not work, as Windows is also mirroring the "stored-zeroes" instead of skipping them. The only means seems to be the offline methods described in my answer above.
Last edited:
Unfortunately this link adds nothing to the current discussion, because it doesn't write anything about the problem, but only some general remarks on the really good feature of thin-ZVOLs.
If you don't want to use real compression, there is a special compression option ZLE, that compresses ONLY zeros. Interestingly everything needed for real thin ZVOLs is already implemented (zero detection and UNMAP), the only thing is that the combination is missing as a feature: full-zero-blocks => UNMAP instead of ZLE-compress.
I think ZFS does make a difference between non-zero and zero (actually compressible data), but you need compression enabled for that. The highly compressible data does not have a block allocated at all, being embedded in block pointer.
https://illumos.org/issues/4757
https://illumos.org/issues/4757
Code:
zpool upgrade -v
...
embedded_data
Blocks which compress very well use even less space.
...that is because you keep insisting that you understand how it should work, better, than people who are trying to point out for you that in fact you do not understand how it actually works. sorry but that is how your post,including one I am quoting, looks.
let start with this pearl of misunderstanding,
just because ZVOL, ZFS or any other FS returns zero for unused block, doesn't mean there is actually a zero there.
while there is such thing as "virtual blocks", there is no such thing as "virtual zero" .
and a file system even on a real device can have a virtual block, when you drop a partition table on a device you create a table and tell the file system that this is what you have on this disk. when you write something to a disk, 2 things happens,
first FS decides where to put the data and fills in the table with location info. i.e. this bit-go-here kind of things.
than it actually writes the data based on what it wrote in the table.
hence when you query the FS for info it simply reads a table and returns the value. since NULL is not a very good value to work with,
on table read it simply converts any null value to implied zero just for show. it knows very well that there is actually NULL there.
NULL as in absence of value, not zero as in a value of "0" , remember 4th grade algebra? "0" is still a value.
and now we got here. and you are right, "ven filling a ZVOL with only zeros " will force the ZFS and any other FS to allocate a full size to the volume. that is because you are WRITING ACTUAL DATA to the volume. Remeber? a "0" is still a value , hence it is a REAL data.
now on a more polite note, you may try writing a NULL instead of zeros to the volume.
as in !!!WARNING THIS WILL DESTROY YOUR DATA !!! "dd if=/dev/NULL of=/dev/<my volume here> bs=500M count=1"
and than shrink the volume.
I am not sure it would work for OP.
it is not an issue of volume size running a mak.
OP creates a virtual volume thinly provisioned. and it is fine. let say the vSize = 1T but the actual size of the volume file, remember we used thin provisioning, is a few KB/MB if partitioned but empty
now when OP runs df or other info utility on the drive to show what is written in each block , it shows "0" in all blocks that have not been used yet. OP calls this "Virtual Zero" since he/she knows that nothing have been written to the disk and hence by all definition no space should be used.
than OP runs a tool to actually go on and write Zeros to the drives i.e. Zero out the drive or even only write zeros to all unused blocks.
and than looks in surprise that now the drive files grows to occupy the full size of what it should be by config as in a full 1TB.
and it is shrinking by any means.
hence the question by OP , why does it do that? I am only writing zeros.
as I point it out in my last post, OP is confusing a logical zero value returned by info utility for readability sake and actually representing NULL with an actual value of "0" that he/she is writing to the disk and that actually force the allocation of actual space on the disk since it represent a bit of data.
I believe OP needs to stop writing zeros, but rather use NULL on the blocks that should be cleared.
once they are cleared the volume should be able to shrink using what ever means exist for the File system used.
the only other way I can think of is run the disk through gparted or similar tool that can read the data and discern an actual used block from a zero out one. but I never done that myself. before
You can't write NULLs and you can't read from /dev/null. But you can unmap blocks which are all zero (either manually by running unmap under linux/ optimize disk in windows. Or automatically what windows does since 2008 or linux when you use the discard mount option, which does unmap blocks when deleting files.)
I have zfs thin and thin volumes with virtio-scsi with windows as guest and it does work (in Proxmox 5.0, didn't test any other version).
I have zfs thin and thin volumes with virtio-scsi with windows as guest and it does work (in Proxmox 5.0, didn't test any other version).
my bad, I was thinking of the tool I used in windows that used to clear out unused blocks.
I though using dd if=/dev/null .... was similar to it.
if you can do dd-if=/dev/zero .....
I though using dd if=/dev/null .... was similar to it.
if you can do dd-if=/dev/zero .....
Not completely sure how I got here but it seems to me that you should create a big file in your VM (zeros or not) and then delete it. Writing zeros on its own would never issue a trim/discard but then deleting the file definitely would.