How to reclaim space in ZFS sparse volume?

Rob Loan

Member
Mar 25, 2017
48
5
8
54
how about present the guest another (sparse) lun and client side mirror it and destroy the fat lun.
 

carsten2

Member
Mar 25, 2017
61
3
8
50
What do you mean by "client-side-mirror"? Windows dynamic dynamic disk mirroring? This does not work, as Windows is also mirroring the "stored-zeroes" instead of skipping them. The only means seems to be the offline methods described in my answer above.
 
Last edited:

carsten2

Member
Mar 25, 2017
61
3
8
50
If you don't want to use real compression, there is a special compression option ZLE, that compresses ONLY zeros. Interestingly everything needed for real thin ZVOLs is already implemented (zero detection and UNMAP), the only thing is that the combination is missing as a feature: full-zero-blocks => UNMAP instead of ZLE-compress.
 

sigxcpu

Member
May 4, 2012
433
9
18
Bucharest, Romania
I think ZFS does make a difference between non-zero and zero (actually compressible data), but you need compression enabled for that. The highly compressible data does not have a block allocated at all, being embedded in block pointer.

https://illumos.org/issues/4757

Code:
zpool upgrade -v 
...
embedded_data
     Blocks which compress very well use even less space.
...
 
  • Like
Reactions: carsten2

jim.bond.9862

Active Member
Apr 17, 2015
344
28
28
It is a little strange that although I explained the problem, people keep stating, that they suppose I didn't understand ZFS-(thin-provisioning). I just want to repeat that I DID fully understand ZFS in this respect. Unfortunately they didn't get the problem.
.
that is because you keep insisting that you understand how it should work, better, than people who are trying to point out for you that in fact you do not understand how it actually works. sorry but that is how your post,including one I am quoting, looks.



To repeat (assume compression disabled):
1) A new thin-provosioned ZVOL occupies (almost) no real space in ZFS. When reading from such a ZVOL, ZFS delivers zeros for every non-existing block, i.e. it delivers "virtual zeros" for "virtual blocks".
.
let start with this pearl of misunderstanding,
just because ZVOL, ZFS or any other FS returns zero for unused block, doesn't mean there is actually a zero there.
while there is such thing as "virtual blocks", there is no such thing as "virtual zero" .

and a file system even on a real device can have a virtual block, when you drop a partition table on a device you create a table and tell the file system that this is what you have on this disk. when you write something to a disk, 2 things happens,
first FS decides where to put the data and fills in the table with location info. i.e. this bit-go-here kind of things.
than it actually writes the data based on what it wrote in the table.
hence when you query the FS for info it simply reads a table and returns the value. since NULL is not a very good value to work with,
on table read it simply converts any null value to implied zero just for show. it knows very well that there is actually NULL there.
NULL as in absence of value, not zero as in a value of "0" , remember 4th grade algebra? "0" is still a value.

2) Only by writing to the ZVOL blocks get allocated. Further ZVOL (unfortunately) makes no difference between bllocks filled with real data and blocks filled with zeros, so even filling a ZVOL with only zeros allocated the full size of the ZVOL from the pool. From a filesystem and application view however the ZVOL didn't change, because it contains only zeroes, which are "virtual zeros" before filling the volume and "stored zeroes" after filling the volume with zeros.
.
and now we got here. and you are right, "ven filling a ZVOL with only zeros " will force the ZFS and any other FS to allocate a full size to the volume. that is because you are WRITING ACTUAL DATA to the volume. Remeber? a "0" is still a value , hence it is a REAL data.

now on a more polite note, you may try writing a NULL instead of zeros to the volume.
as in !!!WARNING THIS WILL DESTROY YOUR DATA !!! "dd if=/dev/NULL of=/dev/<my volume here> bs=500M count=1"
and than shrink the volume.
 

Ruffy91

New Member
May 23, 2017
7
0
1
28
Have you activated discard for the volume? This works with virtio-scsi on Windows and will allow unmap to do its magic. This will also work with linux when you either unmap manually or add discard to the mount options to unmap continuously.
 

Attachments

jim.bond.9862

Active Member
Apr 17, 2015
344
28
28
Have you activated discard for the volume? This works with virtio-scsi on Windows and will allow unmap to do its magic. This will also work with linux when you either unmap manually or add discard to the mount options to unmap continuously.
I am not sure it would work for OP.

it is not an issue of volume size running a mak.

OP creates a virtual volume thinly provisioned. and it is fine. let say the vSize = 1T but the actual size of the volume file, remember we used thin provisioning, is a few KB/MB if partitioned but empty

now when OP runs df or other info utility on the drive to show what is written in each block , it shows "0" in all blocks that have not been used yet. OP calls this "Virtual Zero" since he/she knows that nothing have been written to the disk and hence by all definition no space should be used.
than OP runs a tool to actually go on and write Zeros to the drives i.e. Zero out the drive or even only write zeros to all unused blocks.
and than looks in surprise that now the drive files grows to occupy the full size of what it should be by config as in a full 1TB.
and it is shrinking by any means.

hence the question by OP , why does it do that? I am only writing zeros.
as I point it out in my last post, OP is confusing a logical zero value returned by info utility for readability sake and actually representing NULL with an actual value of "0" that he/she is writing to the disk and that actually force the allocation of actual space on the disk since it represent a bit of data.
I believe OP needs to stop writing zeros, but rather use NULL on the blocks that should be cleared.
once they are cleared the volume should be able to shrink using what ever means exist for the File system used.

the only other way I can think of is run the disk through gparted or similar tool that can read the data and discern an actual used block from a zero out one. but I never done that myself. before
 

Ruffy91

New Member
May 23, 2017
7
0
1
28
You can't write NULLs and you can't read from /dev/null. But you can unmap blocks which are all zero (either manually by running unmap under linux/ optimize disk in windows. Or automatically what windows does since 2008 or linux when you use the discard mount option, which does unmap blocks when deleting files.)

I have zfs thin and thin volumes with virtio-scsi with windows as guest and it does work (in Proxmox 5.0, didn't test any other version).
 

jim.bond.9862

Active Member
Apr 17, 2015
344
28
28
my bad, I was thinking of the tool I used in windows that used to clear out unused blocks.
I though using dd if=/dev/null .... was similar to it.
if you can do dd-if=/dev/zero .....
 

Andrew Hart

Member
Dec 1, 2017
68
9
8
48
Not completely sure how I got here but it seems to me that you should create a big file in your VM (zeros or not) and then delete it. Writing zeros on its own would never issue a trim/discard but then deleting the file definitely would.
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE and Proxmox Mail Gateway. We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get your own in 60 seconds.

Buy now!