[SOLVED] Undestrand discard option

Valerio Pachera

Active Member
Aug 19, 2016
131
6
38
42
How discard generally work:
consider a logical volume (lv) of a thin pool.
Once created a lv, the allocated spaced is 0.
I create a filesystem on in (i.e. ext4), add some data till it allocates 5G.
I remove 2G of files but the allocated space is still 5G.
By fstrim, it will de-allocate/discard the 2G so only 3G will be allocated.

I the example above, the filesystem (ext4) is stored directly on the lv, without another partition table as it happens if the lv is used as virtual disk for qemu/proxmox.

My first question is, does discard work also in this case?
Or shall I use discard only on lv with no partition table?

My second question is: when will data be discard by proxmox? At guest shutdown/restart?

Third question: do all thin provisioning storage (lvm, zfs, qcow2) work the same way?

Thank you.
 
My first question is, does discard work also in this case?
Or shall I use discard only on lv with no partition table?
The filesystem tells the underlying storage to discard blocks, as only the filesystem knows what to throw away and what not. So it doesn't matter if partitioned or not, discard is done on a mounted filesystem basis.

My second question is: when will data be discard by proxmox? At guest shutdown/restart?
Discard is a linux feature, nothing special is done by proxmox. Depending on how you set it up, then either it will be done immediate or on your manual schedule (see links below).

Third question: do all thin provisioning storage (lvm, zfs, qcow2) work the same way?
No. But for discard, they all support it.

Setup and use on PVE:
https://pve.proxmox.com/pve-docs/pve-admin-guide.html#qm_virtual_machines_settings
https://pve.proxmox.com/wiki/Shrink_Qcow2_Disk_Files

Further infos:
https://wiki.debian.org/SSDOptimization
https://wiki.archlinux.org/index.php/Solid_State_Drives
 
I read the documentation and I get confused because pve-admin guide says
your VM has a SCSI controller you can activate the Discard
while shrink qocw2 file guide says
Virtio-SCSI disks with the discard option enabled
.

I find it more logic to be virtio-scsi the required controller.

Please, help me finding the way to discard the unused blocks from this thin volume:

Code:
  LV Path                /dev/pve-big/vm-100-disk-1
  LV Name                vm-100-disk-1
  VG Name                pve-big
  LV UUID                DVg5YO-ZcNe-C4HV-mrfj-Z0rq-FgeO-2TFRv2
  LV Write Access        read/write
  LV Creation host, time cavaion-proxmox-ve, 2016-12-20 08:34:28 +0100
  LV Pool name           vm
  LV Status              available
  # open                 1
  LV Size                200,00 GiB
  Mapped size            99,62%
  Current LE             51200
  Segments               1
  Allocation             inherit
  Read ahead sectors     auto
  - currently set to     256
  Block device           251:6

It's used by a windows 2012 server R2 guest.
Drive C: was almost full, then lot's of data has been removed.

upload_2017-9-29_16-44-6.png


The guest has virtio controller as controller and discard is enabled as you can see here
Code:
-drive file=/dev/pve-big/vm-100-disk-1,if=none,id=drive-virtio0,format=raw,cache=none,aio=native,detect-zeroes=on

I tried to stop and restart the guest, but the thin volume is still 99% mapped.

For those Windows versions which support TRIM you should not have to setup anything, Windows should periodically trigger a TRIM to the disks.
upload_2017-9-29_16-52-20.png

I hope I'm not missing anything.

PS: the host is a bebian jessie, proxmox 4.4-1/eb2d6f1e.
 
How does your VM config look like 'qm config <vmid>'? Looks to me, as if discard is not activated.
 
Here it is

Code:
qm config 100
agent: 1
balloon: 2048
boot: c
bootdisk: ide0
cores: 8
cpu: host
ide2: local:iso/virtio-win.iso,media=cdrom,size=152204K
keyboard: it
localtime: 1
memory: 6144
name: win2012
net0: e1000=0E:4C:07:AF:81:06,bridge=vmbr0
numa: 1
ostype: win8
scsihw: virtio-scsi-pci
smbios1: uuid=1270bf4b-1e1c-4df3-918a-a2414488e156
sockets: 1
unused0: big:vm-100-disk-2
virtio0: big:vm-100-disk-1,size=200G
virtio1: big:vm-100-disk-3,backup=0,cache=writeback,discard=on,size=100G
 
Have you tried with disk type SCSI instead of Virtio? I have some Windows 2008R2 and 2012R2 VMs and with discard=on, SCSI disk and Virtio-SCSI, I see the correct behaviour, when Windows perform maintenance and there are a lot of deleted files, the ZFS used size will decrease. I think should be the same with LVM-thin.
 
Ops, I forgot to enable options discard the last time I removed and re-added the vdi to be sure it was using virtio controller.
Now I checked it twice :)
Anyway, after the stop and restart of the guest with discard option enabled, the "Mapped size" is still 99%.

Code:
qm config 100 | grep vm-100-disk-1
virtio0: big:vm-100-disk-1,discard=on,size=200G

lvdisplay /dev/pve-big/vm-100-disk-1 | grep Mapped
  Mapped size            99,64%

When should I expect windows to trigger discard/trim?
I'll try later with scsi controller if I see no results.

Thank you.
 
Hi, I checked the maintenance schedule and it was run last time this morning.
I also started it manually as you can see in this image.

upload_2017-10-3_16-9-34.png

My server language is set in italian.
I used this guide as reference.

Mapped size is stll 99% even after running maintenance.
 
This is a config on one of my Windows 2012 VM

Code:
qm config 100
-cut-
scsi0: local-zfs:vm-100-disk-1,cache=writeback,discard=on,size=64G
scsi1: local-zfs:vm-100-disk-2,cache=writeback,discard=on,size=250G
scsi2: local-zfs:vm-100-disk-3,cache=writeback,discard=on,size=220G
scsi3: local-zfs:vm-100-disk-4,cache=writeback,discard=on,size=50G
-cut-
scsihw: virtio-scsi
-cut-

I use ZFS and not LVM and I use SCSI driver (with virtio-scsi driver installed in Windows) and VirtIO SCSI controller.
Have you tried switching to virtio-scsi driver?
 
Have you tried switching to virtio-scsi driver?
I'm using virtio driver.
I notice anyway that you config says
scsihw: virtio-scsi
and mine
scsihw: virtio-scsi-pci

Any idea about the difference and if it may influence discard?
 
I success with a linux guest and scsi driver

Code:
scsi0: big:vm-103-disk-1,cache=writeback,discard=on,size=30G
scsihw: virtio-scsi-pci

I'll try also with windows asap.
 
When I have switched my Windows VMs from virtio to scsi disk I had to wotk this way:
- poweroff the Windows VM
- add a new disk with scsi typ, I just used 1GB in size
- started Windows VM
- from Device Manager, installed the virtio-scsi driver on the unknown device (the new disk)
- made a test in disk management, new disk is online, formatted and so on
- powered off the Windows VM
- removed the virtio disks and re-added as scsi disks
- started Windows VM

During the Windows maintenance time, the space used in ZFS pool has decreased.
 
Great, it worked also on windows!
Thank you all for you support!

Code:
lvdisplay /dev/pve-big/vm-100-disk-1
  --- Logical volume ---
  LV Path                /dev/pve-big/vm-100-disk-1
  LV Name                vm-100-disk-1
  VG Name                pve-big
  LV UUID                DVg5YO-ZcNe-C4HV-mrfj-Z0rq-FgeO-2TFRv2
  LV Write Access        read/write
  LV Creation host, time cavaion-proxmox-ve, 2016-12-20 08:34:28 +0100
  LV Pool name           vm
  LV Status              available
  # open                 1
  LV Size                200,00 GiB
  Mapped size            57,57%
  Current LE             51200
  Segments               1
  Allocation             inherit
  Read ahead sectors     auto
  - currently set to     256
  Block device           251:6
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!