[SOLVED] Undestrand discard option

[Valerio Pachera]

How discard generally work:
consider a logical volume (lv) of a thin pool.
Once created a lv, the allocated spaced is 0.
I create a filesystem on in (i.e. ext4), add some data till it allocates 5G.
I remove 2G of files but the allocated space is still 5G.
By fstrim, it will de-allocate/discard the 2G so only 3G will be allocated.

I the example above, the filesystem (ext4) is stored directly on the lv, without another partition table as it happens if the lv is used as virtual disk for qemu/proxmox.

My first question is, does discard work also in this case?
Or shall I use discard only on lv with no partition table?

My second question is: when will data be discard by proxmox? At guest shutdown/restart?

Third question: do all thin provisioning storage (lvm, zfs, qcow2) work the same way?

Thank you.

 

[Alwin]

My first question is, does discard work also in this case?
Or shall I use discard only on lv with no partition table?

The filesystem tells the underlying storage to discard blocks, as only the filesystem knows what to throw away and what not. So it doesn't matter if partitioned or not, discard is done on a mounted filesystem basis.

My second question is: when will data be discard by proxmox? At guest shutdown/restart?

Discard is a linux feature, nothing special is done by proxmox. Depending on how you set it up, then either it will be done immediate or on your manual schedule (see links below).

Third question: do all thin provisioning storage (lvm, zfs, qcow2) work the same way?

No. But for discard, they all support it.

Setup and use on PVE:
https://pve.proxmox.com/pve-docs/pve-admin-guide.html#qm_virtual_machines_settings
https://pve.proxmox.com/wiki/Shrink_Qcow2_Disk_Files

Further infos:
https://wiki.debian.org/SSDOptimization
https://wiki.archlinux.org/index.php/Solid_State_Drives

 

[Valerio Pachera]

I read the documentation and I get confused because pve-admin guide says

your VM has a SCSI controller you can activate the Discard

while shrink qocw2 file guide says

Virtio-SCSI disks with the discard option enabled

.

I find it more logic to be virtio-scsi the required controller.

Please, help me finding the way to discard the unused blocks from this thin volume:

  LV Path                /dev/pve-big/vm-100-disk-1
  LV Name                vm-100-disk-1
  VG Name                pve-big
  LV UUID                DVg5YO-ZcNe-C4HV-mrfj-Z0rq-FgeO-2TFRv2
  LV Write Access        read/write
  LV Creation host, time cavaion-proxmox-ve, 2016-12-20 08:34:28 +0100
  LV Pool name           vm
  LV Status              available
  # open                 1
  LV Size                200,00 GiB
  Mapped size            99,62%
  Current LE             51200
  Segments               1
  Allocation             inherit
  Read ahead sectors     auto
  - currently set to     256
  Block device           251:6

It's used by a windows 2012 server R2 guest.
Drive C: was almost full, then lot's of data has been removed.




The guest has virtio controller as controller and discard is enabled as you can see here

-drive file=/dev/pve-big/vm-100-disk-1,if=none,id=drive-virtio0,format=raw,cache=none,aio=native,detect-zeroes=on

I tried to stop and restart the guest, but the thin volume is still 99% mapped.

For those Windows versions which support TRIM you should not have to setup anything, Windows should periodically trigger a TRIM to the disks.

I hope I'm not missing anything.

PS: the host is a bebian jessie, proxmox 4.4-1/eb2d6f1e.

 

[Alwin]

How does your VM config look like 'qm config <vmid>'? Looks to me, as if discard is not activated.

 

[Valerio Pachera]

Here it is

qm config 100
agent: 1
balloon: 2048
boot: c
bootdisk: ide0
cores: 8
cpu: host
ide2: local:iso/virtio-win.iso,media=cdrom,size=152204K
keyboard: it
localtime: 1
memory: 6144
name: win2012
net0: e1000=0E:4C:07:AF:81:06,bridge=vmbr0
numa: 1
ostype: win8
scsihw: virtio-scsi-pci
smbios1: uuid=1270bf4b-1e1c-4df3-918a-a2414488e156
sockets: 1
unused0: big:vm-100-disk-2
virtio0: big:vm-100-disk-1,size=200G
virtio1: big:vm-100-disk-3,backup=0,cache=writeback,discard=on,size=100G

 

[mbaldini]

Have you tried with disk type SCSI instead of Virtio? I have some Windows 2008R2 and 2012R2 VMs and with discard=on, SCSI disk and Virtio-SCSI, I see the correct behaviour, when Windows perform maintenance and there are a lot of deleted files, the ZFS used size will decrease. I think should be the same with LVM-thin.

 

[Valerio Pachera]

Ops, I forgot to enable options discard the last time I removed and re-added the vdi to be sure it was using virtio controller.
Now I checked it twice
Anyway, after the stop and restart of the guest with discard option enabled, the "Mapped size" is still 99%.

qm config 100 | grep vm-100-disk-1
virtio0: big:vm-100-disk-1,discard=on,size=200G

lvdisplay /dev/pve-big/vm-100-disk-1 | grep Mapped
  Mapped size            99,64%

When should I expect windows to trigger discard/trim?
I'll try later with scsi controller if I see no results.

Thank you.

 

[mbaldini]

Windows Server will perform TRIM operation during maintenance time

 

[Valerio Pachera]

Hi, I checked the maintenance schedule and it was run last time this morning.
I also started it manually as you can see in this image.



My server language is set in italian.
I used this guide as reference.

Mapped size is stll 99% even after running maintenance.

 

[mbaldini]

This is a config on one of my Windows 2012 VM

qm config 100
-cut-
scsi0: local-zfs:vm-100-disk-1,cache=writeback,discard=on,size=64G
scsi1: local-zfs:vm-100-disk-2,cache=writeback,discard=on,size=250G
scsi2: local-zfs:vm-100-disk-3,cache=writeback,discard=on,size=220G
scsi3: local-zfs:vm-100-disk-4,cache=writeback,discard=on,size=50G
-cut-
scsihw: virtio-scsi
-cut-

I use ZFS and not LVM and I use SCSI driver (with virtio-scsi driver installed in Windows) and VirtIO SCSI controller.
Have you tried switching to virtio-scsi driver?

 

[Valerio Pachera]

Have you tried switching to virtio-scsi driver?

I'm using virtio driver.
I notice anyway that you config says
scsihw: virtio-scsi
and mine
scsihw: virtio-scsi-pci

Any idea about the difference and if it may influence discard?

 

[robhost]

Afaik you have to change this directly in the <VMID>.conf under /etc/pve:

From

virtio0: big:vm-100-disk-1,size=200G
virtio1: big:vm-100-disk-3,backup=0,cache=writeback,discard=on,size=100G

to

scsi0: big:vm-100-disk-1,size=200G
scsi1: big:vm-100-disk-3,backup=0,cache=writeback,discard=on,size=100G

 

[Valerio Pachera]

I success with a linux guest and scsi driver

scsi0: big:vm-103-disk-1,cache=writeback,discard=on,size=30G
scsihw: virtio-scsi-pci

I'll try also with windows asap.

 

[mbaldini]

When I have switched my Windows VMs from virtio to scsi disk I had to wotk this way:
- poweroff the Windows VM
- add a new disk with scsi typ, I just used 1GB in size
- started Windows VM
- from Device Manager, installed the virtio-scsi driver on the unknown device (the new disk)
- made a test in disk management, new disk is online, formatted and so on
- powered off the Windows VM
- removed the virtio disks and re-added as scsi disks
- started Windows VM

During the Windows maintenance time, the space used in ZFS pool has decreased.

 

[Valerio Pachera]

Great, it worked also on windows!
Thank you all for you support!

lvdisplay /dev/pve-big/vm-100-disk-1
  --- Logical volume ---
  LV Path                /dev/pve-big/vm-100-disk-1
  LV Name                vm-100-disk-1
  VG Name                pve-big
  LV UUID                DVg5YO-ZcNe-C4HV-mrfj-Z0rq-FgeO-2TFRv2
  LV Write Access        read/write
  LV Creation host, time cavaion-proxmox-ve, 2016-12-20 08:34:28 +0100
  LV Pool name           vm
  LV Status              available
  # open                 1
  LV Size                200,00 GiB
  Mapped size            57,57%
  Current LE             51200
  Segments               1
  Allocation             inherit
  Read ahead sectors     auto
  - currently set to     256
  Block device           251:6