trim discard support for ceph block device

[xxyton]

Proxmox does support Ceph block devies (RBD) but I am not sure if there is support for trim discard yet?

I could find a discussion on the pve-devel mailing list from 2012: http://pve.proxmox.com/pipermail/pve-devel/2012-November/004419.html
At that time the feature was experimental so there was no conclusion. But nowadays trim is supported within recent kernel and qemu versions.

Please clarify if recent Proxmox version 3.1 does support trim discard on RBD.

 

[dietmar]

You can set the option manually, for example:

virtio0: local:100/vm-100-disk-1.qcow2,discard=on

I have node idea how stable that is.

 

[xxyton]

I tried discard=on but thats not working :-(
pve:/nodes/test/qemu/100> set config -virtio0 ceph:vm-100-disk-1,discard=on
400 Parameter verification failed.
virtio0: invalid format - unable to parse drive options

pveversion -v
proxmox-ve-2.6.32: 3.1-114 (running kernel: 2.6.32-26-pve)
pve-manager: 3.1-21 (running version: 3.1-21/93bf03d4)
pve-kernel-2.6.32-20-pve: 2.6.32-100
pve-kernel-2.6.32-26-pve: 2.6.32-114
lvm2: 2.02.98-pve4
clvm: 2.02.98-pve4
corosync-pve: 1.4.5-1
openais-pve: 1.1.4-3
libqb0: 0.11.1-2
redhat-cluster-pve: 3.2.0-2
resource-agents-pve: 3.9.2-4
fence-agents-pve: 4.0.0-2
pve-cluster: 3.0-8
qemu-server: 3.1-8
pve-firmware: 1.0-23
libpve-common-perl: 3.0-8
libpve-access-control: 3.0-7
libpve-storage-perl: 3.0-17
pve-libspice-server1: 0.12.4-2
vncterm: 1.1-4
vzctl: 4.0-1pve4
vzprocps: 2.0.11-2
vzquota: 3.1-2
pve-qemu-kvm: 1.4-17
ksm-control-daemon: 1.1-1
glusterfs-client: 3.4.1-1

 

[tom]

post your vm config.

> qm config VMID

(I can start a ceph disk with discard=on here.)

 

[dietmar]

That needs latest pakackes from pve-no-subscription.

 

[spirit]

also, you need to use scsi or virtio-scsi controller for your guest to get it work.
(and also discard support in guest filesystem)

 

[xxyton]

Thank you, dietmar and spirit.
With latest packages from pve-no-subscription and virtio-scsi it is working as expected with Linux VMs.

However I discovered that trim is not working with Windows VMs.
This is caused by missing -M pc,trim flag, see: http://lists.gnu.org/archive/html/qemu-devel/2013-07/msg04136.html

@dietmar: could you change Proxmox's behavior to add this flag automatically once discard=on is set? A checkbox for the discard option within the webinterface would be nice as well.

 

[spirit]

Thank you, dietmar and spirit.
With latest packages from pve-no-subscription and virtio-scsi it is working as expected with Linux VMs.

However I discovered that trim is not working with Windows VMs.
This is caused by missing -M pc,trim flag, see: http://lists.gnu.org/archive/html/qemu-devel/2013-07/msg04136.html

@dietmar: could you change Proxmox's behavior to add this flag automatically once discard=on is set? A checkbox for the discard option within the webinterface would be nice as well.

Hi, the msg in said

" With -M pc,trim works well in windows guest with ide device and ahci bus"

so for ide and sata. But for virtio-iscsi or scsi, doesn't it work out of the box ?

 

[dietmar]

@dietmar: could you change Proxmox's behavior to add this flag automatically once discard=on is set?

I am unable to find the machine flag 'trim'. It is not documented and I cannot see it in the qemu source code?

 

[xxyton]

@spirit: It does not work out of box for Windows guests. You need to have virtio-scsi AND '-M pc' flag added.

@dietmar: I'm sorry for the confusion, I made a typo. I meant to write: "This is caused by missing -M pc trim flag" So the flag that's to be added is '-M pc'

 

[aderumier]

@spirit: It does not work out of box for Windows guests. You need to have virtio-scsi AND '-M pc' flag added.

@dietmar: I'm sorry for the confusion, I made a typo. I meant to write: "This is caused by missing -M pc trim flag" So the flag that's to be added is '-M pc'

Hi, this is strange, because "-M pc", is the default configuration of kvm.

this can be used to simulate previous qemu hardware implementation (('pc is an alias of last version)

# kvm -machine ?
Supported machines are:
none                 empty machine
pc                   Standard PC (i440FX + PIIX, 1996) (alias of pc-i440fx-1.7)
pc-i440fx-1.7        Standard PC (i440FX + PIIX, 1996) (default)
pc-i440fx-1.6        Standard PC (i440FX + PIIX, 1996)
pc-i440fx-1.5        Standard PC (i440FX + PIIX, 1996)
pc-i440fx-1.4        Standard PC (i440FX + PIIX, 1996)
pc-1.3               Standard PC (i440FX + PIIX, 1996)
pc-1.2               Standard PC (i440FX + PIIX, 1996)
pc-1.1               Standard PC (i440FX + PIIX, 1996)
pc-1.0               Standard PC (i440FX + PIIX, 1996)
pc-0.15              Standard PC (i440FX + PIIX, 1996)
pc-0.14              Standard PC (i440FX + PIIX, 1996)
pc-0.13              Standard PC (i440FX + PIIX, 1996)
pc-0.12              Standard PC (i440FX + PIIX, 1996)
pc-0.11              Standard PC (i440FX + PIIX, 1996)
pc-0.10              Standard PC (i440FX + PIIX, 1996)
isapc                ISA-only PC
q35                  Standard PC (Q35 + ICH9, 2009) (alias of pc-q35-1.7)
pc-q35-1.7           Standard PC (Q35 + ICH9, 2009)
pc-q35-1.6           Standard PC (Q35 + ICH9, 2009)
pc-q35-1.5           Standard PC (Q35 + ICH9, 2009)
pc-q35-1.4           Standard PC (Q35 + ICH9, 2009)

are you sure that virtio-scsi is not enough ?

 

[xxyton]

are you sure that virtio-scsi is not enough ?

For Linux it works just fine but apparently not for Windows.
I tested with newest Proxmox version from pve-no-subscription, virtio-scsi, discard=on and a Windows 7 guest.
While fsutil behavior query DisableDeleteNotify returns 0 (means discard is on) deleted disk space won't be freed on the Ceph device, even after more than 24 hours.

@dietmar: Can you confirm that '-m pc' is set by default? Any other suggestions?

 

[dietmar]

@dietmar: Can you confirm that '-m pc' is set by default?

Yes

 

[xxyton]

Is anyone successfully* running Windows guests with trim support?

* blocks on the storage are actually freed

 

[spirit]

Is anyone successfully* running Windows guests with trim support?

* blocks on the storage are actually freed

just found a bug report on redhat bugzilla
https://bugzilla.redhat.com/show_bug.cgi?format=multiple&id=1056379

"
I found this in the MS documentation:

> Windows Optimize Drives Utility
>
> End users or system administrators can use the Optimize Drives utility to
> reclaim space either by creating a manual request or by optimizing the
> schedule configuration. If the disk drive is a thin provisioning LUN, the
> media type of the disk drive appears as “Thin Provisioning Drive”.

To access the utility:

1) Right click on computer, "Manage".

2) Click on storage, right click on "disk management", "Properties".

3) Tools tab, then "Optimize".

I don't yet have access to Windows 8 or Server 2012, will look at this as soon as possible.
"