How to shrink ZFS within VM?

[turnicus]

Hello,

I converted an .OVA Nextcloud appliance from hanssonit to use on proxmox. The created VM had 1x 40GB disk for the OS + 1x 40GB disk for the data (which is a zfs "ncdata" pool mounted at /mnt/ncdata within the VM). Both are virtual disks hosted on a ZFS storage of PVE (vm-101-disk-0 and vm-101-disk-1), so I am actually using ZFS on top of ZFS (ZFS in the VM which is hosted on a ZFS PVE storage).

40GB was not enough for the data so I expanded vm-101-disk-1 up to 5000GB! My idea was to give a lot with thin provisioning so I wouldn't have to expand again. But I realized afterwards that nextcloud performs its own hourly ZFS snapshots so the disk is filling very quickly when I add / delete many files in nextcloud (it doesn't prune these snapshots because there is still a lot of space). Right now, only ~300GB of files are hosted on nextcloud but it uses ~1100GB because of the many files I added/deleted which are in snapshots.

I am not ready to give 5000GB yet so I would like to shrink back to 1500GB. The documentation at https://pve.proxmox.com/wiki/Resize_disks explains that I need to prepare the VM before executing a qm resize (which sounds logical). I know how to shrink an EXT4 partition within the VM with gparted but I read it is impossible to shrink a ZFS pool. I also read contradictory methods and I already screwed it (currently restoring from backup...)

What method would you use in my case?

Thanks for any help!

 

[wolfgang]

Hi,

you can't shrink a zvol, either a zpool.
Generally, I would not recommend you to use ZFS on ZFS.

 

[Rainerle]

@wolfgang , so what would you recommend, if you want ZFS within the VM to use snapshots on a regular basis like 15 minutes? Is ZFS within the VM on top of CephFS ok?

 

[wolfgang]

Hi Rainerle,

No this is more or less the same.

If you like snapshots do it on the host.

 

[Rainerle]

So which Filesystem do you recommend within the VM? How to prevent bit rotting - should that not be done on the top level within the VM?

 

[turnicus]

Hi,

you can't shrink a zvol, either a zpool.
Generally, I would not recommend you to use ZFS on ZFS.

Hello. Unfortunately I have no choice because Nextcloud ships with ZFS built-in... Can you recommend a technique to shrink the ZFS pool created by Nextcloud (within the VM)?

 

[wolfgang]

It is not possible to shrink a zfs pool with ZFS command.
You must copy or sync/receive the pool.
I would recommend you to install Nexcloud on your own. [1]

1.) https://docs.nextcloud.com/server/20/admin_manual/installation/example_ubuntu.html

 

[turnicus]

It is not possible to shrink a zfs pool with ZFS command.
You must copy or sync/receive the pool.
I would recommend you to install Nexcloud on your own. [1]

1.) https://docs.nextcloud.com/server/20/admin_manual/installation/example_ubuntu.html

Thanks Wolfgang. Here is what I did to successfully "shrink" the pool (in case someone else is interested):

1- boot into nextcloud and note the values of: zpool get all ncdata

2- zpool export ncdata

3- shutdown nextcloud

4- add a smaller disk to the instance

5- boot on live ubuntu and:
zpool import ncdata
zpool create ncdatabackup /dev/sdX (the smaller disk)
- change the zpool settings of ncdatabackup to match those of ncdata (I had to change "failmode=continue" and "ashift=12")
zfs snapshot -r ncdata@backup
zfs send -R ncdata@backup | zfs receive -Fdvu ncdatabackup
zpool export ncdata
zpool export ncdatabackup
zpool import ncdatabackup ncdata (it changes the name of ncdatabackup to ncdata)
- double check everything looks OK with the values noted in step1
zpool export ncdata
shutdown

6- detach live CD and the original big drive from the instance (+ update the boot order)

7- boot into nextcloud