Not sure I set my server up the best way. Question about storage

ILikeBeans86

New Member
Jan 28, 2022
5
0
1
37
I just set up proxmox and evrerything has been running fine but now i'm wondering if i set it up the best way for my system. I have 1 SSD i formatted as ext4 for the proxmox install and then i have a 1TB SSD I formatted with ZFS for the VMs. I think i used this video to set up the ZFS drive. https://www.youtube.com/watch?v=m0dY4OJ9FWk&t=238s&ab_channel=DylanBlanque. The ZFS drive is mounted as a directory which seems to not be what is normally done. However I did migrate from HyperV so I did need somewhere to copy my VMs to since i want creating them from scratch. I converted them to qcow2 I think because I read those can be snapshotted? Im used to HyperVs dynamic discs where the VM file takes up smaller amount of space than what it is offered if its not needed. I think thats why i set it up the way i did since that seemed to make sense. It seems you can also achive this with zvols or lvm-thin. Is that correct? Could I even convert my VMs to something like that (going from just a file containing VMs to block level)? Hope my question makes sense. Thanks.
 
The simplest way would be to have a storage of type ZFS. If you do not have it already, create one. For some separation, maybe create a ZFS dataset for it first. Enable the "thin provision" checkbox if you want. This setting comes into play, whenever a new disk image is created.

Then you can use the "move disk" option in the VMs hardware panel to move the disk from the old qcow file on ZFS to the native ZFS storage (even while the VM is running).

ZFS offers two types of datasets. The most common one is the file system and the other, less known, is the volume (zvol) dataset type. It provides a block device which is used directly for the VM disk. It will not show up in the file system hierarchy, but you can still see it by running zfs list. The block devices are exposed in the /dev directory as /dev/zdX. Since that does not help to map the dataset name to the actual block device, you also have the /dev/zvol/<pool>/<dataset> paths.
 
The simplest way would be to have a storage of type ZFS. If you do not have it already, create one. For some separation, maybe create a ZFS dataset for it first. Enable the "thin provision" checkbox if you want. This setting comes into play, whenever a new disk image is created.

Then you can use the "move disk" option in the VMs hardware panel to move the disk from the old qcow file on ZFS to the native ZFS storage (even while the VM is running).

ZFS offers two types of datasets. The most common one is the file system and the other, less known, is the volume (zvol) dataset type. It provides a block device which is used directly for the VM disk. It will not show up in the file system hierarchy, but you can still see it by running zfs list. The block devices are exposed in the /dev directory as /dev/zdX. Since that does not help to map the dataset name to the actual block device, you also have the /dev/zvol/<pool>/<dataset> paths.
So i already have a ZFS pool for the 1tb SSD which is mounted in a directory and my qcow2 disks are stored there. From what I can tell they seem to be "dynamic disks" or spare or whatever the terminoligy is on proxmox. They don't seem to take up the ammounted alloted for the disk. So would doing it the other way be worth it? I have backups of everything could i j ust remove the directory mount and restore the VMs to the zfs pool? Would Moving the disk directly to the ZFS pool while it is still mounted be possible? would lvm-thin be a better option if my VMs live on 1 single SSD (OS is on a differnt SSD)? Thanks
 
A bit of background: The Storage configurations that you have in the GUI under Datacenter -> Storage (/etc/pve/storage.cfg is the config file) tells Proxmox VE where it can store things and how.

If you already have a ZFS pool, you can add additional storages there. I recommend to at least manually create a new dataset to have a clear separation. For example zfs create <pool>/<new dataset>. If you add a new storage of the type ZFS you can select which dataset should be used for it.

If you do that, you have two datasets, the one of type directory that you configured previously and the new one. This makes it possible to use the "Move Disk" functionality to (live) move the disk images between storages. Once you have nothing left in the old one, you could remove it. This way you avoid a full restore of the VMs.

From what I can tell they seem to be "dynamic disks" or spare or whatever the terminoligy is on proxmox. They don't seem to take up the ammounted alloted for the disk.
This is called thin provisioning or sparse files. The same can be achieved by enabling the "thin provision" checkbox in the storage configuration for the "new" ZFS based storage. Technically in the case of ZFS this just makes a difference of reserving the space or not for the dataset containing the disk image.

would lvm-thin be a better option if my VMs live on 1 single SSD
I personally would still choose ZFS, as it will tell you if data is broken (due to a bit flip for example) and you still have the option to easily add another disk of similar size and attaching it in a way to form a mirror and get some redundancy later on.
 
If you already have a ZFS pool, you can add additional storages there. I recommend to at least manually create a new dataset to have a clear separation. For example zfs create <pool>/<new dataset>. If you add a new storage of the type ZFS you can select which dataset should be used for it.
I think ended up doing that when i created the mountpoint. This was more of less the command i ran except i named mine a little differently. zfs create -o mountpoint=/mnt/ZFSA zfsa/ZFSA. In the gui i can add things to both entries and if I go through the motions of moving a disk i am able to select the non directory version.
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!