Proxmox disks pool design.

[daniel1302]

Hello. I am trying to migrate the test environments for my company out of the cloud. Currently, We are using ZFS everywhere but do not manage physical servers in the cloud.

I have two versions of the same application.

app_a: Running on `proxmox_node_a`. It has EXT4 on the physical server and ZFS in the VM. 800GB disk is passed to VM.
app_b: Running on `proxmox_node_b`. It has ZFS on the physical server and EXT4 in the VM. 800GB disk is passed to VM.

I can see the following differences

app_a(ZFS in VM):

daniel@app_a:~$ df -h
Filesystem                         Size  Used Avail Use% Mounted on
...
data_pool                          772G  270G  502G  35% /mnt/data

root@app_a:/mnt/data# du -sh ./*
270G    ./data
root@app_a:/mnt/data# du -sh ./* --apparent-size
538G    ./data

daniel@app_a:~$ zfs get compressratio
NAME           PROPERTY       VALUE  SOURCE
data_pool     compressratio  2.00x  -

app_b(ZFS on Proxmox):

daniel@app_b:~$ df -h
Filesystem                         Size  Used Avail Use% Mounted on
...
/dev/sdb                           787G  538G  209G  73% /mnt/data
root@app_b:/mnt/data# du -sh ./*
538G    ./data
16K     ./lost+found
root@app_b:/mnt/data# du -sh ./* --apparent-size
538G    ./data
16K     ./lost+found

I have compression enabled for both apps:
- For the app_a compression is configured on the VM level
- For the app_b compression is configured on the Proxmox ZFS Level.

This is what I can see proxmox:
- In the GUI for both servers, I can see 800 GB is reserved for VMs,


proxmox_node_a:

root@proxmox_node_a:/home/daniel# lvs -a
  LV              VG            Attr       LSize    Pool Origin Data%  Meta%  Move Log Cpy%Sync Convert
  vm-1007-disk-0  lvm-ssd-small -wi-ao----  800.00g

proxmox_node_b:

daniel@proxmox_node_b:~$ zfs list
NAME                              USED  AVAIL  REFER  MOUNTPOINT
zfs-ssd-small-1/vm-1006-disk-0    813G   410G   423G  -

root@proxmox_node_b:/home/daniel# zfs get compressratio
NAME                             PROPERTY       VALUE  SOURCE
zfs-ssd-small-1/vm-1006-disk-0   compressratio  1.43x  -

So the difference is that on node_b my VM sees only 200GB free, and 538GB used, whereas proxmox sees a full 813 GB used.


I have ZFS in the VM I can see much better reporting in proxmox(equal 800GB vs 813GB), a much better compression ratio(2.0x vs 1.4x), and files are taking less in the VM, so the VM can write more data.


Questions:

1. Why would I create ZFS on Proxmox if compression is not reported in the VM?
2. Why compression for app_b is not reflected in the proxmox? I can see 813GB where the compression ratio is 1.41 and I should see about 576 GB.
3. My assumption is ZFS on Proxmox makes sense only with LXC, where We have mount points. Is it correct?

 

[aaron]

1. Why would I create ZFS on Proxmox if compression is not reported in the VM?

I am a bit confused. In case a, the VM is using ZFS itself and can therefore know how much the data is compressed. In case b, the disk image of the VM is stored on ZFS handles by the host. How should the VM know about the compression level? It has no knowledge of where and how its disk is actually stored. It could be any storage type Proxmox VE has to offer, or even a physical disk directly, the VM doesn't care or know.

Did you enable "thin provision" for the ZFS storage? If not, then the disk image will reserve the overall space. Run zfs get all zfs-ssd-small-1/vm-1006-disk-0 and post the results.