VM Storage Location

[Splise]

Hi,

When creating a new VM, the VM images are located in a different location than what I have specified. I also see four copies of the disk image when creating a VM. How do I take /dev out of the picture, and use only /rpool/data/vmstorage/vm-images, and only have one copy of the image? Any help is appreciated.

# cat /etc/pve/storage.cfg
dir: local
        path /var/lib/vz
        content iso,vztmpl,backup

zfspool: local-zfs
        pool rpool/data
        content rootdir,images
        sparse 1

zfspool: zfs-containers
        pool rpool/data/vmstorage
        content rootdir
        mountpoint /rpool/data/vmstorage
        sparse 0

zfspool: vm-disks
        pool rpool/data/vmstorage/vm-disks
        content images,rootdir
        mountpoint /rpool/data/vmstorage/vm-disks
        sparse 1

dir: zfs-templates
        path /rpool/data/vmstorage
        content iso,vztmpl
        shared 0

dir: zfs-backups
        path /rpool/data/share
        content backup
        maxfiles 7
        shared 0

find / -name vm-101-disk-0
/dev/rpool/data/vm-101-disk-0
/dev/rpool/data/vmstorage/vm-disks/vm-101-disk-0
/dev/zvol/rpool/data/vm-101-disk-0
/dev/zvol/rpool/data/vmstorage/vm-disks/vm-101-disk-0

lsblk
NAME   MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
sda      8:0    0 232.9G  0 disk
├─sda1   8:1    0  1007K  0 part
├─sda2   8:2    0   512M  0 part
└─sda3   8:3    0 232.4G  0 part
sdb      8:16   0 232.9G  0 disk
├─sdb1   8:17   0  1007K  0 part
├─sdb2   8:18   0   512M  0 part
└─sdb3   8:19   0 232.4G  0 part
zd0    230:0    0    60G  0 disk
zd16   230:16   0     2G  0 disk
zd32   230:32   0    32G  0 disk

zfs list
NAME                                          USED  AVAIL     REFER  MOUNTPOINT
rpool                                        8.40G   216G      104K  /rpool
rpool/ROOT                                   1.88G   216G       96K  /rpool/ROOT
rpool/ROOT/pve-1                             1.88G   216G     1.63G  /
rpool/data                                   6.51G   216G      104K  /rpool/data
rpool/data/share                             5.70G   216G     5.70G  /rpool/data/share
rpool/data/vm-101-disk-0                       56K   216G       56K  -
rpool/data/vmstorage                          835M   216G      834M  /rpool/data/vmstorage
rpool/data/vmstorage/limited                   96K   216G       96K  /rpool/data/vmstorage/limited
rpool/data/vmstorage/vm-disks                 276K   216G       96K  /rpool/data/vmstorage/vm-disks
rpool/data/vmstorage/vm-disks/vm-100-disk-0    56K   216G       56K  -
rpool/data/vmstorage/vm-disks/vm-100-disk-1    56K   216G       56K  -
rpool/data/vmstorage/vm-disks/vm-101-disk-0    68K   216G       68K  -

 

[macleod]

well, you have 3 nested zfs mounts in your storage.cfg; although zfs can handle that, for me it's just "asking for trouble"
it should try to separate (remember: divide et impera) and put the filesystems on the same level, it will be easier for you in the future

 

[Splise]

well, you have 3 nested zfs mounts in your storage.cfg; although zfs can handle that, for me it's just "asking for trouble"
it should try to separate (remember: divide et impera) and put the filesystems on the same level, it will be easier for you in the future

Thanks, that makes sense for the most part. I was trying to configure a storage structure that is organized/logical, but does not deviate from what Proxmox is expecting. Is the new storage configuration defined below a better way to go?

Using the structure below (and this is probably a dumb question), will there always be a duplicate path when creating a new VM? When creating anew VM, the disk image is located under both '/dev/rpool/vm-disks/' and '/dev/zvol/rpool/vm-disks/'. Is that by design? Why is it not located directly under 'rpool/vm-disks/' instead? Even if I remove the 'vm-disks-thick' and 'vm-disks-thin'ZFS pools, and just use 'vm-disks', there is always two images created under' dev/', but never under 'rpool/'.

zfs list
NAME               USED  AVAIL     REFER  MOUNTPOINT
rpool             2.70G   222G      834M  /rpool
rpool/ROOT        1.88G   222G       96K  /rpool/ROOT
rpool/ROOT/pve-1  1.88G   222G     1.63G  /
rpool/vm-disks      96K   222G       96K  /rpool/vm-disks

/etc/pve/storage.cfg
dir: local
        path /var/lib/vz
        content backup,vztmpl,iso

zfspool: zfs-containers
        pool rpool
        content rootdir
        mountpoint /rpool
        sparse 0

dir: zfs-templates
        path /rpool
        content vztmpl,iso
        shared 0

dir: zfs-backups
        path /rpool
        content backup
        maxfiles 7
        shared 0

zfspool: vm-disks-thick
        pool rpool/vm-disks
        blocksize 4k
        content rootdir,images
        mountpoint /rpool/vm-disks
        sparse 0

zfspool: vm-disks-thin
        pool rpool/vm-disks
        blocksize 4k
        content images,rootdir
        mountpoint /rpool/vm-disks
        sparse 1

Thanks again for your help.

 

[macleod]

I would move everything from STRAIGHT /rpool (container, templates, etc.) to /rpool/something
also the usage of same pool for thin but also thick images is "interesting"; cannot say if it's good or bad without further testing; but for you it will be hard to tell from a glance if a zvol is thick or thin just by looking at its path (or maybe it's not really important)