Understanding lvm-thin volumes and storage

[fenichelar]

On a brand new install of 4.2, I created a thin volume in the "pve/data" pool that was created automatically. I then created a new container and assigned the new thin volume as the root storage. However, in the GUI, I see the disk image under both the new lvm-thin volume and the "local-lvm" volume that was created automatically. I thought that it would only show up under the new volume. Why does it show up under both? Thanks!

 

[dietmar]

However, in the GUI, I see the disk image under both the new lvm-thin volume and the "local-lvm" volume that was created automatically.

Sorry, I do not understand what you mean exactly?

 

[fenichelar]

If I select either lvm-thin volume ("local-lvm" or "plex") and click "Content" I see "vm-100-disk-1". I was only expecting to see this under "plex" as that is what I selected as the root disk for container 100. Why do I also see it when I select "local-lvm"?

 

[fabian]

please post your "pveversion -v" output and storage configuration ("/etc/pve/storage.cfg")

 

[fenichelar]

pveversion -v

proxmox-ve: 4.3-66 (running kernel: 4.4.19-1-pve)
pve-manager: 4.3-1 (running version: 4.3-1/e7cdc165)
pve-kernel-4.4.19-1-pve: 4.4.19-66
lvm2: 2.02.116-pve3
corosync-pve: 2.4.0-1
libqb0: 1.0-1
pve-cluster: 4.0-46
qemu-server: 4.0-88
pve-firmware: 1.1-9
libpve-common-perl: 4.0-73
libpve-access-control: 4.0-19
libpve-storage-perl: 4.0-61
pve-libspice-server1: 0.12.8-1
vncterm: 1.2-1
pve-qemu-kvm: 2.6.1-6
pve-container: 1.0-75
pve-firewall: 2.0-29
pve-ha-manager: 1.0-35
ksm-control-daemon: 1.2-1
glusterfs-client: 3.5.2-2+deb8u2
lxc-pve: 2.0.4-1
lxcfs: 2.0.3-pve1
criu: 1.6.0-1
novnc-pve: 0.5-8
zfsutils: 0.6.5.7-pve10~bpo80

/etc/pve/storage.cfg

dir: local
    path /var/lib/vz
    content iso,backup,vztmpl

lvmthin: local-lvm
    vgname pve
    thinpool data
    content rootdir,images

lvmthin: plex
    vgname pve
    thinpool data
    content images,rootdir

lvs

  LV                    VG   Attr       LSize  Pool Origin Data%  Meta%  Move Log Cpy%Sync Convert
  data                 pve  twi-aotz--  8.43g             30.11  10.84
  root                  pve  -wi-ao----  3.75g
  swap                pve  -wi-ao----  1.88g
  vm-100-disk-1  pve  Vwi-aotz--  9.00g data        9.70

 

[dietmar]

Your storage named 'plex' is a duplicate of storage 'local-lvm'. It is a bad idea to define things twice.

 

[fenichelar]

I am new and trying to understand lvm. I created a new "lvm-thin" through the GUI. I selected the "pve" volume group and "data" thin pool. What should I have done?

 

[dietmar]

What should I have done?

Why don't you simply use the existing storage named 'local-lvm' ?

 

[fenichelar]

Why don't you simply use the existing storage named 'local-lvm' ?

So let me take a step back and explain exactly what I am trying to do. I want to create 2 containers, one to run Plex and one to rip blu rays. I have a 512GB SSD and a 2TB HDD. I want to install Proxmox and both containers on the SSD for performance. I have an NFS server that I plan to use for daily backups of both containers. The blu ray ripping machine will mount the 2TB HDD as temporary storage, I will just tell it not to backup the mount point. The problem comes in because of the Plex container. My Plex library folder (thumbnails, metadata, etc.) is 100GB and needs to be on the SSD. For the Plex box, I don't want it to backup the library folder using Proxmox because it would be too large. But I will backup the Plex library folder using something like rsync. So what I was planning to do is create a thin volume on the SSD for each container's root disk. I would also create a thin volume just for the Plex library data. Then I would create a volume group on the HDD and a thin volume to use as temporary storage for the ripping box. This would allow me to create additional thin volumes on the HDD if needed later. Does this sound like a reasonable plan? How do I create these thin volumes? I thought what I did was creating a thin volume for the root disk but I apparently just made a duplicate.

 

[fabian]

you cannot create a PV, VG, thin pool or other LV via the GUI/API. you can only add a storage definition for an already existing VG or thin pool. check the man pages for "pvcreate", "vgcreate" and "lvcreate" if you want to find out how to prepare your disks for LVM usage.

the only storage which you can actually create on the GUI/API/CLI is Ceph (which is not the right choice for your scenario ) - hopefully this will improve in the future.

 

[fenichelar]

you cannot create a PV, VG, thin pool or other LV via the GUI/API. you can only add a storage definition for an already existing VG or thin pool. check the man pages for "pvcreate", "vgcreate" and "lvcreate" if you want to find out how to prepare your disks for LVM usage.

the only storage which you can actually create on the GUI/API/CLI is Ceph (which is not the right choice for your scenario ) - hopefully this will improve in the future.

Okay, just to make sure I understand. I will use "local-lvm" as the root drive for both containers. I will create the additional volumes like so:

$ lvcreate -n plex -V 100G pve/data
$ mkfs.ext4 /dev/pve/plex
$ mkdir /mnt/plex
$ echo '/dev/pve/plex /mnt/plex ext4 defaults 0 2' >> /etc/fstab
$ mount -a

$ sgdisk -N 1 /dev/sdb
$ pvcreate --metadatasize 250k -y -ff /dev/sdb1
$ vgcreate hdd1 /dev/sdb1
$ lvcreate -L 2000G -T -n rip hdd1
$ mkfs.ext4 /dev/hdd1/rip
$ mkdir /mnt/rip
$ echo '/dev/hdd1/rip /mnt/rip ext4 defaults 0 2' >> /etc/fstab
$ mount -a

I will then add the directory "/mnt/plex" and "/mnt/rip" to storage using the GUI. Should the content be "Container"? I will then add a mount point to each container for the needed directory using the GUI. What should be selected as the disk size when creating the mount point? Does this sound like the best way to set everything up? Thanks for the help.

 

[fabian]

Creating volumes/disks for containers and assigning (and mounting) them is all handled by PVE - you only need to create the storage itself. if I understood you correctly, you just need to create an LVM storage on your big hard disk (pvcreate, vgcreate) and add it as storage in PVE. then you can select the correct storage when creating a container or adding a mount point to an existing container. PVE will automatically create the needed volumes (logical volumes in this case, since the storage is LVM(-thin)) with the configured size.

There is no need to use lvcreate manually unless you want to create an LVM thin-pool (which is actually a special type of logical volume).

Maybe it helps if you take a look at http://pve.proxmox.com/pve-docs/pve-admin-guide.html#chapter_storage

 

[fenichelar]

So here is a graphical representation of what I am trying to do:

                      SSD
----------------------------------------------
|  -----------    -----------    -----------   |
| |           |  |           |  |           |  |
| |   Plex    |  |  Ripping  |  |   Plex    |  |
| |  Server   |  |  Software |  |  Library  |  |
| | Container |  | Container |  |   Files   |  |
| |           |  |           |  |           |  |
|  -----------    -----------    -----------   |
----------------------------------------------

              HDD
--------------------------------
|  -----------    -----------   |
| |           |  |           |  |
| |  Ripping  |  |   Other   |  |
| | Temporary |  | Temporary |  |
| |  Storage  |  |  Storage  |  |
| |           |  |           |  |
|  -----------    -----------   |
--------------------------------

I would like both containers and the host to be able to access "Plex Library Files", "Ripping Temporary Storage", and "Other Temporary Storage". I want to do daily backups of the containers but I don't want "Plex Library Files", "Ripping Temporary Storage", and "Other Temporary Storage" to be part of those backups. What is the best way to setup the box to do this? I am reading all the documentation about storage but still very confused. Especially confused about how to setup "Plex Library Files". Right now I have nothing setup at all, waiting for the SSD to arrive. I am doing all my testing on a VM. Thanks for the help! I know I probably seem like an idiot. I am very new to virtualization and my only filesystem/volume experience is formatting a whole drive to ext4.

 

[fabian]

if you want mount points that are shared between containers, then you need to use bind mount points (they basically allow you to mount arbitrary directories from the host into the container). in that case, you actually need to create those directories on the host (I would suggest making them actual LVs and mounting them, like you originally proposed). bind mount points are automatically excluded from backups, so you don't need to set anything special there.

so you can simply create the two containers (which will create the two volumes you labeled "Plex Server Container" and "Ripping Software Container" for their respective rootfs), create the three other volumes manually and setup mount points for them, then configure the containers to use those as additional mountpoints. the last step is not possible in the GUI, but rather easy as well, and documented in http://pve.proxmox.com/pve-docs/pve-admin-guide.html#pct_mount_points

 

[fenichelar]

Okay, just to make sure I understand. I will create both containers using "local-lvm" as the root drive. I will then create the additional volumes like so:

For the Plex library files:

$ lvcreate -n plex -V 100G pve/data
$ mkfs.ext4 /dev/pve/plex
$ mkdir /mnt/plex
$ echo '/dev/pve/plex /mnt/plex ext4 defaults 0 2' >> /etc/fstab
$ mount -a
$ pct set 100 -mp0 /mnt/plex,mp=/mnt/plex
$ pct set 101 -mp0 /mnt/plex,mp=/mnt/plex

For the 2 temporary storages:

$ sgdisk -N 1 /dev/sdb
$ pvcreate --metadatasize 250k -y -ff /dev/sdb1
$ vgcreate hdd1 /dev/sdb1
$ lvcreate -L 1000G -n rip hdd1
$ lvcreate -L 1000G -n other hdd1
$ mkfs.ext4 /dev/hdd1/rip
$ mkfs.ext4 /dev/hdd1/other
$ mkdir /mnt/rip
$ mkdir /mnt/other
$ echo '/dev/hdd1/rip /mnt/rip ext4 defaults 0 2' >> /etc/fstab
$ echo '/dev/hdd1/other /mnt/other ext4 defaults 0 2' >> /etc/fstab
$ mount -a
$ pct set 100 -mp1 /mnt/rip,mp=/mnt/rip
$ pct set 101 -mp1 /mnt/rip,mp=/mnt/rip
$ pct set 100 -mp2 /mnt/other,mp=/mnt/other
$ pct set 101 -mp2 /mnt/other,mp=/mnt/other

For the Plex media files:

$ mkdir /mnt/media
$ echo '172.16.0.102:/volume1/Media /mnt/media nfs defaults,noatime 0 0' >> /etc/fstab
$ mount -a
$ pct set 100 -mp3 /mnt/media,mp=/mnt/media
$ pct set 101 -mp3 /mnt/media,mp=/mnt/media

Does that seem right? Thanks for all the help!

 

[fabian]

almost:

• you overwrite the mount points you set in the first code block when you set mp0 again in the second. if you want to keep it, you need to start with mp1 in the second block and continue with mp2 (you can set from mp0 to mp9 in addition to the rootfs)
• "lvcreate -L 1000G -T -n rip hdd1" means "create a thin pool called rip of size 1000G in the volume group hdd1", you probably want a regular volume instead (just drop the "-T" to "create a logical volume rip of size 1000G in the VG hdd1"), otherwise you need to pass "-V SIZE" to tell LVM to "create a thin pool of size 1000G in the VG hdd1 and create a thin logical volume called rip of size SIZE in the pool" (same goes for "other")

using a thin pool does not make much sense here - as far as I understood, you want to allocate the whole disk anyway?

 

[fenichelar]

almost:

• you overwrite the mount points you set in the first code block when you set mp0 again in the second. if you want to keep it, you need to start with mp1 in the second block and continue with mp2 (you can set from mp0 to mp9 in addition to the rootfs)
• "lvcreate -L 1000G -T -n rip hdd1" means "create a thin pool called rip of size 1000G in the volume group hdd1", you probably want a regular volume instead (just drop the "-T" to "create a logical volume rip of size 1000G in the VG hdd1"), otherwise you need to pass "-V SIZE" to tell LVM to "create a thin pool of size 1000G in the VG hdd1 and create a thin logical volume called rip of size SIZE in the pool" (same goes for "other")

using a thin pool does not make much sense here - as far as I understood, you want to allocate the whole disk anyway?

Ahh, yes, I forgot to increment the mount point number. Thanks.

I think regular volumes makes more sense here.

I have updated my code. I also added an NFS mount. I have two NFS shares I want to mount. One is for backups, the other contains the actual media files used by Plex. I think the best thing to do is add the NFS share for backups through the GUI. But then add the NFS share for the Plex media files manually and then use mount points as stated above.I have a direct connection between my Proxmox box and NFS box with two gigabit links in balance-rr. It would be much easier to mount the NFS shares in the host because then I don't have to deal with the direct link in the containers. Does that seem like the best thing to do?

Thanks for all the help!

 

[fabian]

yes, you should mount NFS shares on the host and then bind-mount them into the container (you cannot directly mount them in the container anyway without lifting security restrictions)