[SOLVED] how to adding a second drive for backups, VM's and CT's ?

[Spirog]

hello I am using latest PVE,

proxmox-ve: 7.1-1 (running kernel: 5.13.19-6-pve)
pve-manager: 7.1-11 (running version: 7.1-11/8d529482)
pve-kernel-helper: 7.1-13
pve-kernel-5.13: 7.1-9
pve-kernel-5.4: 6.4-11
pve-kernel-5.13.19-6-pve: 5.13.19-14
pve-kernel-5.13.19-5-pve: 5.13.19-13
pve-kernel-5.13.19-2-pve: 5.13.19-4
pve-kernel-5.4.157-1-pve: 5.4.157-1
pve-kernel-5.4.73-1-pve: 5.4.73-1
ceph-fuse: 15.2.16-pve1
corosync: 3.1.5-pve2
criu: 3.15-1+pve-1
glusterfs-client: 9.2-1
ifupdown: 0.8.36+pve1
ksm-control-daemon: 1.4-1
libjs-extjs: 7.0.0-1
libknet1: 1.22-pve2
libproxmox-acme-perl: 1.4.1
libproxmox-backup-qemu0: 1.2.0-1
libpve-access-control: 7.1-6
libpve-apiclient-perl: 3.2-1
libpve-common-perl: 7.1-5
libpve-guest-common-perl: 4.1-1
libpve-http-server-perl: 4.1-1
libpve-storage-perl: 7.1-1
libqb0: 1.0.5-1
libspice-server1: 0.14.3-2.1
lvm2: 2.03.11-2.1
lxc-pve: 4.0.11-1
lxcfs: 4.0.11-pve1
novnc-pve: 1.3.0-2
proxmox-backup-client: 2.1.5-1
proxmox-backup-file-restore: 2.1.5-1
proxmox-mini-journalreader: 1.3-1
proxmox-widget-toolkit: 3.4-7
pve-cluster: 7.1-3
pve-container: 4.1-4
pve-docs: 7.1-2
pve-edk2-firmware: 3.20210831-2
pve-firewall: 4.2-5
pve-firmware: 3.3-6
pve-ha-manager: 3.3-3
pve-i18n: 2.6-2
pve-qemu-kvm: 6.1.1-2
pve-xtermjs: 4.16.0-1
qemu-server: 7.1-4
smartmontools: 7.2-pve2
spiceterm: 3.2-2
swtpm: 0.7.1~bpo11+1
vncterm: 1.7-1
zfsutils-linux: 2.1.2-pve1

I added a second drive [sda]

root@proxmox:~# lsblk
NAME                                                MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
sda                                                   8:0    0 931.5G  0 disk
sdb                                                   8:16   0   931G  0 disk
├─sdb1                                                8:17   0  1007K  0 part
├─sdb2                                                8:18   0   512M  0 part
└─sdb3                                                8:19   0 930.5G  0 part
  ├─pve-swap                                        253:0    0     8G  0 lvm  [SWAP]
  ├─pve-root                                        253:1    0    96G  0 lvm  /
  ├─pve-data_tmeta                                  253:2    0   8.1G  0 lvm
  │ └─pve-data-tpool                                253:6    0 794.3G  0 lvm
  │   ├─pve-data                                    253:7    0 794.3G  1 lvm
  │   ├─pve-vm--101--disk--0                        253:8    0   500G  0 lvm
  │   ├─pve-vm--104--disk--0                        253:9    0   500G  0 lvm
  │   ├─pve-vm--100--disk--0                        253:11   0    96G  0 lvm
  │   ├─pve-vm--102--disk--0                        253:12   0    96G  0 lvm
  │   └─pve-vm--104--state--look_Snap_Shot_3_27_2002
  │                                                 253:13   0 180.5G  0 lvm
  └─pve-data_tdata                                  253:3    0 794.3G  0 lvm
    └─pve-data-tpool                                253:6    0 794.3G  0 lvm
      ├─pve-data                                    253:7    0 794.3G  1 lvm
      ├─pve-vm--101--disk--0                        253:8    0   500G  0 lvm
      ├─pve-vm--104--disk--0                        253:9    0   500G  0 lvm
      ├─pve-vm--100--disk--0                        253:11   0    96G  0 lvm
      ├─pve-vm--102--disk--0                        253:12   0    96G  0 lvm
      └─pve-vm--104--state--look_Snap_Shot_3_27_2002
                                                    253:13   0 180.5G  0 lvm

I wanted to create it as LVM-Thin, but I can not select backups on this drive from [sdb] VM's ?

how can I make this drive capable of backups and VM and CT so its using LVM-Thin.

do I need to do 2 partitions - it's 1 TB drive


- my 1st drive [sdb] has my VM's running on it.

I would like to get backups onto this drive [sda], but also be able to create VM and CT on this drive as well for testing only.

- I tried from GUI different ways and can't seem to get it ?

( Can anyone outline the steps for me (Noob) sorry and thanks in advance for your knowledge and assistance )

Thanks
Spiro

 

[Spirog]

right now I have local this is for backups images etc
then I have local-lvm thinpool so my vm and ct goes here

so anyway to create this below config.

Storage> local2 use this for backups 700gb allocated space (but take backups from both [sda] and [sdb]

storage > local-lvm2 ThinPool 300gb allocated space for Vm and CT setups ?

thanks for guidance and step by step

Kind Regards,
Spiro

 

[Dunuin]

I don't think you can do that using the GUI. Many storages like LVM-Thin and ZFS won't allow you to store backups there, because they are blocklevel storages and Vzdump backups are files, so you need a filelevel storage for that.
You could manually create a new LV ontop of your LVM-Thin. Then format that LV, mount it locally on your PVE host and then create a "Directory" storage in PVE pointing to the mountpoint of that LV. This "Directory" storage then is a filelevel storage and you can use it to store backups.
But that all means you need to learn how LVM and LVM-Thin is working and how to use all the commands to administrate the LVM using the CLI.

Another option could be to just create a single partition, format it with something like ext4, mount it, add that mountpoint as a "Directory" storage and then use that directory storage for VMs/LXCs too. Then you got your guests as qcow2 image files instead of block devices on a LVM-Thin. And that should be possible to setup by GUI. Go to "YourNode -> Disks -> Directory -> Create: Directory" in WebUI and PVE should do all the above for you. Yo ucan then edit that Directory Storage (Datacenter -> Storage -> YourDirectoryStorage -> Edit) and add other content types.

 

[Spirog]

Another option could be to just create a single partition, format it with something like ext4, mount it, add that mountpoint as a "Directory" storage and then use that directory storage for VMs/LXCs too. Then you got your guests as qcow2 image files instead of block devices on a LVM-Thin. And that should be possible to setup by GUI. Go to "YourNode -> Disks -> Directory -> Create: Directory" in WebUI and PVE should do all the above for you. Yo ucan then edit that Directory Storage (Datacenter -> Storage -> YourDirectoryStorage -> Edit) and add other content types.

Hello and thanks for your help @Dunuin

my question here would be if backups are happening on a single partition vs creating 2 partitions would this slow down the vm's or CT's or its the same drive so there is no difference ?
I was thinking to create 2 partitions and then resize to what I need on each and then do what you mentioned
Go to "YourNode -> Disks -> Directory -> Create: Directory" in WebUI
then on the second one
Go to "YourNode -> Disks -> LVM-Thin -> Create: thinpool" in WebUI

do you think there is a difference from just adding the one partition and creating the Directory vs 2 partitions?

thanks so much
Spiro

 

[Dunuin]

Hello and thanks for your help @Dunuin

my question here would be if backups are happening on a single partition vs creating 2 partitions would this slow down the vm's or CT's or its the same drive so there is no difference ?

You could use the same partition for both. Should be no performance difference if you use one or two partitions because its the same drive. What might differ between qcow2 and LVM-Thin is the overhead.

I was thinking to create 2 partitions and then resize to what I need on each and then do what you mentioned
Go to "YourNode -> Disks -> Directory -> Create: Directory" in WebUI
then on the second one
Go to "YourNode -> Disks -> LVM-Thin -> Create: thinpool" in WebUI

Not sure if the PVE WebUI can work with partitions or just complete disks. Got no spare disk in the servers to test that.

do you think there is a difference from just adding the one partition and creating the Directory vs 2 partitions?

thanks so much
Spiro

Using two partitions you waste more space. Lets say you always want 100GB free on each partition to be on the save side that they don't run out of space. When you just got one big 1TB partition you can use 900GB and have 100GB free. When using one 300GB + 700GB partition you can only use 200Gb + 600GB, so additional 100GB wasted. Or you later realize that 700GB is too small for your longterm backups but at the same time you maybe just need 100GB for VMs. Then you wasted 200GB that can't be used by your backups. So I personally prefer a storage layout that is more dynamic and won't limit me years later, because I needed to allocate fixed sizes back then.

 

[Spirog]

You could use the same partition for both. Should be no performance difference if you use one or two partitions because its the same drive. What might differ between qcow2 and LVM-Thin is the overhead.

Not sure if the PVE WebUI can work with partitions or just complete disks. Got no spare disk in the servers to test that.

Using two partitions you waste more space. Lets say you always want 100GB free on each partition to be on the save side that they don't run out of space. When you just got one big 1TB partition you can use 900GB and have 100GB free. When using one 300GB + 700GB partition you can only use 200Gb + 600GB, so additional 100GB wasted. Or you later realize that 700GB is too small for your longterm backups but at the same time you maybe just need 100GB for VMs. Then you wasted 200GB that can't be used by your backups. So I personally prefer a storage layout that is more dynamic and won't limit me years later, because I needed to allocate fixed sizes back then.

ok perfect sense so I will just use 1 partition and do that.

Another option could be to just create a single partition, format it with something like ext4, mount it, add that mountpoint as a "Directory" storage and then use that directory storage for VMs/LXCs too. Then you got your guests as qcow2 image files instead of block devices on a LVM-Thin

so by doing this Directory my VM's and LXC's will use qcow2 and they will be using LVM-Thin < Is that correct. all my VM's icreate will use LVM-thin

that seems perfect if that works that way and so much more easier and Logical..

Kind Regards,
Spiro

 

[Dunuin]

so by doing this Directory my VM's and LXC's will use qcow2 and they will be using LVM-Thin < Is that correct. all my VM's icreate will use LVM-thin

that seems perfect if that works that way and so much more easier and Logical..

Kind Regards,
Spiro

No, you can either use LVM-Thin or a Directroy with qcow2. LVM-Thin is a blocklevel storage. Directory with qcow2 is a filelevel storage. If you want to store backups you need a filelevel storage. Only way to have a directory storage for backups + a LVM-Thin pool for VMs that share the complete 1TB is to do it yourself in CLI like describes:

You could manually create a new LV ontop of your LVM-Thin. Then format that LV, mount it locally on your PVE host and then create a "Directory" storage in PVE pointing to the mountpoint of that LV. This "Directory" storage then is a filelevel storage and you can use it to store backups.
But that all means you need to learn how LVM and LVM-Thin is working and how to use all the commands to administrate the LVM using the CLI.

Or your try it with two partitions using the GUI, but then you are limited to these partition sizes and they can't dynamically share the full 1TB.

 

[Spirog]

No, you can either use LVM-Thin or a Directroy with qcow2. LVM-Thin is a block storage. Directory with qcow2 is a file storage. If you want to store backups you need a file storage.

But I can use the directory for both backup storage and vm - lxc’s. I guess when I need more space for vm lxc’s I can add another hard drive and use that with lvm-thin.

Zfs is this a better option when having 2 hard drives. Better than LVM - Thin
Faster or not much of a difference ?
And also security I saw another post about encryption on the disks. Is this possible to encrypt directory backups ? And lvm- thin ? Or just zfs ?

Thanks again for all you powerful knowledge. I appreciate you !!

Kind Regards
Spiro

 

[Dunuin]

But I can use the directory for both backup storage and vm - lxc’s. I guess when I need more space for vm lxc’s I can add another hard drive and use that with lvm-thin.

Zfs is this a better option when having 2 hard drives. Better than LVM - Thin
Faster or not much of a difference ?

The overhead with ZFS is way higher because it is more complex and offers more features. I guess your SSD will die 2-3 times sooner with ZFS. And the GUI won't allow you to put backups on a ZFS storage. But you can manually create a "Directory" storage ontop of ZFS via CLI so both (guests and backups) can use thin-provisioning and share the full space. But a ZFS pool should always got 20% of free space, so you just would get 800GB of usable storage of your 1TB SSD. And ZFS would use more RAM.

And also security I saw another post about encryption on the disks. Is this possible to encrypt directory backups ? And lvm- thin ? Or just zfs ?

Got both encrypted here. ZFS got native encrytion out of the box but not using the GUI. LVM-Thin got no encryption, so you would need to create your LVM-Thin ontop of a LUKS encrypted partition which is again not possible using the GUI.

Like with most advanced things and proxmox...you can do basically everything, but you need to set it up yourself without a GUI.