LVM, LVM-Thin, LUKS, file systems, PVE and PBS, snapshots - suggestions solicited

[gctwnl]

I have experimented with Proxmox and Linux (new for me, though I have a unix-like background experience) and set up the following setup:

• Hardware: standard x86 Core i5
  • External 2TB SSD RAID (pvehost:/dev/sda)
  • Internal 1TB SSD (pvehost:/dev/nvme0n1 hardware encrypted, key to be entered at boot)
    • PVE installed, so divided into local and local-lvm
      • pvehost:/dev/sda1 (the external RAID) setup with LUKS
        • auto-unlocked at boot
        • Added one big PV on pvehost:/dev/sda1-LUKS, so the PV setup looks like this:

            PV                                                    VG          Fmt  Attr PSize    PFree
            /dev/mapper/luks-fa1483bd-f599-4dcf-9732-c09069472150 rna-mepdm-1 lvm2 a--    <1.75t 788.36g
            /dev/nvme0n1p3                                        pve         lvm2 a--  <931.01g  15.99g

        • Two LVs on the LUKS PV, so the total LV setup looks like this:

            LV              VG          Attr       LSize    Pool Origin Data%  Meta%  Move Log Cpy%Sync Convert
            data            pve         twi-aotz-- <794.79g             1.26   0.28                        
            root            pve         -wi-ao----   96.00g                                                
            swap            pve         -wi-ao----    8.00g                                                
            vm-100-disk-0   pve         Vwi-aotz--   32.00g data        31.29                              
            rna-pbs-mepdm-1 rna-mepdm-1 -wi-ao----  500.00g                                                
          vm-100-disk-0 rna-mepdm-1 -wi-ao---- 500.00g

          • rna-mepdm-1/rna-pbs-mepdm-1 contains an ext4 filesystem and is mounted on pvehost:/mnt/pbs-backup-1 and used as a datastore for PBS
          • rna-mepdm-1/vm-100-disk-0 contains an ext4 filesystem and is mounted on a client
      • Client pveclient: Ubuntu 22.04 Server VM resides on local-lvm
        • LV rna-mepdm-1/vm-100-disk-0 (on external RAID) is mounted on pveclient:/mnt/ServerData
    • PBS uses pvehost:/mnt/pbs-backup-1 as data store. This Datastore is used as pbs storage on PVE. As a result, PVE can backup VMs that live on the internal SSD volume local-lvm to the external RAID volume mounted on pvehost:/mnt/pbs-backup-1. Important: I must exclude the disk pveclient:/dev/sdb1 from that backup, or it goes to on to backup an additional 500GB (and not just the VM's / disk itself)

One reason for this setup is that I want all data to be encrypted when the system is turned off (hence LUKS automounting of the external RAID) against physical data theft. I also want internal backups between the internal NVMe and external SSD RAID for continuity (off site backups to be added later). Another is that I want the server data to live on the RAID and not on the internal single SSD.

This works.

But I found out that I have three(?) issues:

1. Because I have added rna-mepdm-1/vm-100-disk-0 as hardware to VM pveclient I cannot take snapshots of that VM. Other than with backups, the snapshot feature requires the entire VM including all attached hard disks to be 'snapshottable' and /mnt/ServerData is LVM and not LVM-Thin. Backups of just the VM itself (and not the attached 'disk') works fine. I was wondering why this is so. Is 'exclude hardware on snapshot' just a feature that could be there but isn't? Or is there a fundamental reason this would not work?
2. I cannot my pveclient:/mnt/ServerData on a LVM-Thin volume because I cannot put a file system on an LVM-Thin volume. Is this true?
3. And if the previous is true, I can also not use local-lvm as target for backups from the external RAID, true?

I am uncertain here because I am a beginner. But I was wondering if there is a way I can enable snapshots for my VM pveclient and use the internal NVMe SSD as backup storage for the data living on the external RAID SSD.

[UPDATE: I have been notified that I have another issue as well: if I restore my VM backup, the disk I have excluded from the backup will be wiped, not ignored...]

[It is actually fun to learn all this, keep my tech side alive, but it is a steep learning curve]

 

[Dunuin]

1. I cannot my pveclient:/mnt/ServerData on a LVM-Thin volume because I cannot put a file system on an LVM-Thin volume. Is this true?
2. And if the previous is true, I can also not use local-lvm as target for backups from the external RAID, true?

Yes, a "lvmthin" storage in PVE can only store virtual disks, as it is just a block storage that stores LVs. What you can do is manually createing a LV on that thin pool, format it with something like ext4, mount that ext4 on your PVE host using fstab and then create a new "Directory" storage pointing to that mountpoint. You can then use that directory storage to store your backups and the data will end up on that LVM-thin pool.

 

[gctwnl]

I have been notified that I have another issue as well: if I restore my VM backup, the disk I have excluded from the backup will be wiped, not ignored...

 

[Dunuin]

I have been notified that I have another issue as well: if I restore my VM backup, the disk I have excluded from the backup will be wiped, not ignored...

Jup, see my feature request thread: https://forum.proxmox.com/threads/feature-request-advanced-restore-options-in-gui.109707/

There are also some workarounds like restoring the guest with another VMID, then removing the disk you want to overwrite from the existing guest and then move the disks from the restored guest to the existing guest to "merge" them.