Setup space for backups

[miha.adzic]

Hi!

Great work on 4.3!

Question is how do I setup more space for dump?

df -h show only 95G on /dev/dm-0 that is local (pve)
The rest of the disks space is not showing at all and I have 6TB in RAID10
local-lvm (pve) has more than enough space but I cannot use it for VZDump

Is there a tutorial?

Thanx!

 

[fkh]

So you are talking about local storage correct? Cannot you do something like this?

 

[miha.adzic]

Hi!

Yes thats it. But if I make a directory on the root like in your case /backup - there is only 95GB of space there.
The server was setup fresh using the default setting and I have 6TB of disk space. So most of the disk space is set for LVM but I cannot add VZDump in that space.

The question is how do I assign more space for a backup directory. To be more precise where on the disk should I create a backup directory - not on root. How?

Thanx!

 

[Alexander Pytlev]

Hi!
The question is how do I assign more space for a backup directory. To be more precise where on the disk should I create a backup directory - not on root. How?

Create logical disk in LVM group and mount it as /backup

 

[miha.adzic]

Create logical disk in LVM group and mount it as /backup

Hi! Happy New Year!

Sory for the late reply. Can anyone please help?

lvcreate --name backup --size 2T pve
Volume group "pve" has insufficient free space (4051 extents): 524288 required

The local-lvm (pve) shows 2.70TiB of Available space and local (pve) shows 87.60GiB if Available space. So I figure that the above command is trying to add a logical volume to the local (pve). I guess the question is how to add a logical volume to local-lvm (pve)?

Thank you!

 

[miha.adzic]

Hi!

Just figured it out

-v does the trick.

 

[Alexander Pytlev]

pvdisplay

lvdisplay -m

And look, where free space stolen.

 

[Riesling.Dry]

Hi!

Just figured it out

-v does the trick.

Dear Miha,
could you kindly elaborate on that?
What exactly did you type, what is the exact command?
I tried several variations but still get the same insufficient free space error.
Just trying to get my vzdumps to the 5TB 'empty space' on me lil server
Cheers,
R.

 

[Riesling.Dry]

pvdisplay
lvdisplay -m
And look, where free space stolen.

Apparently proxmox resp. data stole the space - how do I get it back to use it for backups/dumps?

 --- Logical volume ---
  LV Name                data
  VG Name                pve
  LV UUID                qOKuXk-Z0K7-BziW-1Uc7-qltl-Kbme-wTmcnZ
  LV Write Access        read/write
  LV Creation host, time proxmox, 2018-02-21 14:54:16 +0100
  LV Pool metadata       data_tmeta
  LV Pool data           data_tdata
  LV Status              available
  # open                 13
  LV Size                4.70 TiB
  Allocated pool data    10.07%
  Allocated metadata     5.06%
  Current LE             446140
  Segments               1
  Allocation             inherit
  Read ahead sectors     auto
  - currently set to     256
  Block device           253:4
   
  --- Segments ---
  Logical extents 0 to 446139:
    Type        thin-pool
    Monitoring        monitored
    Chunk size        1.00 MiB
    Discards        passdown
    Thin count        14
    Transaction ID    18
    Zero new blocks    yes

 

[totalimpact]

lvcreate -V 1000G -T pve/data -n backups

But.... doing -V means he didnt count the available size properly, and is over provisioning, which is fine, if you actually put that amount of data on there you will have a bomb unless you add a disk to the group, it is suggested to count your GBs/extents, and underprovision, you can always extend later when you add new disks...

lvdisplay pve/data

on top of all this, it appears he is placing backups on the pve data volgroup, which is fine if you know what that means.... that means unless he has another storage location where VMs are running from, he is backing up to the same location as where they are running, in the case that the volgroups crashes, you have lost everything.

Ideally you setup a separate location for your backups, format it something, ie ext4, mount it on a directory, ie /mnt/backups, and add that storage location in the gui.

If you understand the consequences of this, and wanted put something on the data volgroup, check the size available in the pool using lvdisplay.

subtract and VMs you have running on it, or over provision (thats the point of Thin - right) and keep a close eye on your available space, having a VM with 200gb used, and storing 5 backups per week is recipe for disaster, the backup will fail smartly when space is low, but it could potentially get close to the limit, and if you consume additional space in your vm - kaboom.

 

[Riesling.Dry]

lvcreate -V 1000G -T pve/data -n backups

I love you totalimpact!
That was exactly what I was looking for: -V (capital V instead of -v, as proposed by dear Miha ;°)

For noobs like me who come across this post:
When the LV is finally created, it must of course be formatted and mounted to show up in GUI:

mkfs.ext4 /dev/pve/backups
mount /dev/pve/backups /mnt

Yes, you are definitely right, totalimpact: when it blows, also the backups are gone. That's why I rsync the backups frequently to a remote server (which unfortunately has a dynamic IP), but I will consider you train of thought to somehow mount the remote server to the production server and backup directly remotely.

Many thanks,
~R.

 

[Dwain]

I love you totalimpact!
That was exactly what I was looking for: -V (capital V instead of -v, as proposed by dear Miha ;°)

For noobs like me who come across this post:
When the LV is finally created, it must of course be formatted and mounted to show up in GUI:

mkfs.ext4 /dev/pve/backups
mount /dev/pve/backups /mnt

Yes, you are definitely right, totalimpact: when it blows, also the backups are gone. That's why I rsync the backups frequently to a remote server (which unfortunately has a dynamic IP), but I will consider you train of thought to somehow mount the remote server to the production server and backup directly remotely.

Many thanks,
~R.

Hi,

This was really helpful. I added a Marvell 88SE9230 PCIe SATA 6Gb/s Controller to my server to extend my current storage. I have my main proxmox install on a 500GB SSD which after a few VMs, CTs and backups ran low on space. So I added that controller so I can have a dedicated storage attached for backups only, but couldn't get it assigned to proxmox GUI to allow VZDump backup file type.
These are the steps I executed to fix this issue:
1. Find the drive that needs to be formatted to type ext4

pvdisplay

2. Format the drive:

mkfs.ext4 /dev/<drive>

3. Make a directory under /mnt

mkdir backup

4. Mount the drive and test availability

mount /dev/<drive> /mnt/backup

5. Add the drive to proxmox GUI as a Directory




6. Then add an auto mount when the system starts into /etc/fstab

/dev/<drive> /mnt/backup ext4 nofail 0 0

 

[totalimpact]

Actually, things have come a long way thanks to PVE dev team, all of that can now be done in the gui in just 1 step.

That is it, you are done....

If there is no available disk in the drop down you may need to wipe it, verify the serial number before plugging in the disk, verify its letter and SN on the DISK page, then clear its MBR-

Clear the MBR of sdm:

dd if=/dev/zero of/dev/sdm bs=512 count=1

 

[Dwain]

Actually, things have come a long way thanks to PVE dev team, all of that can now be done in the gui in just 1 step.

That is it, you are done....

If there is no available disk in the drop down you may need to wipe it, verify the serial number before plugging in the disk, verify its letter and SN on the DISK page, then clear its MBR-

Clear the MBR of sdm:

dd if=/dev/zero of/dev/sdm bs=512 count=1

Yes I saw that option but like you mentioned no disk were available in the drop down. So I decided to do to it the way I used above, but didn't think to only wipe the disk and try again.

 

[silentreefer]

So I hate to beat a dead horse here, but this is the closest I have come to finding the answer I am looking for...

I have Prox up and running on my system, and have for a VERY long time now, but I am to the point where I want to start making some VM changes and expansions. I would like to back up the VM's first beforehand just in case but the way I want to do it is as follows:

Currently -
D1 - Local - 256 SSD as the Prox OS
D2 - Raid 6 - 25.47 TB Raid Array as my Primary Storage - Media
D3 - Raid 5 - 3.46TB Raid Array as my Secondary Storage - Only half of this drive is being used for Torrents to spin here and if it blows up, it's Raided





Now all I want to do is take D3 and use the other half of the drive to dump Backup's on for the temp...

I can NOT format the drive without some serious work ahead of me... So how do I add VZDump as an option to this drive without blowing the whole drive away?

I THINK I need to create something and then mount it to the PVE system and then map that to somehow enable this???

 

[Dominic]

Can you do something like the following?

lvcreate --size 5G Prox-Backup -n test
mkfs.ext4 /dev/mapper/pve-test
mkdir /backup
mount /dev/mapper/pve-test /backup
pvesm add dir backup --path /backup --content backup