Backups

[rcd]

Situation: A Proxmox hypervisor with a couple of Cpanel hosts needs to be backed up to off-server storage.

Cpanel has it's own backup system. I realize Proxmox has snapshots etc, but the cpanel backups allow restore of individual files. The cpanel backups are quite large and server disk space is at a prime so the backups need to moved off-server for storage. Also, since the server runs thin-lvm it is probably not a good idea to use the servers own disk space for the temporary backups.

I was thinking I could setup an nfs share on the hypervisor, using the /var/lib/vz space as temporary storage - so have cpanel backup to the nfs storage, copy the backups offsite with scp or whatever, and delete them from the local storage. That way I won't waste a lot of premium disk space for backups.

Does this make sense? I read the admin guide and while it has a lot of reference material, I find it a bit short on guidance. Maybe someone has a better way to do this?

 

[LnxBil]

Does this make sense?

As you laid it out coarsely, yes!

In detail, I have more questions:

- Does cpanel runs inside QEMU VM or LX(C) container?
- If your cpanel only runs inside of your lx(c) container and if you backup there, your NFS-share has to be bind-mounted into your container. That results in dependency problems if your NFS is not working, you will not be able to start your container. This approach does not work in a QEMU-VM, you need to copy of the file via scp or mount the NFS directly.
- What about storing your cpanel backup inside your guest and keep e.g. x days and only backup your guest in PVE also every x days and keep y days on your NFS. Now you can go back x*y days and only store x days of backups on your guest

 

[rcd]

Cpanel runs inside VM's.

I don't have disk space to store any backups online. By sharing the backup space I can backup to the local storage, but then need to move it offsite and free the storage.

Alternatives to NFS? Native way to mount disk space in Proxmox perhaps?

 

[LnxBil]

Alternatives to NFS?

There is also CIFS/SMB

Native way to mount disk space in Proxmox perhaps?

Works too, but is more error prone. Why not stuff in more disk space?

 

[velocity08]

Hi RCD

If you have external storage or can create another pool locally of slower disks that’s your best bet.
Anything else is going to be a headache to manage.

If you can create another local pool with slower drives for backup sweet.
On this pool you can create a file server cantos vm and do NFS with VDO for deduplication or install R1soft to back up every cPanel account on a schedule and keep backups out of cPanel.

If it’s just an nfs share you can mount it in cPanel as a volume and tell cPanel to store backups on the nfs share.

Be aware if the connection drops between the nfs share and cPanel server it will create IO issues in cPanel server,

R1 soft is a good cheap solution, take a look into this as first option.

If you have an external box, nas etc even better as its off server.

Good luck.
“”Cheers
G

 

[rcd]

It's a OVH server, they come with either a small amount of fast SDD storage or a lot of slow HDD storage, you choose. Yes I could spend (a lot) more money and build one with both, but that is not what the question is about, but how to best get around the constraints there are.

So the thing is, if I allocate, say, 200G of scratch/work space on a partition, and mount that wherever I need to do backups, I'll be fine. When the backup is done, I copy the files off to a dedicated fileserver, unmount the drive, clear the content and ready for next backup job.

I don't think SMB is the best option for this environment. I was more thinking about a way of mounting the scratch space through Proxmox. It ought to be possible to do, the question is if it's practical and if it a better solution than just using simple NFS. Yes I know about IO issues, but given the NFS server is on the same hypervisor as the client, I dont' think that ought to be a real issue?

As for R1soft, I had a look at their website but even at the "buy now" page there wasn't any pricing info. Anyway, the cpanel backup system is actually very very good, the only issue I have is with the space, and I don't think even R1soft can solve that.

 

[velocity08]

HI RCD

See below:

It's a OVH server, they come with either a small amount of fast SDD storage or a lot of slow HDD storage, you choose. Yes I could spend (a lot) more money and build one with both, but that is not what the question is about, but how to best get around the constraints there are.

So the thing is, if I allocate, say, 200G of scratch/work space on a partition, and mount that wherever I need to do backups, I'll be fine. When the backup is done, I copy the files off to a dedicated fileserver, unmount the drive, clear the content and ready for next backup job.

Ok it sounds like you know what you would like to do.
This could be achieved with a USB drive and USB pass through to the VM
The only other way to have a scratch disk and attach to a VM is either as a virtual drive or over a file share from within another VM or external file server of some type.

There is no ability to pass through RAW storage to a VM from PVE or any hypervisor for that matter as far as im aware.
It’s a virtual environment designed to create seperate isolated instances, to attach storage into an instance you can use one of the following:

• ISCSI from external device
• NFS/ SMB or other file sharing protocol into the VM from External device
• USB device pass through from PVE > VM
• Local Virtual Disk can be mounted to other VM’s in PVE (but limited as you cant get the data out without any of the above connected into VM)

Please reference the PVE user guide for more information on the above.

I don't think SMB is the best option for this environment. I was more thinking about a way of mounting the scratch space through Proxmox. It ought to be possible to do, the question is if it's practical and if it a better solution than just using simple NFS. Yes I know about IO issues, but given the NFS server is on the same hypervisor as the client, I dont' think that ought to be a real issue?

Practical no, hence why the other options have been proposed but any solution will always be a compromise between cost, resources and time you wish to invest in the ongoing process to maintain the solution.

It will come down to perspective for 1 person saving time is more important than the added cost of putting in more storage, for another person they have more time than money so its more practical to manually perform tasks until such time that they dont have the time any more or wish to automate or provide more consistent stratagy.

As for R1soft, I had a look at their website but even at the "buy now" page there wasn't any pricing info. Anyway, the cpanel backup system is actually very very good, the only issue I have is with the space, and I don't think even R1soft can solve that.

It’s an automated solution that runs inside a VM and connects into cPanel over file share protocol.
Cost is about $10 usd roughly per month for the license, you can either get it directly from R1soft or from a reseller there are many so a little google will reveal the cheapest price.

Best of luck.


“”Cheers
G

 

[rcd]

Thanks for the backup solution suggestion, but I am really not looking for a backup solution - the one in cpanel works just fine - i just need some space for temporary storage before moving it off to a 20 TB vault i have somewhere else.

So if I understand right, it is not considered a practical solution to nfs-export a directory from the hypervisor. So, alternatively can I add some extra disk space to a VM and have that nfs-export it ?

I tried creating a (normal, not-thin) LV on the hypervisor - but I don't find any way to mount that onto the VM. From the VM I can add a second disk, but it only allows me to allocate data from "local-lvm". What is the correct way to do this?

I also tried simply adding a second disk (off the local-lvm storage pool) from the GUI. I can see in the hypervisor but not in the VM. What am I missing?
I'm sorry if these come across as really stupid questions. I have read the documentation and this forum as well as googled high and low, but I don't find the answers I look for. Maybe I ask the wrong questions, or I just expect too much?

 

[velocity08]

Thanks for the backup solution suggestion, but I am really not looking for a backup solution - the one in cpanel works just fine - i just need some space for temporary storage before moving it off to a 20 TB vault i have somewhere else.

So if I understand right, it is not considered a practical solution to nfs-export a directory from the hypervisor. So, alternatively can I add some extra disk space to a VM and have that nfs-export it ?

I tried creating a (normal, not-thin) LV on the hypervisor - but I don't find any way to mount that onto the VM. From the VM I can add a second disk, but it only allows me to allocate data from "local-lvm". What is the correct way to do this?

I also tried simply adding a second disk (off the local-lvm storage pool) from the GUI. I can see in the hypervisor but not in the VM. What am I missing?
I'm sorry if these come across as really stupid questions. I have read the documentation and this forum as well as googled high and low, but I don't find the answers I look for. Maybe I ask the wrong questions, or I just expect too much?

Hi RCD

when adding a new disk to a VM via PVE it won't be seen within the VM OS until you log into the VM OS via SSH or console > perform a Disk partition on the new drive > format the drive with the appropriate file system > mount it so it is available for use.

some basic Linux skills will be required and the commands will differ slightly based on the OS version you are using.

the above overview should get you started.

Google > adding a disk/ drive to " replace this content with the linux OS version you are using"

google > formatting drive on " replace this content with the linux OS version you are using" (i recommend using ext 4 as the file system for too many reasons i won't go into or you can look into others if thats a topic that interests you)

google > mounting a drive for persistent reboots on " replace this content with the linux OS version you are using"

always take a snapshot or backup before starting the process just incase you kill the system by accident and need to roll back to a previous state (actually take both if you are able)

if i remember correctly you can connect an FTP server to the cPanel server and export the backups to another location.

How to generate a cPanel backup and sent to FTP Server?
https://www.webdomain.com/knowledge...e-a-cPanel-backup-and-sent-to-FTP-Server.html

cPanel Automated Backup Script (this can export to FTP server)
http://www.thecpaneladmin.com/cpanel-automated-backup-script/

Creating an FTP server is out side the scope of PVE so something you'll need to explore your self.

What you are actually asking is out side the scope of PVE and more an admin question on how to manage your cPanel backups so in answer to your statement "yes expectations are not realistic" as this is a PVE forum specifically for PVE questions relating to PVE management and virtualisation etc not cPanel backups or how to use an OS.

in any case happy to help guide in the right direction.

good luck..

""Cheers
G

 

[rcd]

Thanks for the cpanel/backup info, but that wasn't actually what I asked for, nor am I asking for general LInux information. I just need specific information on how this is done on Proxmox. I guess it was late and maybe I didn't explain very well. Let me try again.

I want to create a new disk for a VM. Using the GUI I can create a disk from the local-lvm space on the host. That then appears as a new disk device (/dev/sdX) on the VM, and I can fdisk-lvm-mkfs-mount that space on the VM. No problem so far.

Now, for whatever reason I'd like to not use the (thin) local-lvm space. I know how to create a standard LV on the host, but how do I export that so I can use it on the VM? I guess that is essentially the question.

 

[LnxBil]

Now, for whatever reason I'd like to not use the (thin) local-lvm space. I know how to create a standard LV on the host, but how do I export that so I can use it on the VM? I guess that is essentially the question.

You cannot via the GUI. Just create an additional regular LVM storage in the PVE GUI and use it as the backing storage for a new disk.