Backup methods for Servers and Users

[Eingedi]

Hello,


I am an IT Manager at a small company. As soon as I got there I've noticed that none of the data is being backed-up, I would like to ask for help in this issue.
1. we got a Proxmox VE Server, which runs: "Git", "Trac", "CRM" and "scipy" (May add Mail Server such as "Citadel" in the near future).
2. We got a Shared Storage (NAS), which is located in a different area on a different PC(Windows) - Would like to move it ALL to the PVE Server.
The result will be that all of the data of the company will be located on a single Server (PVE) - And start the Backup from there.


Now, I would like to ask in which methods of backup should I use? Obviously each OS on Proxmox uses a different backup method, and I've no idea where to start from. Or even what kind of data is important for backup, and which is not.


My idea was to backup all of the data from each OS on PVE --> To a Data Server(Backup Server? Called SQL?) which I have access to, and from that Server, use a Remote Backup method, such as "SpiderOAK" or "Crashplan".


That's much or less it. I appreciate any kind of help and information you can give me.
Thanks,
Barak.

 

[wolfgang]

Hi,
why you don't use the build in backup tool and save the Vm somewhere you like?
and if you need a second set of backup copys copy them.

https://pve.proxmox.com/wiki/Backup_and_Restore

 

[Eingedi]

Hi,
why you don't use the build in backup tool and save the Vm somewhere you like?
and if you need a second set of backup copys copy them.

https://pve.proxmox.com/wiki/Backup_and_Restore

Thank for the answer.

Yea, I figured that's what I'll do. But it's not effective to backup the whole stuff every day, I need an option to do an Incremental backup, which I can't seem to find in Proxmox.
Plus, when I go for the "Backup" tab in Proxmox, there is no storage to select, therefore no backup. How can I add there a storage? I need a network storage but it doesn't seem there is one.

 

[wolfgang]

You must activate this in Datacenter -> Storage -> select your storage where you backup -> Content -> VZDUmp backupfile.

What do you mean with network storage?
If SMB you must mount it with the fstab and add it as local dir.

 

[Eingedi]

You must activate this in Datacenter -> Storage -> select your storage where you backup -> Content -> VZDUmp backupfile.

What do you mean with network storage?
If SMB you must mount it with the fstab and add it as local dir.

Umm, this may be a really silly question, but when I go for Storage and try to create "NFS Share" (Since I want to save the backups on a different computer, I figured this was the solution), I really have no idea what to put in the credentials. The problem is the "Export" tab, it is empty. What do I put there?
And that mounting through fstab, yea I know how to get there, but how do I do this? (You can see I'm pretty new at this)

 

[m.ardito]

You can see I'm pretty new at this)

you should probably familiarize yourself with this
http://pve.proxmox.com/wiki/Storage_Model

Marco

 

[Eingedi]

you should probably familiarize yourself with this
http://pve.proxmox.com/wiki/Storage_Model

Marco

Well, I did. It doesn't say what to do when there is no "Export", no matter how much I've searched I couldn't find an answer.
Just let me understand something - If I want to backup proxmox to an other server on the network, I should use NFS Share, right? So what does the ZFS used for?
Problem is the my Proxmox isn't updated, and I can't update it because I don't have a subscription, and I can't even Upgrade through the console, cause I have a plugin problem which I found out cannot be fixed.
What am I supposed to do?

 

[Nemesiz]

Backup all VM sometimes is not the best choice. I recommend you bareos https://www.bareos.org/en/

 

[wolfgang]

you can update without a subscription.
See her
https://pve.proxmox.com/wiki/Package_repositories

ZFS is a filesystem were you can store your VMs.
You can generally backup your Vm everywhere you want.
NFS is only one possibility.

 

[RobFantini]

We back up to a pve server that runs nfs . That system is powerful enough to run most of our vm's if necessary. We've 4 pve systems, Each is backed up a different day at 22:00. We've have issues with cluster when multiple systems are backed up at the same time. For data backup that of course depends on the system. For our main data we backup 2x per hour off and on site. Dovecot imap send to a second dovecot system every 10 minutes. We use Owncloud for all documents. Is easy to set up for real time off site backup to another oc kvm. We use usb 3.0 and backup from kvm's at nfs to usb , and restore that off site at least weekly - to pve systems at two places. We've also 2 desktops with extra storage and use rsync nightly to send kvm files to both. The pcs' are encrypted. The usb's should be but have not got to that yet. If you want more info just reply. For users we use ldap and had a support company set up a multi master system.

 

[longhair]

Well, I did. It doesn't say what to do when there is no "Export", no matter how much I've searched I couldn't find an answer.
Just let me understand something - If I want to backup proxmox to an other server on the network, I should use NFS Share, right? So what does the ZFS used for?
Problem is the my Proxmox isn't updated, and I can't update it because I don't have a subscription, and I can't even Upgrade through the console, cause I have a plugin problem which I found out cannot be fixed.
What am I supposed to do?

I had the same problem as you when I first set up a Synology NAS. If I remember correctly (at home trying to wake up), I had to make a shared NFS folder on the Synology before Proxmox was able to find the "Export" path.

 

[m.ardito]

Well, I did. It doesn't say what to do when there is no "Export", no matter how much I've searched I couldn't find an answer.
Just let me understand something - If I want to backup proxmox to an other server on the network, I should use NFS Share, right? So what does the ZFS used for?
Problem is the my Proxmox isn't updated, and I can't update it because I don't have a subscription, and I can't even Upgrade through the console, cause I have a plugin problem which I found out cannot be fixed.
What am I supposed to do?

see this for reference http://en.wikipedia.org/wiki/Network_File_System#Typical_implementation

a NFS share is a way for a server to allow access to its storage from the network. It is done "exporting" a folder with a name, roughly said.
So the "export" you must provide is the name of the NFS share on the NFS server (specified just above the "export" field, btw)
The NFS server is tipically another machine on the network, such as a NAS.

if you fill the "server" field with (eg) the IP address of your NFS server, you will find in the "export" field a list of all available shares, on that server.

ZFS is another storage option in proxmox VE (it has so many!). If you have a ZFS server in the network, you would prefer this way over NFS.

HTH

Marco

 

[Eingedi]

Ok, let's see what I got so far...

* I can use the rsync + cron in order to backup the data. - Is it sufficient? Is this the only backup I need? Will users still be able to work while it's backing?
* If rsync copies the whole data, what do I use to backup only the Database of every server on my PVE? (I need to find the location of the Database in every server?)
* In order to create an NFS Share, I need to mount it using the fstab, right? Well, how do I do that? (Since when I fill the "server" and go for "export" tab, it can't find anything even when I have a shared folder)

Basically, what I need is a backup of everything once a week, and a daily backup of only the Database.

 

[m.ardito]

sorry, what you mean about "the data" and "the database"?
it's just to make clearer what you wish to achieve.

afaik, on pve:
- all nodes in cluster share the same "config" and "db". So, should a node need to be reinstalled, it will get all its configs/from other nodes in the cluster, once it joins
- anyway, most useful configs and data are under /etc/pve (including .conf for all qemu and kvm vms in all nodes). you could find other stuff under /var/lib/pve-* dirs but i think those are rebuilt from the config if needed.
- under /var/lib/vz/ you can find "local" storage resources

But if you wish, instead, to backup your server completely (eg: to be able to restore every single bit of them, with updates, etc), any regular linux backup system could do, I think.
Just try to keep consistency in data you backup (eg: use snapshots lvm or zfs if possible), if you do this online, otherwise use an offline backup like imaging or else, and it should be fine.

About the NFS share, it should work like that. If it doesn't... it could be everything, from network to permissions on NFS server side... maybe more details would help.

Marco

 

[RobFantini]

Ok, let's see what I got so far...* I can use the rsync + cron in order to backup the data. - Is it sufficient? Is this the only backup I need? Will users still be able to work while it's backing?* If rsync copies the whole data, what do I use to backup only the Database of every server on my PVE? (I need to find the location of the Database in every server?)* In order to create an NFS Share, I need to mount it using the fstab, right? Well, how do I do that? (Since when I fill the "server" and go for "export" tab, it can't find anything even when I have a shared folder)Basically, what I need is a backup of everything once a week, and a daily backup of only the Database.

1-NFS : Do you have a NFS server set up yet? As others have written using an existing nfs server from pve is very easy.2- database : which operating system is used? and database type?

 

[Eingedi]

sorry, what you mean about "the data" and "the database"?
it's just to make clearer what you wish to achieve.

afaik, on pve:
- all nodes in cluster share the same "config" and "db". So, should a node need to be reinstalled, it will get all its configs/from other nodes in the cluster, once it joins
- anyway, most useful configs and data are under /etc/pve (including .conf for all qemu and kvm vms in all nodes). you could find other stuff under /var/lib/pve-* dirs but i think those are rebuilt from the config if needed.
- under /var/lib/vz/ you can find "local" storage resources

But if you wish, instead, to backup your server completely (eg: to be able to restore every single bit of them, with updates, etc), any regular linux backup system could do, I think.
Just try to keep consistency in data you backup (eg: use snapshots lvm or zfs if possible), if you do this online, otherwise use an offline backup like imaging or else, and it should be fine.

About the NFS share, it should work like that. If it doesn't... it could be everything, from network to permissions on NFS server side... maybe more details would help.

Marco

By "Data" I meant the actual every piece of data of the server, much like an image, while with "Database" I meant only the system data, which from this data restoration should be able to rebuild the system.
So what I want to do is make a monthly image backup, and daily "database" backup. both should be Incremental probably (Only copying new data and files).

Basically what you're saying is that I only need to backup "/etc/pve" and the servers can be restored from there? Or am I missing something?
Another thing, Let's say I choose the backup with rsync, can the users still work on the servers with no performance issues?

Tell me what details you need and I will do my best to provide them.

 

[Eingedi]

Edit: Sorry for double post.

 

[Eingedi]

1-NFS : Do you have a NFS server set up yet? As others have written using an existing nfs server from pve is very easy.2- database : which operating system is used? and database type?

- NFS Server? Don't I just need a shared folder on a PC that's in the network and provide the path to it? I don't want the NFS Server to be part of the PVE Server, I want to avoid any more data on it. So everything should be external.
- Operating System on the server I to backup to? It's Ubuntu 14.04 LTS. Database type? I have no idea, how do I check it?

 

[RobFantini]

Ubuntu 14.04 LTS is a good choice to store backups. do you have a folder [ directory ] with enough storage for the backups? what is the folder name? answer those and we can help you set up a nfs server. nfs has been around for a long time and just works easy with pve.

 

[m.ardito]

- NFS Server? Don't I just need a shared folder on a PC that's in the network and provide the path to it? I don't want the NFS Server to be part of the PVE Server, I want to avoid any more data on it. So everything should be external.
- Operating System on the server I to backup to? It's Ubuntu 14.04 LTS. Database type? I have no idea, how do I check it?

As RobFantini says, a "nfs server" means that on your storage machine (tough a pc is usually not much redundated and safe enough, but would work) you need to install a piece of software acting as nfs server (any linux would do, your choice is ok)

it is not sufficient to say "a shared folder" because it could be shared in many ways. To use a nfs "export" prom pve, you need a machine in the lan running the "nfs server" software, and configure it to "export" a folder of choice.

Marco