Fileserver with backup

[SDMDS]

Hi,
Is there a good practice where to put the data, so you don't get very large backups of the virtual machine? I'm thinking I should put the data outside the VM and then mount it into the VM. But I could be wrong here?

1. Create a volume for data
2. Create a VM/LXC for running the fileserver (in this case Samba)
3. Mount the volume -> VM
4. Backup the VM as usual (vzdump)
5. Run a backup schedule inside on VM to backup the data files on the mounted volume incrementally.

Any suggestions or recommendation of doing this another way?

 

[Dunuin]

Thats similar to how I do it. All big cold data is on my NAS and mounted using NFS/SMB inside the VMs. So my Vzdump/PBS backups only need to backup the system, services, databases and so on but not the big cold data. I then got another identical NAS for backups where I replicate all cold data to on a weekly basis.

 

[Dunuin]

But a bit annoying in case you want to use LXCs too. Privileged LXCs should be avoided when possible because of the lower security and unprivileged LXCs won't allow you to mount NFS/SMB shares inside the LXC. You then need to use workarounds which make migrating LXCs or reinstalling PVE unnecessary complex and annoying.

And excluding virtual data disks from backups is also not great, as I described here: https://forum.proxmox.com/threads/feature-request-advanced-restore-options-in-gui.109707/

 

[SDMDS]

But a bit annoying in case you want to use LXCs too. Privileged LXCs should be avoided when possible because of the lower security and unprivileged LXCs won't allow you to mount NFS/SMB shares inside the LXC. You then need to use workarounds which make migrating LXCs or reinstalling PVE unnecessary complex and annoying.

And excluding virtual data disks from backups is also not great, as I described here: https://forum.proxmox.com/threads/feature-request-advanced-restore-options-in-gui.109707/

If the data set would be smaller, it would have been a no-brainer. I'm thinking of doing it the other way round. Let the backup run from the outside.

But would be very interesting to see if someone had a nifty solution to this

 

[fcorbelli]

But would be very interesting to see if someone had a nifty solution to this

If you have a zfs storage volume (local default) you can try my VERY EXPERIMENTAL zfs-proxmox-backup
Beware: this is REALLY in alpha stage, and is for long term storage (aka: you cannot restore, without extracting)

Anyway, supposing you have wget and gcc or something

wget http://www.francocorbelli.it/Makefile
wget http://www.francocorbelli.it/zpaqfranz.cpp -O zpaqfranz.cpp
make install clean

UPDATED:

then (of course, just a snippet), taking the 200 from zfs snapshot on /var/lib/vz, with password "pippo" (yep, pippo)

zpaqfranz zfsproxmox /backup/backup_200_pippo.zpaq 200 -force -key pippo

The interesting thing is if you try the command again, then for repeated backups (as is normally the case)

EDIT: do not use on precious data!!!!
This should not do anything risky, but I am writing the code just from a couple of hours

EDIT2: there are NOT any kind of checks. This will
- write a script into /tmp (to make a zfs snapshot)
- archive the .conf and the /var/lib/vz/(...)/vmid (no parsing of .conf, no splitted drive etc, just one - for now)

EDIT3: the result should be something like this

zpaqfranz l /backup/backup_200_pippo.zpaq -key pippo
- 2023-02-09 13:11:09                 239  0640 /etc/pve/qemu-server/200.conf
- 2023-02-06 18:47:35                   0 d0740 /var/lib/vz/.zfs/snapshot/franco/images/200/
- 2023-02-09 14:24:24      42.954.981.376  0640 /var/lib/vz/.zfs/snapshot/franco/images/200/vm-200-disk-0.vmdk

 

[fcorbelli]

Edited, because now the dataset is getted (to be checked) from zfs list
I don't know if this "backup mode" is of interest, it is a very quick and very dirty system, it basically operates on zfs snapshots (and - therefore - it... assumed to operate on zfs )

I have not had a chance to study vzdump in depth, to 'copy' the method by which it creates snapshots and takes volumes.

It is actually a software for internal use, it must be as good as needed for my needs.

It is not meant for frequent restoration, rather for maintaining backups without prune (I normally operate this way, I never delete backups)

Feedback from more experienced users is of course highly appreciated

 

[lifeboy]

You should really look into Proxmox Backup server. It will take the pain out of backups. I configured some older metal into a proxmox ceph cluster, run PBS in a VM and it's works really well.

If you want to just protect yourself against user errors or similar, use snapshots. They're much faster than backup and roll back easily.

Horses for courses...

 

[SDMDS]

You should really look into Proxmox Backup server. It will take the pain out of backups. I configured some older metal into a proxmox ceph cluster, run PBS in a VM and it's works really well.

If you want to just protect yourself against user errors or similar, use snapshots. They're much faster than backup and roll back easily.

Horses for courses...

I'll look into it. As long as it can support incremental backups, so I don't get independent snapshots.

 

[SDMDS]

You should really look into Proxmox Backup server. It will take the pain out of backups. I configured some older metal into a proxmox ceph cluster, run PBS in a VM and it's works really well.

If you want to just protect yourself against user errors or similar, use snapshots. They're much faster than backup and roll back easily.

Horses for courses...

So I looked briefly at PBS. I don't want to create a ceph cluster or add more metal as I have storage already facilitated in a nice NAS with redundancy disks.

Would it be suitable to install PBS as a VM and then use the NAS as backup location for incremental backups?

I'm thinking if the PVE goes down (including the PBS VM), the backups are still located on another device and should be restorable. (I presume another instance of PBS would be able to restore the backups.). Or is my thinking here flawed?

 

[fcorbelli]

So I looked briefly at PBS. I don't want to create a ceph cluster or add more metal as I have storage already facilitated in a nice NAS with redundancy disks.

Would it be suitable to install PBS as a VM and then use the NAS as backup location for incremental backups?

I'm thinking if the PVE goes down (including the PBS VM), the backups are still located on another device and should be restorable. (I presume another instance of PBS would be able to restore the backups.). Or is my thinking here flawed?

I personally adopt both
A proxmox backup [an ESXi-VM in fact], and a 'file storage' backup that I manage manually with a crontab, getting data from zfs snapshots
The latter is (from the point of view of space occupied) definitely efficient
For the former (i.e. the proxmox backup) I have not yet attempted a full-scale restore
What I don't like so much is that [on zfs] it operates on zvol instead of files (at least as a default, maybe that can be changed)
On the other hand, I like the speed of deployment, suitable for users who are not "hardcore *nix" kind and prefer GUI

 

[lifeboy]

So I looked briefly at PBS. I don't want to create a ceph cluster or add more metal as I have storage already facilitated in a nice NAS with redundancy disks.

Would it be suitable to install PBS as a VM and then use the NAS as backup location for incremental backups?

I'm thinking if the PVE goes down (including the PBS VM), the backups are still located on another device and should be restorable. (I presume another instance of PBS would be able to restore the backups.). Or is my thinking here flawed?

PBS as a VM is definitely a good idea. However, PBS generates a fingerprint that you need to save, otherwise another instance won't be able to read to backups.

You can attach storage to PBS in many ways.



On the underlying OS (Debian) you can mount the storage and simply link to it in the PBS disks menu above.

If you're running PBS on a Proxmox Cluster (like I'm doing), you can add storage quite easily:



It installs in 2 ticks, so just install it and play around with it. It works really well and you can restore full VM or LXC images, just a partition, or only a file if you choose to.

 

[toomanylogins]

I run pbs on an old PC. Starts and stops twice a day for 30 mins and run backup tasks inc a fileserver guest . if you have linux fileserver vm the pbs backups are de-duped and very fast. Windows server backups are larger. Great thing is the individual file restore feature. Once host backup arrives in PBS the system will be bullet proof.

 

[SDMDS]

PBS as a VM is definitely a good idea. However, PBS generates a fingerprint that you need to save, otherwise another instance won't be able to read to backups.

From what I've read; the fingerprint is included in the proxy.pem and proxy.key. Once I've copied these two, I presume I can create a new PBS instance and restore from the backups?

You can attach storage to PBS in many ways.

View attachment 47164

On the underlying OS (Debian) you can mount the storage and simply link to it in the PBS disks menu above.

If you're running PBS on a Proxmox Cluster (like I'm doing), you can add storage quite easily:

View attachment 47165

It installs in 2 ticks, so just install it and play around with it. It works really well and you can restore full VM or LXC images, just a partition, or only a file if you choose to.

If I'm not mistaken, you add the storage in PVE and then mount it to the VM with the PBS?

 

[Dunuin]

It's no problem when the fingerprint changes. You will just have to re-add your PBS storages in PVE and update your scripts when using the proxmox-backup-client so the new fingerprint will be used.