Best practice for Proxmox self-backup

Discussion in 'Proxmox VE: Installation and configuration' started by speedbird, Nov 29, 2017.

  1. ulf.kosack

    ulf.kosack New Member
    Proxmox Subscriber

    Joined:
    Jan 28, 2012
    Messages:
    21
    Likes Received:
    3
    Yes, the first line (cat) is not in the crontab file.

    The standard crontab shell is /bin/sh. The script uses the /bin/bash. So you have to use the bash in the crontab file

    Code:
    SHELL=/bin/bash
     
  2. daniello

    daniello New Member

    Joined:
    Oct 13, 2018
    Messages:
    15
    Likes Received:
    0
    Thanks .. but doesn't help either :-(
     
  3. ulf.kosack

    ulf.kosack New Member
    Proxmox Subscriber

    Joined:
    Jan 28, 2012
    Messages:
    21
    Likes Received:
    3
    Can you execute the script in a normal shell?
    You can debug it via
    Code:
    bash -x /root/proxmox_config_backup.sh
    then you see all steps of the script.
     
  4. daniello

    daniello New Member

    Joined:
    Oct 13, 2018
    Messages:
    15
    Likes Received:
    0
    Ulf .. due to the above command I noticed that there was a mismatch in the script's backup path and the path I entered in to the crontab. Further testing showed that the only line I needed to add in the crontab was this one:

    TERM=xterm

    So the path is backup taken correctly from the script and sh vs. bash is not the problem.

    My final crontab looks like this:

    TERM=xterm
    14 1 * * * /usr/bin/curl -I http://192.168.xx.xx:80 >/dev/null 2>&1
    15 1 * * * /bin/echo | /prox_config_backup.sh >/dev/null 2>&1
    16 1 * * * /usr/bin/find /mnt/nas/_system -type f -mtime +7 -exec rm {} + >/dev/null 2>&1

    Thanks for taking your time .. your help is much appreciated!
     
    #44 daniello, Oct 21, 2018
    Last edited: Oct 21, 2018
  5. RobFantini

    RobFantini Well-Known Member
    Proxmox Subscriber

    Joined:
    May 24, 2012
    Messages:
    1,597
    Likes Received:
    26
    as an earlier post suggested - rsnapshot is a superb tool for backing up pve host.

    we've used it since it was written on all systems to backup locally and remote.

    from the man page
    Code:
    DESCRIPTION
           rsnapshot is a filesystem snapshot utility. It can take incremental snapshots of local and remote filesystems for any number
           of machines.
    
           Local filesystem snapshots are handled with rsync(1). Secure remote connections are handled with rsync over ssh(1), while
           anonymous rsync connections simply use an rsync server. Both remote and local transfers depend on rsync.
    
           rsnapshot saves much more disk space than you might imagine. The amount of space required is roughly the size of one full
           backup, plus a copy of each additional file that is changed. rsnapshot makes extensive use of hard links, so if the file
           doesn't change, the next snapshot is simply a hard link to the exact same file.
    
           rsnapshot will typically be invoked as root by a cron job, or series of cron jobs. It is possible, however, to run as any
           arbitrary user with an alternate configuration file.
    
           All important options are specified in a configuration file, which is located by default at /etc/rsnapshot.conf. An
           alternate file can be specified on the command line. There are also additional options which can be passed on the command
           line.
    
    
    
    we use rsnapshot to backup systems and data. it is a great backup tool.
     
  6. DerDanilo

    DerDanilo Member
    Proxmox Subscriber

    Joined:
    Jan 21, 2017
    Messages:
    266
    Likes Received:
    24
    You are welcome to submit your rsnapshot backup code as pull request.
    Maybe it's even better to provide it as an alternative tool than using only utilities that are available on default.
     
  7. guletz

    guletz Active Member

    Joined:
    Apr 19, 2017
    Messages:
    929
    Likes Received:
    124
    https://rsnapshot.org/

    Rsnapshot is present in repo for almost any linux distro(debian/ubuntu/centos/what-ever) for many years:

    apt install rsnapshot
     
  8. guletz

    guletz Active Member

    Joined:
    Apr 19, 2017
    Messages:
    929
    Likes Received:
    124
    In my own oppinion it is far better to use rsnapshot insted of any other backup script, because it have ALL that you need for this task(including: encryption, rsync, ssh, pre/post backup custom scripts, schedule, retention backup policy, hard links for low disk space usag, very well documented) and not least it is rock-solid(I use it for many ...many years on any linux server, and I do not have get any failure when I was need to recover something)!!! And I am very sure that could be easy integrated in PMX.
     
    #48 guletz, Oct 23, 2018
    Last edited: Oct 23, 2018
  9. RobFantini

    RobFantini Well-Known Member
    Proxmox Subscriber

    Joined:
    May 24, 2012
    Messages:
    1,597
    Likes Received:
    26
    Hellpo Guletz .

    here is what we backup using rsnapshot. Is there anything else that you include?

    This is for local backup . We also do remote pulls.
    Code:
    backup  /etc/                           localhost/
    backup  /etc/pve/                       localhost/
    backup  /var/lib/pve-cluster/           localhost/
    backup  /root/.ssh/                     localhost/
    backup  /usr/local/bin/                 localhost/
    
     
  10. guletz

    guletz Active Member

    Joined:
    Apr 19, 2017
    Messages:
    929
    Likes Received:
    124
    Hi,

    I also use this(and include yours):

    /var/lib/pve-firewall
    /var/lib/pve-manager
    /var/lib/pve-zsync
    /var/lib/lxc
    /var/spool/cron
    /root
    /boot
     
  11. daniello

    daniello New Member

    Joined:
    Oct 13, 2018
    Messages:
    15
    Likes Received:
    0
    Is this not redundant?
     
  12. RobFantini

    RobFantini Well-Known Member
    Proxmox Subscriber

    Joined:
    May 24, 2012
    Messages:
    1,597
    Likes Received:
    26
    it is intentional, as sometimes rsync can be configured to not sync a different mounted file system [ within a directory specified to rsync]. so it is insurance in case of a mis configuration.
     
    guletz and daniello like this.
  13. RobFantini

    RobFantini Well-Known Member
    Proxmox Subscriber

    Joined:
    May 24, 2012
    Messages:
    1,597
    Likes Received:
    26
    Thank you for that .

    I added all of those to our off node backup - we use a large lxc named rsnapshot on nfs .

    for local backups - those go to /rsnapshot , added all except /boot as there are sometimes storage space limitations.
     
  14. guletz

    guletz Active Member

    Joined:
    Apr 19, 2017
    Messages:
    929
    Likes Received:
    124
    Wellcome ;)
     
  15. sdet00

    sdet00 New Member

    Joined:
    Nov 18, 2017
    Messages:
    21
    Likes Received:
    1
    Very interesting thread, look forward to testing this out. The limited backup abilities have always made me nervous to the point where I have been manually creating images of the PVE boot SSD just in case.

    That said, can anyone shine some light on what to do if you needed to restore from a backup? Is there a way of testing restores?

    Also, what about tools like Veeam Agent for Linux - would that work with Proxmox?
     
  16. Zaman

    Zaman New Member

    Joined:
    Apr 15, 2019
    Messages:
    19
    Likes Received:
    0
    Hello,
    i search about that,but there is no luck so how to do it.
    thanks
     
  17. guletz

    guletz Active Member

    Joined:
    Apr 19, 2017
    Messages:
    929
    Likes Received:
    124
    Hi,

    I can guessing what @DerDanilo was trying to say ;)

    - you can add a dedicated node or use a existent one only for backup
    - this node as the rest of yours node must use zfs as VM/CT storage
    - in Proxmox you can use for any VM/CT the replications using as destination this dedicated node


    Good luck.
     
    Zaman and DerDanilo like this.
  18. Vladimir Bulgaru

    Joined:
    Jun 1, 2019
    Messages:
    125
    Likes Received:
    17
    If i were to add to this discussion, i personally think it's not the most appropriate way to address the Proxmox backup. I'd separate the backup part into 2 aspects:
    1. backing up the configs (storage, cluster, network, VMs, containers, backup cron)
    2. restoring the OS
    I see that the optimal approach is the following:
    1. for configs - simply have them somewhere offsite. Here's what you need to back up: https://pve.proxmox.com/wiki/Bypassing_backup_and_restore_when_upgrading
    2. for the OS - i think that any enterprise should aim to have a deployment script that would get the machine up and running within 2-3 minutes
    The main reason why i think it's a bad idea to restore the OS from a backup is because it addresses only the hardware failure scenario, while ignoring the data corruption and breach scenarios. Therefore restoring the backup without addressing the vulnerabilities creates the false sense of security.
     
    Zaman, guletz and DerDanilo like this.
  19. guletz

    guletz Active Member

    Joined:
    Apr 19, 2017
    Messages:
    929
    Likes Received:
    124
    Agree. But you can use any HIDS who can check your important OS files. When you need to restore your VM, before go online, you can check it, and the if no checksum is changed then is a good possibility to think that is OK (I know is not 100 % sure, but what it is 100% sure in this days?)
     
  20. Zaman

    Zaman New Member

    Joined:
    Apr 15, 2019
    Messages:
    19
    Likes Received:
    0
    first: thanks
    second:
    Ok,how two connect each other by storage
     
  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.
    Dismiss Notice