proxmox-backup-client bash script for automated backups of laptop

[turnicus]

Hello,

I am planning to backup my linux laptop via proxmox-backup-client. I just wrote this little script to automate the process:

#!/bin/bash

### Depedencies ########################################
# jq (to parse the json output of snapshots)           #
# PBS_PASSWORD must be exported in:                    #
#   ~/.bashrc (to be run by user)                      #
#   ~/.bash_profile (to be run by cron with "bash -l") #
########################################################

### Settings ###################
_pbs_ip_address="192.168.x.x"  #
_pbs_user="user@pbs"           #
_pbs_datastore="backups"       #
################################


# I check that I am not connected to a metered network
# (See https://developer.gnome.org/NetworkManager/stable/nm-dbus-types.html#NMMetered)

_metered_value=$(busctl get-property org.freedesktop.NetworkManager /org/freedesktop/NetworkManager org.freedesktop.NetworkManager Metered)

if [ "$_metered_value" == "u 4" ] || [ "$_metered_value" == "u 2" ]; then
  echo "- The bandwidth on this network is probably not metered"
  # I check that I can ping the server
  ping $_pbs_ip_address -c 1 > /dev/null 2>&1
  if [ $? == 0 ]; then
    echo "- The Proxmox Backup Server is reachable"
    _epoch_last_backup=$(proxmox-backup-client snapshots --output-format json --repository $_pbs_user@$_pbs_ip_address:$_pbs_datastore | jq '[ .[]."backup-time" ] | max')
    _seconds_elapsed_since_last_backup=$(($(date +%s)-$_epoch_last_backup))
    echo "- The last backup was received on "$(date -d @$_epoch_last_backup)" ("$_seconds_elapsed_since_last_backup" seconds ago)"
    # I run the backup if the last one is more than 24H old (86400 seconds)
    if [ $_seconds_elapsed_since_last_backup -gt 86400 ]; then
      echo "- I start the backup"
      proxmox-backup-client backup root.pxar:/ --repository $_pbs_user@$_pbs_ip_address:$_pbs_datastore
    fi
  fi
fi

My daily computer being a laptop, I don't have a fixed time to backup (I close it when I don't use it). So my idea is to run this script via crontab (every hour for example) and have the script check that:

1. I am NOT connected to a metered network (such as my mobile phone acting as a hotspot for example)
2. I can ping the server. If I am at a hotel and forgot to connect to my VPN, I don't want to send my credentials for nothing (I am not sure the communication with PBS is encrypted?)
3. The latest backup is at least 24 hours old

What do you think about this approach?

I still have a few questions though:

• unprivilegied users can't backup some root files. So I am planning to run it via root's crontab. Any better idea?
• On PBS side, I guess I just need the "read" and "backup" permissions (to get the latest backup time, and to backup). So I had to grant 2 separate permissions (1x for DatastoreReader + 1x for DatastoreBackup) in order to limit the permissions to the maximum. Is there a better way?
• If possible, I would like to postpone my backup if another client is already busy backing up on my PBS (to limit the pressure on the network). Is it possible to check if another client is busy (possibly without granting too much permissions)?

Thanks for your advises!

 

[fabian]

My daily computer being a laptop, I don't have a fixed time to backup (I close it when I don't use it). So my idea is to run this script via crontab (every hour for example) and have the script check that:

you could use systemd timers to get more flexible scheduling than regular cron

1. I am NOT connected to a metered network (such as my mobile phone acting as a hotspot for example)

probably a good idea

1. I can ping the server. If I am at a hotel and forgot to connect to my VPN, I don't want to send my credentials for nothing (I am not sure the communication with PBS is encrypted?)

you can of course still do that, but there is no way for the client to communicate with the server over a plain-text link. it's all over TLS (which is why you need either a trusted certificate or give/confirm the fingerprint of the certificate).

1. The latest backup is at least 24 hours old

What do you think about this approach?

I still have a few questions though:

• unprivilegied users can't backup some root files. So I am planning to run it via root's crontab. Any better idea?

obviously the client needs to run as a user that has access to everything that you want to backup.

• On PBS side, I guess I just need the "read" and "backup" permissions (to get the latest backup time, and to backup). So I had to grant 2 separate permissions (1x for DatastoreReader + 1x for DatastoreBackup) in order to limit the permissions to the maximum. Is there a better way?

you guessed wrong, just DatastoreBackup is enough to make a new backup group (first backup of a given type+ID), read existing backups and their snapshots if owned by that user, and make new snapshots into groups owned by that user.

DatastoreReader gives you read access to all backups, irrespective of ownership.

• If possible, I would like to postpone my backup if another client is already busy backing up on my PBS (to limit the pressure on the network). Is it possible to check if another client is busy (possibly without granting too much permissions)?

no, that's not possible currently. but usually the load for host backups is higher on the client than on the server side, since the client has to read+hash all the chunks, and the server only has to verify and write the new ones.

Thanks for your advises!

I'd add to all of the above that you might want to check out the recently added API tokens, which allow you to differentiate permissions and ownership even within a user - so you could create on API token for your laptop client that can only see its own backup snapshots, another one for another system that only sees itself, but you can still manage all those with your single user.

 

[turnicus]

Hello Fabian and thanks for your answer. I see I was heading in the right direction then.

you could use systemd timers to get more flexible scheduling than regular cron

Thanks, I went for cron because I need a dead simple schedule (every hour for example) but I'll keep that in mind.

you can of course still do that, but there is no way for the client to communicate with the server over a plain-text link. it's all over TLS (which is why you need either a trusted certificate or give/confirm the fingerprint of the certificate).

Thanks for clarifying about TLS. I didn't remember I had the accept the certificate the first time (and I didn't bother checking the traffic with wireshark before posting)... I think I'll leave the ping phase anyway, so I don't have to handle the errors if the server is not reachable (it's just one ping anyway...)

you guessed wrong, just DatastoreBackup is enough to make a new backup group (first backup of a given type+ID), read existing backups and their snapshots if owned by that user, and make new snapshots into groups owned by that user.

Good to know that DatastoreBackup is enough, I'll remove the DatastoreReader permission as I don't need it

I'd add to all of the above that you might want to check out the recently added API tokens, which allow you to differentiate permissions and ownership even within a user - so you could create on API token for your laptop client that can only see its own backup snapshots, another one for another system that only sees itself, but you can still manage all those with your single user.

I just checked the documentation but I don't see the added granularity in my case. Do you confirm a user with DatastoreBackup permission can only backup and see the timestamps of its own snapshots? In my case, I don't even need it to be able to read the content of snapshots (except the timestamps) or restore because I would do that "manually" if I need to. Pruning is done by PBS so this permission is not necessary either. Or did you have another idea in mind?

• By the way, do have a plan to store the PBS_PASSWORD in safer way ?
• I am planning to backup some raspberries too but they all have the same hostname... Is there a way to force another name for the backup?

Many thanks for this great product!

 

[fabian]

I just checked the documentation but I don't see the added granularity in my case. Do you confirm a user with DatastoreBackup permission can only backup and see the timestamps of its own snapshots? In my case, I don't even need it to be able to read the content of snapshots (except the timestamps) or restore because I would do that "manually" if I need to. Pruning is done by PBS so this permission is not necessary either. Or did you have another idea in mind?

DatastoreBackup allows
- creating new backup groups (which will set the owner)
- creating new snapshots of owned backup groups
- reading existing snapshots (including restoring contents) of owned backup groups
- uploading the client-side backup log for an owned snapshot
- updating the notes of an owned backup snapshot

• By the way, do have a plan to store the PBS_PASSWORD in safer way ?

you can store it however you want you just need to pass it to proxmox-backup-client (we might create a way to pass it in via stdin or an FD, the latter is already supported for the encryption key). alternatively or in addition, you can setup API tokens which belong to a user and can be revoked on their own to minimize the risk.

• I am planning to backup some raspberries too but they all have the same hostname... Is there a way to force another name for the backup?

when doing host-type backups you can just pass --backup-id SOMETHING, the default value is the hostname.

 

[turnicus]

Hello Fabian and thanks again for your clarifications. I am ready to go now.

regarding this specific use case (backing up a laptop which is not always connected), there will be a lot of aborted backups. Can it lead to problems someday? Is it advisable to run more garbage collections for example? When a backup is aborted, will it start again from the beginning the next time or resume the aborted backup?

 

[fabian]

Hello Fabian and thanks again for your clarifications. I am ready to go now.

regarding this specific use case (backing up a laptop which is not always connected), there will be a lot of aborted backups. Can it lead to problems someday? Is it advisable to run more garbage collections for example? When a backup is aborted, will it start again from the beginning the next time or resume the aborted backup?

it will always use the last finished backup as incremental base, so from the clients point of view it's like the failed one never happened. you might save some disk I/O on the PBS side if chunks from the aborted backup are still around and haven't been GCed yet. I'd not decrease GC intervals just for that reason alone, unless you are running out of space (but then pruning more aggressively will likely help more ).