NAS share backup

trigg3r

Active Member
Aug 3, 2019
41
3
28
23
To backup a QNAP/Synology share I was thinking of mounting this share via NFS on PBS server and running proxmox-backup-client on this server... can this work?

Or is it better to mount the share as a PVE storage and then run proxmox-backup-client from there?
 
I managed to configure everything, but there is something that doesn't work (which makes me think it is necessary to perform some kind of "reset").

This is what I see after starting the systemd service that run the backup:
backup.png

This is what I see after the backup has finished (ie: no backup exists):
after_backup.png

cat /etc/systemd/system/pbs-bkp.service
INI:
[Unit]
Description=Backup NAS2(mainstorage) share: myshare1, myshare2, myshare3, myshare4
Requires=mnt-nas-mainstorage-myshare1.mount mnt-nas-mainstorage-myshare2.mount mnt-nas-mainstorage-myshare3.mount mnt-nas-mainstorage-myshare4.mount

[Service]
Type=simple
ExecStart=/usr/bin/proxmox-backup-client backup \
"myshare1.pxar:/mnt/nas/mainstorage/myshare1" \
"myshare2.pxar:/mnt/nas/mainstorage/myshare2" \
"myshare3.pxar:/mnt/nas/mainstorage/myshare3" \
"myshare4.pxar:/mnt/nas/mainstorage/myshare4" \
--ns "mainstorage"
Nice=19

[Install]
WantedBy=multi-user.target

cat /etc/systemd/system/pbs-bkp.service.d/override.conf
INI:
[Service]
Environment="PBS_PASSWORD_FILE=/etc/proxmox-backup-client/pbs-bkp"
Environment="PBS_REPOSITORY=myuser@pbs!myusertoken@localhost:Backup"

journalctl -u pbs-bkp.service

Code:
Started pbs-bkp.service - Backup NAS2(mainstorage) share: myshare1, myshare2, myshare3, myshare4.
Starting backup: [mainstorage]:host/pbs1/2024-05-06
Client name: pbs1
Starting backup protocol: Mon May 6 2024
No previous manifest available.
Upload directory '/mnt/nas/mainstorage/myshare1' to 'myuser@pbs!myusertoken@localhost:8007:Backup' as myshare1.pxar.didx
myshare1.pxar: had to backup 30.556 GiB of 37.969 GiB (compressed 26.729 GiB) in 1683.75s
myshare1.pxar: average backup speed: 18.583 MiB/s
myshare1.pxar: backup was done incrementally, reused 7.413 GiB (19.5%)
Upload directory '/mnt/nas/mainstorage/myshare2' to 'myuser@pbs!myusertoken@localhost:8007:Backup' as myshare2.pxar.didx
Stopping pbs-bkp.service - Backup share NAS2(mainstorage): myshare1, myshare2, myshare3, myshare4...
pbs-bkp.service: Deactivated successfully.
Stopped pbs-bkp.service - Backup share NAS2(mainstorage): myshare1, myshare2, myshare3, myshare4.
pbs-bkp.service: Consumed 6min 50.258s CPU time.

From the logs it even seems that only myshare1 and myshare2 are backed up and not also myshare3 and myshare4.

What is going on?
 
Here I am...

From CLI everything is ok:

Code:
Starting backup: [mainstorage]:host/pbs1/2024-05-11T06:55:01Z
Client name: pbs1
Starting backup protocol: Sat May 11 08:55:01 2024
Downloading previous manifest (Mon May  6 15:49:42 2024)

Upload directory '/mnt/nas/mainstorage/myshare1' to 'mydomain@pbs!mydomainbackup@localhost:8007:Backup' as myshare1.pxar.didx
myshare1.pxar: had to backup 752.894 MiB of 38.115 GiB (compressed 606.072 MiB) in 2036.31s
myshare1.pxar: average backup speed: 378.607 KiB/s
myshare1.pxar: backup was done incrementally, reused 37.379 GiB (98.1%)

Upload directory '/mnt/nas/mainstorage/myshare2' to 'mydomain@pbs!mydomainbackup@localhost:8007:Backup' as myshare2.pxar.didx
Previous manifest does not contain an archive called 'myshare2.pxar.didx', skipping download..
myshare2.pxar: had to backup 5.106 GiB of 6.075 GiB (compressed 827.012 MiB) in 81.48s
myshare2.pxar: average backup speed: 64.17 MiB/s
myshare2.pxar: backup was done incrementally, reused 991.573 MiB (15.9%)

Upload directory '/mnt/nas/mainstorage/myshare3' to 'mydomain@pbs!mydomainbackup@localhost:8007:Backup' as myshare3.pxar.didx
Previous manifest does not contain an archive called 'myshare3.pxar.didx', skipping download..
myshare3.pxar: had to backup 312.867 MiB of 312.867 MiB (compressed 293.059 MiB) in 9.77s
myshare3.pxar: average backup speed: 32.016 MiB/s

Upload directory '/mnt/nas/mainstorage/myshare4' to 'mydomain@pbs!mydomainbackup@localhost:8007:Backup' as myshare4.pxar.didx
Previous manifest does not contain an archive called 'myshare4.pxar.didx', skipping download..
myshare4.pxar: had to backup 5.41 MiB of 5.41 MiB (compressed 1.642 MiB) in 2.27s
myshare4.pxar: average backup speed: 2.386 MiB/s

Uploaded backup catalog (2.723 MiB)
Duration: 2129.89s
End Time: Sat May 11 09:30:31 2024

These are the logs relating to backups performed via SystemD:

Task:
Code:
2024-05-10T20:30:12+02:00: starting new backup on datastore 'Backup' from ::ffff:127.0.0.1: "ns/mainstorage/host/pbs1/2024-05-10T18:30:12Z"
2024-05-10T20:30:12+02:00: protocol upgrade done
2024-05-10T20:30:12+02:00: GET /previous_backup_time
2024-05-10T20:30:12+02:00: GET /previous
2024-05-10T20:30:12+02:00: download 'index.json.blob' from previous backup.
2024-05-10T20:30:12+02:00: POST /dynamic_index
2024-05-10T20:30:12+02:00: created new dynamic index 1 ("ns/mainstorage/host/pbs1/2024-05-10T18:30:12Z/catalog.pcat1.didx")
2024-05-10T20:30:12+02:00: GET /previous
2024-05-10T20:30:12+02:00: register chunks in 'myshare1.pxar.didx' from previous backup.
2024-05-10T20:30:12+02:00: download 'myshare1.pxar.didx' from previous backup.
2024-05-10T20:30:12+02:00: POST /dynamic_index
2024-05-10T20:30:12+02:00: created new dynamic index 2 ("ns/mainstorage/host/pbs1/2024-05-10T18:30:12Z/myshare1.pxar.didx")
2024-05-10T20:30:43+02:00: PUT /dynamic_index
2024-05-10T20:30:43+02:00: dynamic_append 128 chunks
2024-05-10T20:30:43+02:00: successfully added chunk 88c82378d61e1b918945fd02ceb215305d4f983f268aebed1b85e8b579d28884 to dynamic index 2 (offset 0, size 5965285)
8<
2024-05-10T20:30:43+02:00: successfully added chunk 9e30a84da0070c5f679970bf520a38905b7f18818744af1b3971e756404b9ff6 to dynamic index 2 (offset 630696217, size 7328323)
2024-05-10T20:30:47+02:00: PUT /dynamic_index
2024-05-10T20:30:47+02:00: dynamic_append 128 chunks
2024-05-10T20:30:47+02:00: successfully added chunk 484eda13c395eca534ee51b411d08f9947efdbf8be047ec27eb3c2814479e422 to dynamic index 2 (offset 638024540, size 4797375)
8<
2024-05-10T21:01:40+02:00: successfully added chunk 25a184d91f91930f7a2f33b7a14214aea88b6d620f1d64c4b0baef4393053b10 to dynamic index 2 (offset 40911785411, size 13484256)
2024-05-10T21:01:40+02:00: POST /dynamic_close
2024-05-10T21:01:40+02:00: Upload statistics for 'myshare1.pxar.didx'
2024-05-10T21:01:40+02:00: UUID: 6dd44d5d534346ee8310f88616e955be
2024-05-10T21:01:40+02:00: Checksum: d207fdb4b1cef42c151ad9c9688c20c233f22fb371e5e9b708b233d6e4c6e7c1
2024-05-10T21:01:40+02:00: Size: 40925269667
2024-05-10T21:01:40+02:00: Chunk count: 11026
2024-05-10T21:01:40+02:00: Upload size: 789466452 (1%)
2024-05-10T21:01:40+02:00: Duplicates: 10883+115 (99%)
2024-05-10T21:01:40+02:00: Compression: 80%
2024-05-10T21:01:40+02:00: successfully closed dynamic index 2
2024-05-10T21:01:40+02:00: POST /dynamic_index
2024-05-10T21:01:40+02:00: created new dynamic index 3 ("ns/mainstorage/host/pbs1/2024-05-10T18:30:12Z/myshare2.pxar.didx")
2024-05-10T21:01:40+02:00: POST /dynamic_chunk
2024-05-10T21:01:40+02:00: upload_chunk done: 1941408 bytes, b79e82327a60362bac05a03d5e16e9a5d2f40e21c9411fa8117066e3c42952fe
2024-05-10T21:01:40+02:00: POST /dynamic_chunk
8<
2024-05-10T21:01:48+02:00: upload_chunk done: 8781991 bytes, fdd274e3a54bca09e6c190c11eb727afa4d5bd72585e583febfaef966b680ee8
2024-05-10T21:01:49+02:00: backup failed: connection error: connection reset
2024-05-10T21:01:49+02:00: removing failed backup
2024-05-10T21:01:49+02:00: TASK ERROR: connection error: connection reset

System:
Code:
May 10 21:01:48 pbs1 proxmox-backup-proxy[1020]: upload_chunk done: 4162039 bytes, 8359ea5412565714b125961f545a544f00dfed3d9d9955e60bc01e236c4b0011
May 10 21:01:48 pbs1 proxmox-backup-proxy[1020]: POST /dynamic_chunk
May 10 21:01:48 pbs1 proxmox-backup-proxy[1020]: upload_chunk done: 1551295 bytes, daf64c928ecbb37a645f84abf3c19d5e8353054c09a5e55ef11a5860a5e36b35
May 10 21:01:48 pbs1 systemd[1]: Unmounting mnt-nas-mainstorage-myshare1.mount - Mount QNAP mainstorage share myshare1...
May 10 21:01:48 pbs1 systemd[1]: Stopping pbs-bkp.service - Backup share NAS2(mainstorage): myshare1, myshare2, myshare3, myshare4...
May 10 21:01:48 pbs1 proxmox-backup-proxy[1020]: upload_chunk done: 8781991 bytes, fdd274e3a54bca09e6c190c11eb727afa4d5bd72585e583febfaef966b680ee8
May 10 21:01:49 pbs1 proxmox-backup-proxy[1020]: backup failed: connection error: connection reset
May 10 21:01:49 pbs1 proxmox-backup-proxy[1020]: removing failed backup
May 10 21:01:49 pbs1 proxmox-backup-proxy[1020]: removing backup snapshot "/mnt/datastore/Backup/ns/mainstorage/host/pbs1/2024-05-10T18:30:12Z"
May 10 21:01:49 pbs1 systemd[1]: pbs-bkp.service: Deactivated successfully.
May 10 21:01:49 pbs1 systemd[1]: Stopped pbs-bkp.service - Backup share NAS2(mainstorage): myshare1, myshare2, myshare3, myshare4.
May 10 21:01:49 pbs1 proxmox-backup-proxy[1020]: TASK ERROR: connection error: connection reset
May 10 21:01:49 pbs1 systemd[1]: pbs-bkp.service: Consumed 4min 29.658s CPU time.
May 10 21:01:50 pbs1 systemd[1]: mnt-nas-mainstorage-myshare1.mount: Deactivated successfully.
May 10 21:01:50 pbs1 systemd[1]: Unmounted mnt-nas-mainstorage-myshare1.mount - Mount QNAP mainstorage share myshare1.
May 10 21:01:50 pbs1 systemd[1]: mnt-nas-mainstorage-myshare1.mount: Consumed 1.251s CPU time.
May 10 21:12:43 pbs1 proxmox-backup-proxy[1020]: write rrd data back to disk
May 10 21:12:43 pbs1 proxmox-backup-proxy[1020]: starting rrd data sync
May 10 21:12:43 pbs1 proxmox-backup-proxy[1020]: rrd journal successfully committed (33 files in 0.011 seconds)
May 10 21:17:01 pbs1 CRON[343597]: pam_unix(cron:session): session opened for user root(uid=0) by (uid=0)
May 10 21:17:01 pbs1 CRON[343598]: (root) CMD (cd / && run-parts --report /etc/cron.hourly)
May 10 21:17:01 pbs1 CRON[343597]: pam_unix(cron:session): session closed for user root
May 10 21:40:22 pbs1 smartd[745]: Device: /dev/sdb [SAT], SMART Usage Attribute: 190 Airflow_Temperature_Cel changed from 57 to 58
May 10 21:40:22 pbs1 smartd[745]: Device: /dev/sdb [SAT], SMART Usage Attribute: 194 Temperature_Celsius changed from 57 to 58
May 10 21:40:22 pbs1 smartd[745]: Device: /dev/sdc [SAT], SMART Usage Attribute: 190 Airflow_Temperature_Cel changed from 59 to 60
May 10 21:40:22 pbs1 smartd[745]: Device: /dev/sdc [SAT], SMART Usage Attribute: 194 Temperature_Celsius changed from 59 to 60
May 10 21:42:43 pbs1 proxmox-backup-proxy[1020]: write rrd data back to disk
May 10 21:42:43 pbs1 proxmox-backup-proxy[1020]: starting rrd data sync
 
Seems like systemd is killing the backup task because for some reason mnt-nas-mainstorage-myshare1.mount is unmounted and pbs-bkp.service Requires that service to be running.

Code:
May 10 21:01:48 pbs1 proxmox-backup-proxy[1020]: upload_chunk done: 4162039 bytes, 8359ea5412565714b125961f545a544f00dfed3d9d9955e60bc01e236c4b0011
May 10 21:01:48 pbs1 proxmox-backup-proxy[1020]: POST /dynamic_chunk
May 10 21:01:48 pbs1 proxmox-backup-proxy[1020]: upload_chunk done: 1551295 bytes, daf64c928ecbb37a645f84abf3c19d5e8353054c09a5e55ef11a5860a5e36b35
May 10 21:01:48 pbs1 systemd[1]: Unmounting mnt-nas-mainstorage-myshare1.mount - Mount QNAP mainstorage share myshare1...
May 10 21:01:48 pbs1 systemd[1]: Stopping pbs-bkp.service - Backup share NAS2(mainstorage): myshare1, myshare2, myshare3, myshare4...
May 10 21:01:48 pbs1 proxmox-backup-proxy[1020]: upload_chunk done: 8781991 bytes, fdd274e3a54bca09e6c190c11eb727afa4d5bd72585e583febfaef966b680ee8
May 10 21:01:49 pbs1 proxmox-backup-proxy[1020]: backup failed: connection error: connection reset

Try to find out why the nas is unmounted.
 
It is mindblowing that after years nobody from Proxmox has actually implemented some way to add a NAS smb/nfs share as backup storage...
All my attempts have failed years ago and last few weeks they failed again with inexplicable errors/permissions denied. It is totally unclear how to setup the proper users/permissions/mounts to do this...
 
Fixed! There were two issues:
- the service Type, which being a backup should be oneshot
- the timeout settings were probably the cause of the shares being umounted

(credits: ChatGptO)
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!