NAS share backup

[trigg3r]

To backup a QNAP/Synology share I was thinking of mounting this share via NFS on PBS server and running proxmox-backup-client on this server... can this work?

Or is it better to mount the share as a PVE storage and then run proxmox-backup-client from there?

 

[VictorSTS]

Both options are valid, being the mount on PBS preferable due to it's better network efficiency: data will cross the network only to read from the NFS and stored locally in PBS.

 

[trigg3r]

Thanks @VictorSTS !

 

[trigg3r]

I managed to configure everything, but there is something that doesn't work (which makes me think it is necessary to perform some kind of "reset").

This is what I see after starting the systemd service that run the backup:


This is what I see after the backup has finished (ie: no backup exists):


cat /etc/systemd/system/pbs-bkp.service

[Unit]
Description=Backup NAS2(mainstorage) share: myshare1, myshare2, myshare3, myshare4
Requires=mnt-nas-mainstorage-myshare1.mount mnt-nas-mainstorage-myshare2.mount mnt-nas-mainstorage-myshare3.mount mnt-nas-mainstorage-myshare4.mount

[Service]
Type=simple
ExecStart=/usr/bin/proxmox-backup-client backup \
"myshare1.pxar:/mnt/nas/mainstorage/myshare1" \
"myshare2.pxar:/mnt/nas/mainstorage/myshare2" \
"myshare3.pxar:/mnt/nas/mainstorage/myshare3" \
"myshare4.pxar:/mnt/nas/mainstorage/myshare4" \
--ns "mainstorage"
Nice=19

[Install]
WantedBy=multi-user.target

cat /etc/systemd/system/pbs-bkp.service.d/override.conf

[Service]
Environment="PBS_PASSWORD_FILE=/etc/proxmox-backup-client/pbs-bkp"
Environment="PBS_REPOSITORY=myuser@pbs!myusertoken@localhost:Backup"

journalctl -u pbs-bkp.service

Started pbs-bkp.service - Backup NAS2(mainstorage) share: myshare1, myshare2, myshare3, myshare4.
Starting backup: [mainstorage]:host/pbs1/2024-05-06
Client name: pbs1
Starting backup protocol: Mon May 6 2024
No previous manifest available.
Upload directory '/mnt/nas/mainstorage/myshare1' to 'myuser@pbs!myusertoken@localhost:8007:Backup' as myshare1.pxar.didx
myshare1.pxar: had to backup 30.556 GiB of 37.969 GiB (compressed 26.729 GiB) in 1683.75s
myshare1.pxar: average backup speed: 18.583 MiB/s
myshare1.pxar: backup was done incrementally, reused 7.413 GiB (19.5%)
Upload directory '/mnt/nas/mainstorage/myshare2' to 'myuser@pbs!myusertoken@localhost:8007:Backup' as myshare2.pxar.didx
Stopping pbs-bkp.service - Backup share NAS2(mainstorage): myshare1, myshare2, myshare3, myshare4...
pbs-bkp.service: Deactivated successfully.
Stopped pbs-bkp.service - Backup share NAS2(mainstorage): myshare1, myshare2, myshare3, myshare4.
pbs-bkp.service: Consumed 6min 50.258s CPU time.

From the logs it even seems that only myshare1 and myshare2 are backed up and not also myshare3 and myshare4.

What is going on?

 

[VictorSTS]

Does it work if you run that same command on the shell? You would need to populate and export PBS_PASSWORD_FILE and PBS_REPOSITORY

 

[trigg3r]

Here I am...

From CLI everything is ok:

Starting backup: [mainstorage]:host/pbs1/2024-05-11T06:55:01Z
Client name: pbs1
Starting backup protocol: Sat May 11 08:55:01 2024
Downloading previous manifest (Mon May  6 15:49:42 2024)

Upload directory '/mnt/nas/mainstorage/myshare1' to 'mydomain@pbs!mydomainbackup@localhost:8007:Backup' as myshare1.pxar.didx
myshare1.pxar: had to backup 752.894 MiB of 38.115 GiB (compressed 606.072 MiB) in 2036.31s
myshare1.pxar: average backup speed: 378.607 KiB/s
myshare1.pxar: backup was done incrementally, reused 37.379 GiB (98.1%)

Upload directory '/mnt/nas/mainstorage/myshare2' to 'mydomain@pbs!mydomainbackup@localhost:8007:Backup' as myshare2.pxar.didx
Previous manifest does not contain an archive called 'myshare2.pxar.didx', skipping download..
myshare2.pxar: had to backup 5.106 GiB of 6.075 GiB (compressed 827.012 MiB) in 81.48s
myshare2.pxar: average backup speed: 64.17 MiB/s
myshare2.pxar: backup was done incrementally, reused 991.573 MiB (15.9%)

Upload directory '/mnt/nas/mainstorage/myshare3' to 'mydomain@pbs!mydomainbackup@localhost:8007:Backup' as myshare3.pxar.didx
Previous manifest does not contain an archive called 'myshare3.pxar.didx', skipping download..
myshare3.pxar: had to backup 312.867 MiB of 312.867 MiB (compressed 293.059 MiB) in 9.77s
myshare3.pxar: average backup speed: 32.016 MiB/s

Upload directory '/mnt/nas/mainstorage/myshare4' to 'mydomain@pbs!mydomainbackup@localhost:8007:Backup' as myshare4.pxar.didx
Previous manifest does not contain an archive called 'myshare4.pxar.didx', skipping download..
myshare4.pxar: had to backup 5.41 MiB of 5.41 MiB (compressed 1.642 MiB) in 2.27s
myshare4.pxar: average backup speed: 2.386 MiB/s

Uploaded backup catalog (2.723 MiB)
Duration: 2129.89s
End Time: Sat May 11 09:30:31 2024

These are the logs relating to backups performed via SystemD:

Task:

2024-05-10T20:30:12+02:00: starting new backup on datastore 'Backup' from ::ffff:127.0.0.1: "ns/mainstorage/host/pbs1/2024-05-10T18:30:12Z"
2024-05-10T20:30:12+02:00: protocol upgrade done
2024-05-10T20:30:12+02:00: GET /previous_backup_time
2024-05-10T20:30:12+02:00: GET /previous
2024-05-10T20:30:12+02:00: download 'index.json.blob' from previous backup.
2024-05-10T20:30:12+02:00: POST /dynamic_index
2024-05-10T20:30:12+02:00: created new dynamic index 1 ("ns/mainstorage/host/pbs1/2024-05-10T18:30:12Z/catalog.pcat1.didx")
2024-05-10T20:30:12+02:00: GET /previous
2024-05-10T20:30:12+02:00: register chunks in 'myshare1.pxar.didx' from previous backup.
2024-05-10T20:30:12+02:00: download 'myshare1.pxar.didx' from previous backup.
2024-05-10T20:30:12+02:00: POST /dynamic_index
2024-05-10T20:30:12+02:00: created new dynamic index 2 ("ns/mainstorage/host/pbs1/2024-05-10T18:30:12Z/myshare1.pxar.didx")
2024-05-10T20:30:43+02:00: PUT /dynamic_index
2024-05-10T20:30:43+02:00: dynamic_append 128 chunks
2024-05-10T20:30:43+02:00: successfully added chunk 88c82378d61e1b918945fd02ceb215305d4f983f268aebed1b85e8b579d28884 to dynamic index 2 (offset 0, size 5965285)
8<
2024-05-10T20:30:43+02:00: successfully added chunk 9e30a84da0070c5f679970bf520a38905b7f18818744af1b3971e756404b9ff6 to dynamic index 2 (offset 630696217, size 7328323)
2024-05-10T20:30:47+02:00: PUT /dynamic_index
2024-05-10T20:30:47+02:00: dynamic_append 128 chunks
2024-05-10T20:30:47+02:00: successfully added chunk 484eda13c395eca534ee51b411d08f9947efdbf8be047ec27eb3c2814479e422 to dynamic index 2 (offset 638024540, size 4797375)
8<
2024-05-10T21:01:40+02:00: successfully added chunk 25a184d91f91930f7a2f33b7a14214aea88b6d620f1d64c4b0baef4393053b10 to dynamic index 2 (offset 40911785411, size 13484256)
2024-05-10T21:01:40+02:00: POST /dynamic_close
2024-05-10T21:01:40+02:00: Upload statistics for 'myshare1.pxar.didx'
2024-05-10T21:01:40+02:00: UUID: 6dd44d5d534346ee8310f88616e955be
2024-05-10T21:01:40+02:00: Checksum: d207fdb4b1cef42c151ad9c9688c20c233f22fb371e5e9b708b233d6e4c6e7c1
2024-05-10T21:01:40+02:00: Size: 40925269667
2024-05-10T21:01:40+02:00: Chunk count: 11026
2024-05-10T21:01:40+02:00: Upload size: 789466452 (1%)
2024-05-10T21:01:40+02:00: Duplicates: 10883+115 (99%)
2024-05-10T21:01:40+02:00: Compression: 80%
2024-05-10T21:01:40+02:00: successfully closed dynamic index 2
2024-05-10T21:01:40+02:00: POST /dynamic_index
2024-05-10T21:01:40+02:00: created new dynamic index 3 ("ns/mainstorage/host/pbs1/2024-05-10T18:30:12Z/myshare2.pxar.didx")
2024-05-10T21:01:40+02:00: POST /dynamic_chunk
2024-05-10T21:01:40+02:00: upload_chunk done: 1941408 bytes, b79e82327a60362bac05a03d5e16e9a5d2f40e21c9411fa8117066e3c42952fe
2024-05-10T21:01:40+02:00: POST /dynamic_chunk
8<
2024-05-10T21:01:48+02:00: upload_chunk done: 8781991 bytes, fdd274e3a54bca09e6c190c11eb727afa4d5bd72585e583febfaef966b680ee8
2024-05-10T21:01:49+02:00: backup failed: connection error: connection reset
2024-05-10T21:01:49+02:00: removing failed backup
2024-05-10T21:01:49+02:00: TASK ERROR: connection error: connection reset

System:

May 10 21:01:48 pbs1 proxmox-backup-proxy[1020]: upload_chunk done: 4162039 bytes, 8359ea5412565714b125961f545a544f00dfed3d9d9955e60bc01e236c4b0011
May 10 21:01:48 pbs1 proxmox-backup-proxy[1020]: POST /dynamic_chunk
May 10 21:01:48 pbs1 proxmox-backup-proxy[1020]: upload_chunk done: 1551295 bytes, daf64c928ecbb37a645f84abf3c19d5e8353054c09a5e55ef11a5860a5e36b35
May 10 21:01:48 pbs1 systemd[1]: Unmounting mnt-nas-mainstorage-myshare1.mount - Mount QNAP mainstorage share myshare1...
May 10 21:01:48 pbs1 systemd[1]: Stopping pbs-bkp.service - Backup share NAS2(mainstorage): myshare1, myshare2, myshare3, myshare4...
May 10 21:01:48 pbs1 proxmox-backup-proxy[1020]: upload_chunk done: 8781991 bytes, fdd274e3a54bca09e6c190c11eb727afa4d5bd72585e583febfaef966b680ee8
May 10 21:01:49 pbs1 proxmox-backup-proxy[1020]: backup failed: connection error: connection reset
May 10 21:01:49 pbs1 proxmox-backup-proxy[1020]: removing failed backup
May 10 21:01:49 pbs1 proxmox-backup-proxy[1020]: removing backup snapshot "/mnt/datastore/Backup/ns/mainstorage/host/pbs1/2024-05-10T18:30:12Z"
May 10 21:01:49 pbs1 systemd[1]: pbs-bkp.service: Deactivated successfully.
May 10 21:01:49 pbs1 systemd[1]: Stopped pbs-bkp.service - Backup share NAS2(mainstorage): myshare1, myshare2, myshare3, myshare4.
May 10 21:01:49 pbs1 proxmox-backup-proxy[1020]: TASK ERROR: connection error: connection reset
May 10 21:01:49 pbs1 systemd[1]: pbs-bkp.service: Consumed 4min 29.658s CPU time.
May 10 21:01:50 pbs1 systemd[1]: mnt-nas-mainstorage-myshare1.mount: Deactivated successfully.
May 10 21:01:50 pbs1 systemd[1]: Unmounted mnt-nas-mainstorage-myshare1.mount - Mount QNAP mainstorage share myshare1.
May 10 21:01:50 pbs1 systemd[1]: mnt-nas-mainstorage-myshare1.mount: Consumed 1.251s CPU time.
May 10 21:12:43 pbs1 proxmox-backup-proxy[1020]: write rrd data back to disk
May 10 21:12:43 pbs1 proxmox-backup-proxy[1020]: starting rrd data sync
May 10 21:12:43 pbs1 proxmox-backup-proxy[1020]: rrd journal successfully committed (33 files in 0.011 seconds)
May 10 21:17:01 pbs1 CRON[343597]: pam_unix(cron:session): session opened for user root(uid=0) by (uid=0)
May 10 21:17:01 pbs1 CRON[343598]: (root) CMD (cd / && run-parts --report /etc/cron.hourly)
May 10 21:17:01 pbs1 CRON[343597]: pam_unix(cron:session): session closed for user root
May 10 21:40:22 pbs1 smartd[745]: Device: /dev/sdb [SAT], SMART Usage Attribute: 190 Airflow_Temperature_Cel changed from 57 to 58
May 10 21:40:22 pbs1 smartd[745]: Device: /dev/sdb [SAT], SMART Usage Attribute: 194 Temperature_Celsius changed from 57 to 58
May 10 21:40:22 pbs1 smartd[745]: Device: /dev/sdc [SAT], SMART Usage Attribute: 190 Airflow_Temperature_Cel changed from 59 to 60
May 10 21:40:22 pbs1 smartd[745]: Device: /dev/sdc [SAT], SMART Usage Attribute: 194 Temperature_Celsius changed from 59 to 60
May 10 21:42:43 pbs1 proxmox-backup-proxy[1020]: write rrd data back to disk
May 10 21:42:43 pbs1 proxmox-backup-proxy[1020]: starting rrd data sync

 

[VictorSTS]

Seems like systemd is killing the backup task because for some reason mnt-nas-mainstorage-myshare1.mount is unmounted and pbs-bkp.service Requires that service to be running.

May 10 21:01:48 pbs1 proxmox-backup-proxy[1020]: upload_chunk done: 4162039 bytes, 8359ea5412565714b125961f545a544f00dfed3d9d9955e60bc01e236c4b0011
May 10 21:01:48 pbs1 proxmox-backup-proxy[1020]: POST /dynamic_chunk
May 10 21:01:48 pbs1 proxmox-backup-proxy[1020]: upload_chunk done: 1551295 bytes, daf64c928ecbb37a645f84abf3c19d5e8353054c09a5e55ef11a5860a5e36b35
May 10 21:01:48 pbs1 systemd[1]: Unmounting mnt-nas-mainstorage-myshare1.mount - Mount QNAP mainstorage share myshare1...
May 10 21:01:48 pbs1 systemd[1]: Stopping pbs-bkp.service - Backup share NAS2(mainstorage): myshare1, myshare2, myshare3, myshare4...
May 10 21:01:48 pbs1 proxmox-backup-proxy[1020]: upload_chunk done: 8781991 bytes, fdd274e3a54bca09e6c190c11eb727afa4d5bd72585e583febfaef966b680ee8
May 10 21:01:49 pbs1 proxmox-backup-proxy[1020]: backup failed: connection error: connection reset

Try to find out why the nas is unmounted.

 

[sublightnova]

It is mindblowing that after years nobody from Proxmox has actually implemented some way to add a NAS smb/nfs share as backup storage...
All my attempts have failed years ago and last few weeks they failed again with inexplicable errors/permissions denied. It is totally unclear how to setup the proper users/permissions/mounts to do this...

 

[trigg3r]

Fixed! There were two issues:
- the service Type, which being a backup should be oneshot
- the timeout settings were probably the cause of the shares being umounted

(credits: ChatGptO)