remote sync job - sync group vm/100 failed - No such file or directory (os error 2)

alebeta

Member
Mar 20, 2018
38
3
13
30
gonkar.com
Hi friends,

I have two PBS servers, one which I use for PVE backup and a second one where I want to do remote sync to store the data in a second location.

PBS for PVE version: 1.0-8
PBS Remote version: 1.0-8

The PBS server where I store the PVE backups use a NFS storage. It does works great and no problems had arise. In the other hand the PBS remote, is being configure with S3 storage mount with goofys. For the remote, the data store is added successful, but the error comers when the sync job runs and I see the logs:

Code:
2021-02-23T15:48:51+01:00: found 16 groups to sync
2021-02-23T15:48:53+01:00: sync snapshot "ct/100/2021-02-20T03:30:02Z"
2021-02-23T15:48:54+01:00: percentage done: 1.56% (0 of 16 groups, 1 of 4 group snapshots)
2021-02-23T15:48:54+01:00: sync group ct/100 failed - No such file or directory (os error 2)
2021-02-23T15:48:54+01:00: sync snapshot "ct/104/2021-02-20T03:30:01Z"
2021-02-23T15:48:55+01:00: percentage done: 7.81% (1 of 16 groups, 1 of 4 group snapshots)
2021-02-23T15:48:57+01:00: sync snapshot "vm/1006/2021-01-31T00:18:01Z"
2021-02-23T15:48:58+01:00: percentage done: 26.56% (4 of 16 groups, 1 of 4 group snapshots)
2021-02-23T15:48:58+01:00: sync group vm/1006 failed - No such file or directory (os error 2)
2021-02-23T15:48:58+01:00: sync snapshot "vm/113/2021-02-20T03:33:23Z"
2021-02-23T15:48:59+01:00: percentage done: 32.81% (5 of 16 groups, 1 of 4 group snapshots)
2021-02-23T15:48:59+01:00: sync group vm/113 failed - No such file or directory (os error 2)

When I check the logs on the goofys S3 mount, I can see the message

Code:
s3.DEBUG HEAD vm/905/2021-02-20T03:40:40Z/index.json.tmp/ = no such file or directory


I would like to know if some of you had the experience of using the remote jobs and with S3 and if you could provide some help would be fantastic.

thanks in advance
 

Stefan_R

Proxmox Staff Member
Staff member
Jun 4, 2019
907
162
43
Vienna
I haven't looked into it too much, but just from the GitHub page of goofys (specifically this: https://github.com/kahing/goofys#current-status) I'd say it's probably a bad idea to use as backing storage for a PBS instance. Especially the atime/mtime support missing is going to cause issues with GC and such, and there's probably other incompatibilities as well...
 

alebeta

Member
Mar 20, 2018
38
3
13
30
gonkar.com
I haven't looked into it too much, but just from the GitHub page of goofys (specifically this: https://github.com/kahing/goofys#current-status) I'd say it's probably a bad idea to use as backing storage for a PBS instance. Especially the atime/mtime support missing is going to cause issues with GC and such, and there's probably other incompatibilities as well...
Hi,

thanks for the message and the useful information. Yes so far am doing some testing for remote PBS using a different kind of storages.

I tried s3fs which does not work at all for adding the data store and with goofys it does allow to add the datastore, but it fails when trying to perform the sync job.

At the moment I had been checking similar issues in goofys repository it seems a problem with the type of provider I use OVH Swift/S3

I will try to work a little bit more with goofys code and see if I can move any further.

Just to leave the info here I tried using Swift storage too with https://github.com/ovh/svfs but similar problem happens.

Do you know about another method to mount S3 and use it as datastore? If not rsync/rclone would still be the best choice at the moment to move backups to S3.

Thanks and all the best
 

Stefan_R

Proxmox Staff Member
Staff member
Jun 4, 2019
907
162
43
Vienna
Do you know about another method to mount S3 and use it as datastore? If not rsync/rclone would still be the best choice at the moment to move backups to S3.
Manual sync is what we recommend at the moment, I am not aware of a "direct mount" variant that works without issues currently. We are considering adding official support for S3 and other storage providers as well, but no implementation is currently available. See also our bugtracker for more info.
 

alebeta

Member
Mar 20, 2018
38
3
13
30
gonkar.com
Manual sync is what we recommend at the moment, I am not aware of a "direct mount" variant that works without issues currently. We are considering adding official support for S3 and other storage providers as well, but no implementation is currently available. See also our bugtracker for more info.
Hi Stefan,

thanks, yes a native S3 implementation would be good too. So that one can use the PBS remote function with it.

At the moment I will keep moving with rclone.

thanks a lot
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE and Proxmox Mail Gateway. We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get your own in 60 seconds.

Buy now!