Connecting GCS/Google Cloud Storage to PBS!

[YaseenKamala]

@fabian thanks,
I managed to create the datastore by adding --file-mode and --dir-mode to the mount command. right now I am having another issue which is:

ERROR: VM 10016 qmp command 'backup' failed - backup register image failed: command error: update atime failed for chunk/file "/mnt/GCPStorage/.chunks/bb9f/bb9f8df61474d25e71fa00722318cd387396ca1736605e1248821cc0de3d3af8" - EPERM: Operation not permitted
INFO: aborting backup job
INFO: resuming VM again
ERROR: Backup of VM 10016 failed - VM 10016 qmp command 'backup' failed - backup register image failed: command error: update atime failed for chunk/file "/mnt/GCPStorage/.chunks/bb9f/bb9f8df61474d25e71fa00722318cd387396ca1736605e1248821cc0de3d3af8" - EPERM: Operation not permitted
INFO: Failed at 2022-08-17 12:43:55
INFO: Backup job finished with errors

 

[fabian]

it seems like setting atime doesn't work (which is a requirement for PBS to work). even if you manage to twist all the knobs you likely won't get anything approaching good performance out of such a setup either..

 

[YaseenKamala]

it seems like setting atime doesn't work (which is a requirement for PBS to work). even if you manage to twist all the knobs you likely won't get anything approaching good performance out of such a setup either..

in that case what is your recommendation? I have a local copy but I would to have a copy in the cloud as well, is there another way of doing it?

 

[fabian]

if you have a local copy, you can just rsync the whole datastore. if you want to be extra careful, schedule your rsync runs to not run parallel to GC and rsync the .chunks dir last to reduce the chance of encountering inconsistencies. ensure that all file and directory names are synced correctly, that empty and hidden directories are included, and that the permissions and ownership are correctly preserved.

 

[YaseenKamala]

if you have a local copy, you can just rsync the whole datastore. if you want to be extra careful, schedule your rsync runs to not run parallel to GC and rsync the .chunks dir last to reduce the chance of encountering inconsistencies. ensure that all file and directory names are synced correctly, that empty and hidden directories are included, and that the permissions and ownership are correctly preserved.

Yes, but my local one is already a mounted storage over the network, with rsync, if something is deleted locally will it be delete at the GCS level as well?

 

[fabian]

depends on the flags you pass to rsync..

 

[YaseenKamala]

depends on the flags you pass to rsync..

I see thanks a lot for your help @fabian, Jus to let you know the reason I wanted to do that. I said if one day I have a problem with Synology NAS which is used for the local backups, I still have another backup happening beside of it.
if I backup my actual datastore to cloud with rsync , what if is NAS fails? then the rsync is useless again, don't you think so? my goal is to avoid that!

I WANT TO HAVE TWO BACKUP DESTINATIONS IN PARALLEL

1. NODES -> PBS -> Synology NAS
2. NODES -> PBS -> GCSBucket

 

[fabian]

that's not possible, as GCS doesn't support everything PBS needs to store a datastore.. you can setup a second PBS somewhere else and use the built-in sync if you want to have an offsite backup for disaster recovery purposes..

 

[YaseenKamala]

that's not possible, as GCS doesn't support everything PBS needs to store a datastore.. you can setup a second PBS somewhere else and use the built-in sync if you want to have an offsite backup for disaster recovery purposes..

What about Azure or AWS do you know if they are supporting all what PBS needs? I'll have a look of having secondary PBS somewhere else if possible.

 

[fabian]

most cloud storage providers are rather limited in which features are supported.

 

[YaseenKamala]

most cloud storage providers are rather limited in which features are supported.

SAD, It would a great to make this work I think I hope one day soon.

 

[YaseenKamala]

Hello @oguz
Do you think If I go back to rclone, I will face same problem? or it's something you have already tested?

 

[fabian]

I doubt that will improve things (either on the compat or on the performance side).

 

[YaseenKamala]

I doubt that will improve things (either on the compat or on the performance side).

I wanted to give a try since @oguz recommended to see at least where it goes. but I am having some error as before with rclone, when I try to create the datastore which is:

unable to create chunk store 'rclonetest' subdir "/mnt/GCSBucket/.chunks" - EIO: I/O error

he told me that I should read or check https://cloud.google.com/storage/docs/uniform-bucket-level-access, but I don't know what to change exactly

 

[fabian]

what you want is something like https://bugzilla.proxmox.com/show_bug.cgi?id=3169 , but that doesn't exist (yet). mounting some remote, high-latency, weird-semantics cloud storage and pretending it's a local file system will only cause a world of hurt, believe me.

 

[YaseenKamala]

what you want is something like https://bugzilla.proxmox.com/show_bug.cgi?id=3169 , but that doesn't exist (yet). mounting some remote, high-latency, weird-semantics cloud storage and pretending it's a local file system will only cause a world of hurt, believe me.

@fabian I believe you but sometime it's good to go to up to the end, what about of using a solution like OVH or backblaze services?

 

[fabian]

the OVH one might work with some tinkering (users are using NFS/CIFS here on the forum, but performance is pretty hit and miss, although that might not be as important for the "sync-only" use case). AFAICT backblaze doesn't offer regular filesystem-like access at all, and given that it's not nearby storage like for the OVH server -> OVH NAS case, I doubt performance will be good either.

 

[bearded-donkey]

PBS requires proper file ownership and flock support on the datastore directory tree..

By your above comment, are you saying PBS doesn't support Google Drive via rclone? Or are you saying it's a config issue that can be worked around?

Thanks

 

[fabian]

it's at least a bad idea from a performance standpoint. I am not sure whether it supports all required operations and file attributes, I am not a google drive user

 

[TECNI.sistemas]

Hola
En nuestro caso con el siguiente script:
root@TECNI82:~# cat /CONFIG/backup.sh #!/bin/bash HOST=`cat /etc/hostname` echo `date +%F\ %T` Inciando copia a Google Drive rclone sync -l /BKS drive:/BKS/$HOST echo `date +%F\ %T` Fin de la copia a Google Drive



sincronizamos los archivos de disco PBS Local BKS y lo cargamos en otro disco BKS_drive que creamos montando lo de la siguiente forma:
rclone mount --daemon --allow-other --allow-root --uid 34 --gid 34 drive:/BKS/TECNI82 /media/drive

el uid y gid perteneces al usuario con permisos backup:
root@TECNI82:~# id -u backup 34 root@TECNI82:~# id -g backup 34

Pero, cuando se trata de verificar la información no funciona, siento que falta algo mas, no se si alguien ya supo como usar Google Drive con PBS


Muchas gracias en lo que nos puedan ayudar