Connecting GCS/Google Cloud Storage to PBS!

YaseenKamala · Jul 15, 2022

Hello everybody,

Hope you all are doing well,

Here is my actual Backup infrastructure

I was wandering if it's possible to connect GCS/Google cloud storage to PBS to allow me to restore VMs if NAS fails?

Kind regards,
Kamala

oguz · Jul 18, 2022

hi,

you could use rclone [0] to mount your google cloud on your PBS, and afterwards create a datastore in the mounted folder

[0]: https://rclone.org/googlecloudstorage/

YaseenKamala · Jul 21, 2022

@oguz thanks for your help.

I have create a remote as you can see in the bellow photo:

but I am not sure of what I am trying to use if it's the correct way of mounting, could you please help me on that.

Code:

rclone mount remote:path/to/files /path/to/local/mount

see this link

I was able to create a datastore as well, however I don't know what is this error exactly is about:

Code:

2022/07/21 17:53:29 ERROR : .lock: WriteFileHandle: Can't open for write without O_TRUNC on existing file without --vfs-cache-mode >= writes

But how can I be sure that the data store it's really uses google cloud storage, can we check that? and I was wandering if it can be automated like what we do with fstab/crontab, because when I close the SSH session I got an error saying "unable to open chunck ... "

YaseenKamala · Jul 26, 2022

@oguz

I created a google cloud storage remote type as you can see bellow:
Name Type
==== ====
GCPbucket google cloud storage

when I try to mount a gcp bucket to PBS with the bellow command
rclone mount GCPbucket:"project_id"/my-bucket/TestOnly /mnt/GcpInfraStorage/

it looks like it worked. however when I go to PBS /GUI to create a datastore using local mount point as a backing up path which is pointed to the GCP bucket, I get the bellow error.
Bad Request (400)
unable to open chunk store 'GCPBucket' at "/mnt/GcpInfraStorage/.chunks" - No such file or directory (os error 2)

Do you know how can we check that the mount command worked?

I am even able to list what I have within the "TestOnly" floder at the PBS level.

Could you please help me on that?

Kind regards,

oguz · Jul 27, 2022

YaseenKamala said:
Do you know how can we check that the mount command worked?

I am even able to list what I have within the "TestOnly" floder at the PBS level.

if you go to the mountpoint like cd /mnt/GcpInfraStorage and do ls -al you should see some files from your storage there.
if it's empty then most likely it's not mounted.

YaseenKamala · Jul 27, 2022

It's mounted but I can't see what I have within the local mount point at the GCP level. see what I have locally.

oguz · Jul 27, 2022

if you do this:

Code:

cd /mnt/GcpInfraStorage
touch test.txt

do you see the test.txt file on google cloud?

YaseenKamala · Jul 27, 2022

@oguz until now I was able to create a datastore from the mounted point, however it's about 1h I have this error message.

I have checked /etc/proxmox-backup/datastore.cfg I have only one datastor there

YaseenKamala · Jul 27, 2022

oguz said:
if you do this:

Code:

cd /mnt/GcpInfraStorage touch test.txt

do you see the test.txt file on google cloud?

No I don't see it

oguz · Jul 27, 2022

YaseenKamala said:
No I don't see it

sorry, it should have been cd /mnt/GCPStorage (where your mountpoint is)

YaseenKamala · Jul 27, 2022

no worries I got the point see bellow

nothing at Bucket level:

YaseenKamala · Jul 27, 2022

@oguz you see I am even able to create a datastore which point to that mount point:

YaseenKamala · Jul 27, 2022

@oguz I got this error when tried to backup a vm:
2022/07/27 15:54:41 ERROR : .chunks/5bf4/5bf4490306a6f46befefcf4b2342c3f64457a11e2071f51caa5bbefba0efc6d6: Failed to copy: googleapi: Error 400: Cannot insert legacy ACL for an object when uniform bucket-level access is enabled. Read more at https://cloud.google.com/storage/docs/uniform-bucket-level-access,

YaseenKamala · Jul 29, 2022

I am not able to create a Datastore this is the error I got:

unable to create chunk store 'GCS-Bucket' subdir "/mnt/GCPStorage/.chunks" - EIO: I/O error,

Here is the mount point that I am trying to use:

friendodevil · Aug 6, 2022

@YaseenKamala @oguz

I'm also unable to mount local drive on PBS with Rclone:

root@pbs:~# lsblk
NAME         MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
sda            8:0    0   32G  0 disk 
├─sda1         8:1    0 1007K  0 part 
├─sda2         8:2    0  512M  0 part 
└─sda3         8:3    0 31.5G  0 part 
  ├─pbs-swap 253:0    0  3.9G  0 lvm  [SWAP]
  └─pbs-root 253:1    0 23.8G  0 lvm  /
sdb            8:16   0  500G  0 disk 
└─sdb1         8:17   0  500G  0 part /mnt/datastore/pve-rclone-gdrive

root@pbs:~# rclone mount pve: /mnt/datastore/pve-rclone-gdrive --allow-non-empty 
2022/08/06 19:33:01 Fatal error: failed to mount FUSE fs: fusermount: exec: "fusermount": executable file not found in $PATH

oguz · Aug 8, 2022

friendodevil said:
root@pbs:~# rclone mount pve: /mnt/datastore/pve-rclone-gdrive --allow-non-empty 2022/08/06 19:33:01 Fatal error: failed to mount FUSE fs: fusermount: exec: "fusermount": executable file not found in $PATH

you'll need to install fuse3

friendodevil · Aug 9, 2022

oguz said:
you'll need to install fuse3

Works now - Thank you!

YaseenKamala · Aug 11, 2022

@oguz I gave up with rclone

, I managed with

Code:

gcsfuse

I am able to read and write into the bucket from PBS level with command line.
However I am not able to create a datastore I am getting the bellow error:

Code:

Bad Request (400)
unable to open chunk store 'GCSBucket' at "/mnt/GCSBucket/.chunks" - Permission denied (os error 13)

When I check the rights for the mount point it looks like some rights are missing compared to the Local NAS mounted point as you can see in the below screenshot:

and I am not understanding why rights are changing when I make GCS to one of the mount points. on the screenshot above "GSPStorage" has all the rights and the GCP Bucket currently is mounted to "/mnt/GCSBucket", when I change the mount point the rights are jumping by themself see the screenshot below:

YaseenKamala · Aug 16, 2022

Hello,
Could you please help me why I am getting this error:

Code:

Bad Request (400)
unable to open chunk store 'GCSBucket' at "/mnt/GCSBucket/.chunks" - Permission denied (os error 13)

Here is how I have mounted the bucket to PBS:

Code:

gcsfuse --debug_fs --debug_gcs --debug_fuse --debug_http --log-file=/mnt/output.json --log-format=json --only-dir=GCSBucket my-bucket /mnt/GCSBucket

I am able to read and write into the mounted point without any problem see bello details:

I can see the newly created folder at within the bucket at the cloud level:

I wander to know why I am not able to create a datastore which points to that mount point, cloud you please help me on that?
Thanks for your help

fabian · Aug 17, 2022

PBS requires proper file ownership and flock support on the datastore directory tree..

Connecting GCS/Google Cloud Storage to PBS!

Member

Attachments

Proxmox Retired Staff

Member

Member

Proxmox Retired Staff

Member

Proxmox Retired Staff

Member

Member

Proxmox Retired Staff

Member

Member

Member

Member

Member

Proxmox Retired Staff

Member

Member

Member

Proxmox Staff Member