Connecting GCS/Google Cloud Storage to PBS!

Y

YaseenKamala

Member
Feb 2, 2021
158
5
23
Paris
Hello everybody,

Hope you all are doing well,

Here is my actual Backup infrastructure
1657878518555.png

I was wandering if it's possible to connect GCS/Google cloud storage to PBS to allow me to restore VMs if NAS fails?

Kind regards,
Kamala
 

Attachments

  • 1657878362272.png
    1657878362272.png
    56.7 KB · Views: 9
Last edited:
@oguz thanks for your help.

I have create a remote as you can see in the bellow photo:
1658417455271.png

but I am not sure of what I am trying to use if it's the correct way of mounting, could you please help me on that.

Code: 
rclone mount remote:path/to/files /path/to/local/mount
see this link

1658419276968.png

I was able to create a datastore as well, however I don't know what is this error exactly is about:
Code: 
2022/07/21 17:53:29 ERROR : .lock: WriteFileHandle: Can't open for write without O_TRUNC on existing file without --vfs-cache-mode >= writes



1658419340093.png


But how can I be sure that the data store it's really uses google cloud storage, can we check that? and I was wandering if it can be automated like what we do with fstab/crontab, because when I close the SSH session I got an error saying "unable to open chunck ... "
 
Last edited:
@oguz

I created a google cloud storage remote type as you can see bellow:
Name Type
==== ====
GCPbucket google cloud storage

when I try to mount a gcp bucket to PBS with the bellow command
rclone mount GCPbucket:"project_id"/my-bucket/TestOnly /mnt/GcpInfraStorage/

it looks like it worked. however when I go to PBS /GUI to create a datastore using local mount point as a backing up path which is pointed to the GCP bucket, I get the bellow error.
Bad Request (400)
unable to open chunk store 'GCPBucket' at "/mnt/GcpInfraStorage/.chunks" - No such file or directory (os error 2)

Do you know how can we check that the mount command worked?

I am even able to list what I have within the "TestOnly" floder at the PBS level.

1658842607513.png

Could you please help me on that?

Kind regards,
 
Last edited:
YaseenKamala said:
Do you know how can we check that the mount command worked?

I am even able to list what I have within the "TestOnly" floder at the PBS level.
Click to expand...
if you go to the mountpoint like cd /mnt/GcpInfraStorage and do ls -al you should see some files from your storage there.
if it's empty then most likely it's not mounted.
 
if you do this:
Code: 
cd /mnt/GcpInfraStorage
touch test.txt

do you see the test.txt file on google cloud?
 
@oguz until now I was able to create a datastore from the mounted point, however it's about 1h I have this error message.
1658926293765.png

I have checked /etc/proxmox-backup/datastore.cfg I have only one datastor there
1658926472944.png
 
Last edited:
I am not able to create a Datastore this is the error I got:
1659089109013.png

unable to create chunk store 'GCS-Bucket' subdir "/mnt/GCPStorage/.chunks" - EIO: I/O error,


Here is the mount point that I am trying to use:

1659089087409.png
 
@YaseenKamala @oguz

I'm also unable to mount local drive on PBS with Rclone:

root@pbs:~# lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT sda 8:0 0 32G 0 disk ├─sda1 8:1 0 1007K 0 part ├─sda2 8:2 0 512M 0 part └─sda3 8:3 0 31.5G 0 part ├─pbs-swap 253:0 0 3.9G 0 lvm [SWAP] └─pbs-root 253:1 0 23.8G 0 lvm / sdb 8:16 0 500G 0 disk └─sdb1 8:17 0 500G 0 part /mnt/datastore/pve-rclone-gdrive

root@pbs:~# rclone mount pve: /mnt/datastore/pve-rclone-gdrive --allow-non-empty 2022/08/06 19:33:01 Fatal error: failed to mount FUSE fs: fusermount: exec: "fusermount": executable file not found in $PATH
 
friendodevil said:
root@pbs:~# rclone mount pve: /mnt/datastore/pve-rclone-gdrive --allow-non-empty 2022/08/06 19:33:01 Fatal error: failed to mount FUSE fs: fusermount: exec: "fusermount": executable file not found in $PATH
Click to expand...
you'll need to install fuse3
 
@oguz I gave up with rclone :), I managed with
Code: 
gcsfuse
I am able to read and write into the bucket from PBS level with command line.
However I am not able to create a datastore I am getting the bellow error:
Code: 
Bad Request (400)
unable to open chunk store 'GCSBucket' at "/mnt/GCSBucket/.chunks" - Permission denied (os error 13)
When I check the rights for the mount point it looks like some rights are missing compared to the Local NAS mounted point as you can see in the below screenshot:

1660296610219.png

and I am not understanding why rights are changing when I make GCS to one of the mount points. on the screenshot above "GSPStorage" has all the rights and the GCP Bucket currently is mounted to "/mnt/GCSBucket", when I change the mount point the rights are jumping by themself see the screenshot below:
1660297571134.png
 
Last edited:
Hello,
Could you please help me why I am getting this error:
Code: 
Bad Request (400)
unable to open chunk store 'GCSBucket' at "/mnt/GCSBucket/.chunks" - Permission denied (os error 13)


Here is how I have mounted the bucket to PBS:
Code: 
gcsfuse --debug_fs --debug_gcs --debug_fuse --debug_http --log-file=/mnt/output.json --log-format=json --only-dir=GCSBucket my-bucket /mnt/GCSBucket


I am able to read and write into the mounted point without any problem see bello details:
1660639215198.png

I can see the newly created folder at within the bucket at the cloud level:
1660639317222.png

I wander to know why I am not able to create a datastore which points to that mount point, cloud you please help me on that?
Thanks for your help
 
Last edited:
PBS requires proper file ownership and flock support on the datastore directory tree..
 
  • Like
Reactions: YaseenKamala
You must log in or register to reply here.
Top Bottom