Error creating datastore on NFS mount

G

GP123

Member
Dec 20, 2021
11
0
6
34
I'm encountering this issue when trying to add a NFS mount as a datastore after upgrading my PBS. This was working before upgrading. Any support would be greatly appreciated!



2.png



1.png


Here is my fstab entry.
Code: 
10.50.10.98:/volume1/ncBackupNAS/ncBackup/Unix/PBS  /mnt/volume1-backend nfs sec=sys,rw,anonuid=0,anongid=0,no_root_squash,no_subtree_check 0 0



Here's a snippet of file permissions on the NFS mount.
Code: 
root@pbs:~# ls -lR /mnt/
/mnt/:
total 0
drwxrwxrwx 1 backup backup 52 Sep  5 00:55 volume1-backend

/mnt/volume1-backend:
total 0
drwxrwxrwx 1 backup backup 856 May 21 16:27 ns
drwxrwxrwx 1 backup backup 168 May  5 04:06 vm

/mnt/volume1-backend/ns:
total 0
drwxrwxrwx 1 backup backup 4 May 20 12:06 pve-misc-1
drwxrwxrwx 1 backup backup 0 May 20 12:07 pve-misc-2
drwxrwxrwx 1 backup backup 0 May 20 12:07 pve-misc-3
drwxrwxrwx 1 backup backup 0 May 20 12:07 pve-misc-4
drwxrwxrwx 1 backup backup 0 May 21 16:19 voice-pool-mgt-1
drwxrwxrwx 1 backup backup 0 May 21 16:19 voice-pool-mgt-2
drwxrwxrwx 1 backup backup 0 May 21 16:26 voice-pool-pve-1x
drwxrwxrwx 1 backup backup 0 May 21 16:26 voice-pool-pve-2x
drwxrwxrwx 1 backup backup 0 May 21 16:26 voice-pool-pve-3x
drwxrwxrwx 1 backup backup 0 May 21 16:26 voice-pool-pve-4x
drwxrwxrwx 1 backup backup 0 May 21 16:26 voice-pool-pve-5x
drwxrwxrwx 1 backup backup 0 May 21 16:26 voice-pool-pve-6x
drwxrwxrwx 1 backup backup 0 May 21 16:26 voice-pool-pve-7x
drwxrwxrwx 1 backup backup 0 May 21 16:26 voice-pool-pve-8x
drwxrwxrwx 1 backup backup 0 May 21 16:19 voice-pool-trunk-1
drwxrwxrwx 1 backup backup 0 May 21 16:19 voice-pool-trunk-2

/mnt/volume1-backend/vm:
total 0
drwxrwxrwx 1 backup backup  90 Jun  2 01:00 100
drwxrwxrwx 1 backup backup 100 Jun  2 01:00 101
drwxrwxrwx 1 backup backup 100 Jun  2 01:00 102
drwxrwxrwx 1 backup backup  90 Jun  2 01:00 103
drwxrwxrwx 1 backup backup  90 Jun  2 01:00 104
drwxrwxrwx 1 backup backup 100 Jun  2 01:00 105
drwxrwxrwx 1 backup backup 330 Jun  2 01:00 106
drwxrwxrwx 1 backup backup  90 Jun  2 01:00 107
drwxrwxrwx 1 backup backup  90 Jun  2 01:00 108
drwxrwxrwx 1 backup backup  90 Jun  2 01:00 109
drwxrwxrwx 1 backup backup 130 Jun  2 01:00 110
drwxrwxrwx 1 backup backup 130 Jun  2 01:00 111
drwxrwxrwx 1 backup backup  90 Jun  2 01:00 112
drwxrwxrwx 1 backup backup  90 Jun  2 01:00 113
drwxrwxrwx 1 backup backup  90 Jun  2 01:00 114
drwxrwxrwx 1 backup backup  90 Jun  2 01:00 115
drwxrwxrwx 1 backup backup  90 Jun  2 01:00 116
drwxrwxrwx 1 backup backup  90 Jun  2 01:00 117
drwxrwxrwx 1 backup backup 130 Jun  2 01:00 118
drwxrwxrwx 1 backup backup 130 Jun  2 01:00 119
drwxrwxrwx 1 backup backup 130 Jun  2 01:00 120
drwxrwxrwx 1 backup backup 220 Jun  2 01:00 121
drwxrwxrwx 1 backup backup 130 Jun  2 01:00 122
drwxrwxrwx 1 backup backup 130 Jun  2 01:00 123
drwxrwxrwx 1 backup backup 130 Jun  2 01:00 124
drwxrwxrwx 1 backup backup 130 Jun  2 01:00 125
drwxrwxrwx 1 backup backup  50 Apr 30 18:57 130
drwxrwxrwx 1 backup backup  50 Apr 30 19:29 131

/mnt/volume1-backend/vm/100:
total 4
drwxrwxrwx 1 backup backup 142 May 19 01:24 2024-05-19T05:00:00Z
drwxrwxrwx 1 backup backup 142 May 19 05:00 2024-05-19T09:00:05Z
-rwxrwxrwx 1 backup backup  10 Apr 28 15:18 owner

'/mnt/volume1-backend/vm/100/2024-05-19T05:00:00Z':
total 2176
-rwxrwxrwx 1 backup backup    2708 May 19 01:24 client.log.blob
-rwxrwxrwx 1 backup backup 2215936 May 19 01:23 drive-scsi0.img.fidx
-rwxrwxrwx 1 backup backup     440 May 19 01:23 index.json.blob
-rwxrwxrwx 1 backup backup     322 May 19 01:00 qemu-server.conf.blob
 
Last edited:
GP123 said:
I'm encountering this issue when trying to add a NFS mount as a datastore after upgrading my PBS. This was working before upgrading. Any support would be greatly appreciated!



View attachment 74275



View attachment 74274


Here is my fstab entry.
Code: 
10.50.10.98:/volume1/ncBackupNAS/ncBackup/Unix/PBS  /mnt/volume1-backend nfs sec=sys,rw,anonuid=0,anongid=0,no_root_squash,no_subtree_check 0 0



Here's a snippet of file permissions on the NFS mount.
Code: 
root@pbs:~# ls -lR /mnt/
/mnt/:
total 0
drwxrwxrwx 1 backup backup 52 Sep  5 00:55 volume1-backend

/mnt/volume1-backend:
total 0
drwxrwxrwx 1 backup backup 856 May 21 16:27 ns
drwxrwxrwx 1 backup backup 168 May  5 04:06 vm

/mnt/volume1-backend/ns:
total 0
drwxrwxrwx 1 backup backup 4 May 20 12:06 pve-misc-1
drwxrwxrwx 1 backup backup 0 May 20 12:07 pve-misc-2
drwxrwxrwx 1 backup backup 0 May 20 12:07 pve-misc-3
drwxrwxrwx 1 backup backup 0 May 20 12:07 pve-misc-4
drwxrwxrwx 1 backup backup 0 May 21 16:19 voice-pool-mgt-1
drwxrwxrwx 1 backup backup 0 May 21 16:19 voice-pool-mgt-2
drwxrwxrwx 1 backup backup 0 May 21 16:26 voice-pool-pve-1x
drwxrwxrwx 1 backup backup 0 May 21 16:26 voice-pool-pve-2x
drwxrwxrwx 1 backup backup 0 May 21 16:26 voice-pool-pve-3x
drwxrwxrwx 1 backup backup 0 May 21 16:26 voice-pool-pve-4x
drwxrwxrwx 1 backup backup 0 May 21 16:26 voice-pool-pve-5x
drwxrwxrwx 1 backup backup 0 May 21 16:26 voice-pool-pve-6x
drwxrwxrwx 1 backup backup 0 May 21 16:26 voice-pool-pve-7x
drwxrwxrwx 1 backup backup 0 May 21 16:26 voice-pool-pve-8x
drwxrwxrwx 1 backup backup 0 May 21 16:19 voice-pool-trunk-1
drwxrwxrwx 1 backup backup 0 May 21 16:19 voice-pool-trunk-2

/mnt/volume1-backend/vm:
total 0
drwxrwxrwx 1 backup backup  90 Jun  2 01:00 100
drwxrwxrwx 1 backup backup 100 Jun  2 01:00 101
drwxrwxrwx 1 backup backup 100 Jun  2 01:00 102
drwxrwxrwx 1 backup backup  90 Jun  2 01:00 103
drwxrwxrwx 1 backup backup  90 Jun  2 01:00 104
drwxrwxrwx 1 backup backup 100 Jun  2 01:00 105
drwxrwxrwx 1 backup backup 330 Jun  2 01:00 106
drwxrwxrwx 1 backup backup  90 Jun  2 01:00 107
drwxrwxrwx 1 backup backup  90 Jun  2 01:00 108
drwxrwxrwx 1 backup backup  90 Jun  2 01:00 109
drwxrwxrwx 1 backup backup 130 Jun  2 01:00 110
drwxrwxrwx 1 backup backup 130 Jun  2 01:00 111
drwxrwxrwx 1 backup backup  90 Jun  2 01:00 112
drwxrwxrwx 1 backup backup  90 Jun  2 01:00 113
drwxrwxrwx 1 backup backup  90 Jun  2 01:00 114
drwxrwxrwx 1 backup backup  90 Jun  2 01:00 115
drwxrwxrwx 1 backup backup  90 Jun  2 01:00 116
drwxrwxrwx 1 backup backup  90 Jun  2 01:00 117
drwxrwxrwx 1 backup backup 130 Jun  2 01:00 118
drwxrwxrwx 1 backup backup 130 Jun  2 01:00 119
drwxrwxrwx 1 backup backup 130 Jun  2 01:00 120
drwxrwxrwx 1 backup backup 220 Jun  2 01:00 121
drwxrwxrwx 1 backup backup 130 Jun  2 01:00 122
drwxrwxrwx 1 backup backup 130 Jun  2 01:00 123
drwxrwxrwx 1 backup backup 130 Jun  2 01:00 124
drwxrwxrwx 1 backup backup 130 Jun  2 01:00 125
drwxrwxrwx 1 backup backup  50 Apr 30 18:57 130
drwxrwxrwx 1 backup backup  50 Apr 30 19:29 131

/mnt/volume1-backend/vm/100:
total 4
drwxrwxrwx 1 backup backup 142 May 19 01:24 2024-05-19T05:00:00Z
drwxrwxrwx 1 backup backup 142 May 19 05:00 2024-05-19T09:00:05Z
-rwxrwxrwx 1 backup backup  10 Apr 28 15:18 owner

'/mnt/volume1-backend/vm/100/2024-05-19T05:00:00Z':
total 2176
-rwxrwxrwx 1 backup backup    2708 May 19 01:24 client.log.blob
-rwxrwxrwx 1 backup backup 2215936 May 19 01:23 drive-scsi0.img.fidx
-rwxrwxrwx 1 backup backup     440 May 19 01:23 index.json.blob
-rwxrwxrwx 1 backup backup     322 May 19 01:00 qemu-server.conf.blob
Click to expand...
Hi,
so it seems you are trying to create a new datastore over the already existing one? That will not work. To readd the existing datastore to the instance, it is best to readd the datastore via the config in /etc/proxmox-backup/datastore.cfg
 
Hi everyone, I'm new here.
My PBS 2.2 (hosted on ESXi) with NFS datastores on qnap nas has an issue, after a blackout my datastores where ureachable so i tried to restart the nfs service on the nas but nothing changed, so i tried to mount them with the 'mount' command but the shell's cursor freezes (only the shell not the pbs). So i rebooted the system and the command 'mount' worked but still something is wrong because if i click on the datastore tab in the pbs gui it gives the information on screen (usage history, ecc...) but if i click on the summary or content tabs i can't see nothing and i get the these errors:
Content Tab "Bad Request (400) Eacces Permission denied"
Summary Tab "Datastore is not available" and i can see only the storage usage.
Another thing My PVE Gives me an erro when finishes the backup job

PBS.png

Also I checked the foldr permission of the datastore folder everything seems ok there.

PBS2.png

Please can someone explain me what's going wrong?
Thanks
 
g.migliaccio said:
Hi everyone, I'm new here.
My PBS 2.2 (hosted on ESXi) with NFS datastores on qnap nas has an issue, after a blackout my datastores where ureachable so i tried to restart the nfs service on the nas but nothing changed, so i tried to mount them with the 'mount' command but the shell's cursor freezes (only the shell not the pbs). So i rebooted the system and the command 'mount' worked but still something is wrong because if i click on the datastore tab in the pbs gui it gives the information on screen (usage history, ecc...) but if i click on the summary or content tabs i can't see nothing and i get the these errors:
Content Tab "Bad Request (400) Eacces Permission denied"
Summary Tab "Datastore is not available" and i can see only the storage usage.
Another thing My PVE Gives me an erro when finishes the backup job

View attachment 74298

Also I checked the foldr permission of the datastore folder everything seems ok there.

View attachment 74300

Please can someone explain me what's going wrong?
Thanks
Click to expand...
Hi,
as first step please try to restart the pbs services systemctl restart proxmox-backup.service proxmox-backup-proxy.service as I assume the nfs was mounted after the services were already started. Also, check your systemd journal on the PBS for errors, maybe that gives a further hint on what is going wrong. Is the NFS share mounted on the expected mountpoint? Doublecheck with your datastore config, which you can get via cat /etc/proxmox-backup/datastore.cfg

Furhter, consider upgrading to the latest version of PBS, 2.2 is already EOL since quite some time.
 
Hi Chris, thank you for the reply.
here is my journalctl:
Sep 05 16:14:02 pbs systemd-logind[623]: New session 7 of user root.
Sep 05 16:14:02 pbs systemd[1]: Started Session 7 of user root.
Sep 05 16:14:02 pbs login[1086]: pam_unix(login:session): session opened for user root(uid=0) by root(uid=0)
Sep 05 16:13:59 pbs proxmox-backup-proxy[1039]: GET /api2/json/admin/datastore/NFS_Backup_P/status/?verbose=true: 400 Bad Request: [client [::ffff:192.168.208.122]:51266] EACCES: Pe>
Sep 05 16:13:58 pbs proxmox-backup-proxy[1039]: GET /api2/json/admin/datastore/NFS_Backup_P/snapshots: 400 Bad Request: [client [::ffff:192.168.208.122]:51266] EACCES: Permission de>
Sep 05 16:13:57 pbs proxmox-backup-proxy[1039]: GET /api2/json/admin/datastore/NFS_Backup_P/status/?verbose=true: 400 Bad Request: [client [::ffff:192.168.208.122]:51266] EACCES: Pe>
Sep 05 16:13:55 pbs proxmox-backup-proxy[1039]: GET /api2/json/admin/datastore/NFS_Backups/status/?verbose=true: 400 Bad Request: [client [::ffff:192.168.208.122]:51265] EACCES: Permissio>
Sep 05 16:13:52 pbs proxmox-backup-proxy[1039]: GET /api2/json/admin/datastore/NFS_Backup_P/status/?verbose=true: 400 Bad Request: [client [::ffff:192.168.208.122]:51262] EACCES: Pe>
Sep 05 16:13:52 pbs systemd-logind[623]: Removed session 5.
Sep 05 16:13:52 pbs systemd[1]: session-5.scope: Succeeded.
Sep 05 16:13:52 pbs systemd-logind[623]: Session 5 logged out. Waiting for processes to exit.
Sep 05 16:12:04 pbs login[1075]: ROOT LOGIN on '/dev/pts/0'
Sep 05 16:12:04 pbs systemd[1]: Started Session 5 of user root.
Sep 05 16:12:04 pbs systemd[1061]: Startup finished in 85ms.
Click to expand...
I also restarted the proxy and backup service as you suggested but nothing changed and checked the datastore.cfg file that seems to be ok

PBS4.png
 
/api2/json/admin/datastore/NFS_Backup_P
Click to expand...
So it seems like your client is trying the other datastore, not the one you showed. Check that datastores moutpoint for issues.
 
I checked them...
GNU nano 5.4 /etc/fstab *
# <file system> <mount point> <type> <options> <dump> <pass>
/dev/pbs/root / ext4 errors=remount-ro 0 1
/dev/pbs/swap none swap sw 0 0
proc /proc proc defaults 0 0
192.168.208.4:/ShareVM /var/nfs_backups nfs defaults 0 0
192.168.208.4:/nfs_backup_pointel /var/nfs_backup_p nfs defaults 0 0
Click to expand...
Really i can't understand this issue
 
Chris said:
So it seems like your client is trying the other datastore, not the one you showed. Check that datastores moutpoint for issues.
Click to expand...
Both of them same exactly same issue. You can find both in the journal transcription tht I posted.
 
Last edited:
Hi Chris,
the task folder is correctly owned by 'backup' but i'll check for the '.chunk' folder
 
Last edited:
The '.chunks' folder has permission and the right owner:

Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
Last login: Fri Sep 6 12:14:47 CEST 2024 on pts/0
You have new mail.
root@pbs:~# cd /var/nfs_backups/
root@pbs:/var/nfs_backups# ls -la
total 16616
drwxrwxrwx 6 backup backup 9 Sep 5 11:55 .
drwxr-xr-x 13 root root 4096 Sep 4 17:01 ..
drwxrwxrwx 65538 backup backup 65538 Jun 27 2022 .chunks
drwxrwxrwx 3 backup backup 3 Apr 13 2023 ct
-rwxrwxrwx 1 backup backup 332 Dec 21 2023 .gc-status
-rwxrwxrwx 1 backup backup 0 Sep 5 11:55 .lock
drwxrwxrwx 2 backup backup 2 Oct 28 2022 ns
-rwxrwxrwx 1 root root 5 Jun 27 2022 test
drwxrwxrwx 19 backup backup 19 Aug 30 13:02 vm
Click to expand...
 
What does proxmox-backup-debug api get /admin/datastore/NFS_Backup_P/status return when executed in the PBS cli as root user?
 
Hope this helps...

root@pbs:~# proxmox-backup-debug api get /admin/datastore/NFS_Backup_P/status
┌───────┬───────────────┐
│ Name │ Value │
╞═══════╪═══════════════╡
│ avail │ 527710552064 │
├───────┼───────────────┤
│ total │ 1041162567680 │
├───────┼───────────────┤
│ used │ 513452015616 │
└───────┴───────────────┘
root@pbs:~# proxmox-backup-debug api get /admin/datastore/NFS_Backups/status
┌───────┬───────────────┐
│ Name │ Value │
╞═══════╪═══════════════╡
│ avail │ 483043311616 │
├───────┼───────────────┤
│ total │ 2199023255552 │
├───────┼───────────────┤
│ used │ 1715979943936 │
└───────┴───────────────┘
Click to expand...
 
Please post also the output of find /var/nfs_backup_p ! -user backup, I do suspect one of the snapshot folders not having the correct ownership, as the backup seems to fail when trying to purge old backups, and the verbose status does iterate over the snapshots to count them as well.
 
Hi Chris,
I tried the find /var/nfs_backup_p ! -user backup command but it dosen't work in details my cursor stays and nothing happends for more than an hour.
 
g.migliaccio said:
Hi Chris,
I tried the find /var/nfs_backup_p ! -user backup command but it dosen't work in details my cursor stays and nothing happends for more than an hour.
Click to expand...
As this goes trough all the files on the datastore, this might take some time, depending on how many files the datastore contains. But it will list you all files not currently owned by the backup user for that datastore. So please let it run till completion.
 
Good morning to everyone,
so I send the command from the nas side
@NASQNAP886 ~]$ sudo find /share/nfs_backup_p ! -user 34
this is the response : /share/nfs_backup_p
user 34 is the backup user
 
Last edited:
g.migliaccio said:
this is the response : /share/nfs_backup_pointel
Click to expand...
Ahm, this seems strange to me, because the find command should only look into /share/nfs_backup_p. Do you have some symlinks pointing to different folder? Something seems off...

What is the output if you run the same on the other share, backing /var/backups as seen by the PBS instance?
 
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom
Back