Proxmox Backup Server creating datastore directories

[Sidhana]

Hello everyone,

I have 96 Terrabyte disks capacity configured as hardware RAID 6.
I need to create different datastores for different costumers, how can I do that?
Should I create directory or ZFS? Right now I can create only one directory. How can I create different directories or ZFS for each customer/user?
I need exact commands if possible.

 

[Dunuin]

ZFS shouldn't be used with HW raid.
Just create a directory per datastore using the CLI (mkdir command) on the filesystem of your HW raid array and than add a datastore pointing to that directory.

PBS2.2 also added namespaces. So you could also just use a single datastore and create a namespace per customer.

But I think a datastore per customer might be better. I guess the deduplication benefit between customers wouldn't be that great and using multiple datastores would allow you to give the customer access to his backups in case the customer also wants a offline backup (making the datastore folder accessible by some protocol like ftp, webdav, smb or similar) or move the backups to another hoster.

 

[Sidhana]

Thanks Dunuin for your reply.
I am just getting an unknown error while creating a directory. Do you have any idea?

 

[Sidhana]

ZFS shouldn't be used with HW raid.
Just create a directory per datastore using the CLI (mkdir command) on the filesystem of your HW raid array and than add a datastore pointing to that directory.

PBS2.2 also added namespaces. So you could also just use a single datastore and create a namespace per customer.

But I think a datastore per customer might be better. I guess the deduplication benefit between customers wouldn't be that great and using multiple datastores would allow you to give the customer access to his backups in case the customer also wants a offline backup (making the datastore folder accessible by some protocol like ftp, webdav, smb or similar) or move the backups to another hoster.

I have now created diffrent ZFS which automaticall created datastores and I want to create different paths/directories under them for each costumer so they can access only their backups when needed.
How can I do that?

 

[Dunuin]

For easier management I would create a dedicated ZFS dataset for each PBS datastore, as each dataset is its own filesystem.

 

[Sidhana]

For easier management I would create a dedicated ZFS dataset for each PBS datastore, as each dataset is its own filesystem.

So you mean I should create different ZFS RAIDs which automatically creates a new datastores?
But the problem is I have to assign at least 16TB disk/datastore for each costumer created minimum as a RAID1 as I have to be sure the data doesn't gets lost on a disk failure. (I have 16TB 30 disks avaliable)

Can I not create different directories under those datastores?

I tried creating namespace but then while adding the backup server to the cluster I have to add the whole datastore and cannot add only the namespace

 

[Dunuin]

So you mean I should create different ZFS RAIDs which automatically creates a new datastores?
But the problem is I have to assign at least 16TB disk/datastore for each costumer created minimum as a RAID1 as I have to be sure the data doesn't gets lost on a disk failure. (I have 16TB 30 disks avaliable)
Can I not create different directories under those datastores?

You don't need to create different pools. You can create as much datasets as you like per pool. Like zfs create MyPool/Customer1Datastore, zfs create MyPool/Customer2Datastore and so on. And then point your datastore to "/MyPool/Customer1Datastore", "/MyPool/Customer2Datastore", ...
Creating just a folder on your pool is another thing. You could also create a folder on your pools root named "Customer1Datastore" or "Customer2Datastore" and it would look similar (even the same paths "/MyPool/Customer1Datastore", "/MyPool/Customer2Datastore") but the difference is that each dataset is its own filesystem. Each datasets can have its own configs and you can only snapshot/replicate whole datasets.
If you just got one dataset with folders in it for each customer you can only snapshot, replicate or configure (for example quotas) the whole dataset. So it would not be possible to rollback the datastore of a single customer, you can't decide that you want block level compression for one customer but not for another customer and so on. So from a management view it is better have one dataset per datastore.

I tried creating namespace but then while adding the backup server to the cluster I have to add the whole datastore and cannot add only the namespace

You can add PBS with namespaces to your PVE cluster. But you need PVE 7.2 for that.

 

[Neobin]

Sorry to say that, but reading this thread, you should really read, learn and understand about the basics and the management of storage and filesystems in general and specific of ZFS (if you want to go that route and already dropped your hardware raid and flashed your raid controller to IT-mode, if even possible), before you put such an setup in business production and provide it to others.
At the moment this all sounds not really trustworthy, sorry.

Just my 2 cents.

 

[Sidhana]

Sorry to say that, but reading this thread, you should really read, learn and understand about the basics and the management of storage and filesystems in general and specific of ZFS (if you want to go that route and already dropped your hardware raid and flashed your raid controller to IT-mode, if even possible), before you put such an setup in business production and provide it to others.
At the moment this all sounds not really trustworthy, sorry.

Just my 2 cents.

We are currently in the learning phase and not provinding any service to anyone until we have learned and tested out everything.
So chill out