[SOLVED] Max number of subfolders reached in .chunks/ folder

[Taledo]

Hey all,

I'm trying to use an old Dell VNXE Storage as a sync target for cold storage out of site backups copy.

The datastore creation fails with an IO Output error. As it turns out, it seems I cannot create more than 65533 folders :

root@definitelynotwhereiam-pbs2:/vnxe/.chunks# mkdir foo
mkdir: cannot create directory ‘foo’: Input/output error
root@definitelynotwhereiam-pbs2:/vnxe/.chunks# ls -lah | wc -l
65536

My fstab is as follow :

vnxe_ip:/backups /vnxe nfs defaults,proto=tcp,nfsvers=3 0 0

This seems to be a hard limit set by Dell for some obscure reasons : https://www.dell.com/community/VNX/...arent-directory-hard-limit-65533/td-p/6893847

Are there any workarounds that would allow me to create a datastore on this old piece of equipment? It's still 14Tb of available disk space, and I'd rather use it that way than bin it.

Cheers

 

[dcsapak]

no, the way the chunk folder is organized is pretty deeply integrated and there is no way to change it

 

[Taledo]

Alright, well, that's a shame. Guess I'll have to find another use for the VNXE.

Thanks for the reply.

 

[mow]

Is file size limited? Could you create a usable filesystem with a loop mount?

 

[Taledo]

As I understand it, there's no way to create more than 65535 subfolders in a folder. File size doesn't matter.

As for the loop mount, I've honestly got no idea, but seeing as how I get an IO error when manually creating a folder, I do not think the VNXE would allow it.

 

[mow]

As I understand it, there's no way to create more than 65535 subfolders in a folder. File size doesn't matter.
As for the loop mount, I've honestly got no idea, but seeing as how I get an IO error when manually creating a folder, I do not think the VNXE would allow it.

as long as it allows to create a file several TB in size that you create a filesystem in, and it is reliably accessible so that filesystem doesn't get corrupted, vnxe is literally out of the loop.

 

[kash83]

We have the same Problem on our DELL Storage solution.
Is there really no way we can use them? No workarounds - i cant believe.
This 65533 or 65535 Limitation is part of many storage solutions -_-

 

[fabian]

don't use those broken storage solutions then, it's 2024.

 

[tcabernoch]

It's 2024 and we are all gathered in a forum to discuss a free hosting solution ... and the staff are mocking the end users.

don't use those broken storage solutions then, it's 2024.

Funny thing about 'free' solutions. They are frequently paired with decrepit old hardware.
I too have a VNXe sitting there collecting dust because I hate troubleshooting iSCSI errors all over my network.
These limits in PBS may not be something that can be changed, but they are a completely legitimate subject of discussion.
Thanks to the forum users that provided this info.

 

[fabian]

There's a difference between older, or not top-of-the-line hardware (although even then, you can only expect performance within those constraints) and plain broken stuff like this.

It's 2024 and we are all gathered in a forum to discuss a free hosting solution ... and the staff are mocking the end users.

I'm not mocking anyone (and this is not a forum to "discuss a free hosting solution"?), but we don't want to or need to support targets that are just plain broken.

 

[tcabernoch]

I'm not mocking anyone

Your post history is full of similar commentary.

 

[tcabernoch]

I thought maybe we could use the VNXe with a Virtual Tape Library (VTL) as the front end to the VNXe and connect that to PBS using the native Tape Backup features.
https://quadstor.com/virtual-tape-library.html
https://pbs.proxmox.com/docs/tape-backup.html

But I looked at the PBS tape backup stuff, and I don't see how you would make it talk to the VTL. I'm probably overlooking something.

Edit: Here's the official Proxmox directions on connecting to QuadStore VTL.
https://pbs.proxmox.com/wiki/index.php/Installing_a_Virtual_Tape_Library

Make a VM (running in your Proxmox!) ... Give it iSCSI to the the VNXe ... Install (totally free!) QuadStore VTL ... and the VTL will talk to PBS.
I've never used a VTL. I might try this just for kicks.

---------------
Another idea I had for my VNXe (mine only does blocks storage, no fileshares) was mapping it to a TrueNAS server, and using TrueNAS to share it out. That still wouldn't make it useful with PBS tho ...