Skip to content

Preferred way to store many small files? #187

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
yanshay opened this issue May 13, 2025 · 3 comments
Open

Preferred way to store many small files? #187

yanshay opened this issue May 13, 2025 · 3 comments

Comments

@yanshay
Copy link

yanshay commented May 13, 2025

If I want to store many small files, a few hundreds to a few thousands, is it better to store them all in a single directory or split to several directories?

@thejpster
Copy link
Member

Reading from a directory in FAT is a linear walk so you should always limit how many files you put in one directory. The best value depends on your use case - you can do some benchmarks to find the sweet spot.

But please also note that this crate does not come with a warranty and should not be used to store anything you care about!

@yanshay
Copy link
Author

yanshay commented May 13, 2025

Don't worry, no bitcoins using this crate 😀

I guess (only guess) the time involves both reading a block and then going through entries in block? How many entries fit a single block?

@thejpster
Copy link
Member

A FAT dir entry is 32 bytes. A sector is 512 bytes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants