is s3fs support cache directory?

[qiankunli]

In our scenario, S3 files may be accessed by multiple steps. Does s3fs support configuring a cache directory (with the ability to set the cache size) that can store files retrieved from S3? This way, as long as the cache directory has sufficient space, subsequent accesses to the file using s3fs.open can be directly read from the local cache directory, avoiding the need to download the file from S3 again after the first download.

[martindurant]

fsspec, the base for s3fs, supports caching: https://filesystem-spec.readthedocs.io/en/latest/features.html#caching-files-locally
The first example there is probably what you want. There are multiple cache implementations ( https://filesystem-spec.readthedocs.io/en/latest/api.html#built-in-implementations ), but none of them explicitly measure the current bytescount. It would be reasonable functionality to add to WholeFileCacheFileSystem ( "filecache" ).