Skip to content

Avoid double-fetching of chunk mappings on retries in file downloads #115

@clostao

Description

@clostao

Problem:

Due to the fragility of read operations over the subspace-gateway, the package @autonomys/auto-files that is an interface for handling chunked file downloads implements a retries approach for handling the likely errors during a big file download.

When a retry is submitted for a certain chunk, all the objects whose start is within that object are fetched again, making the subspace gateway to be fetching twice those object mappings.

A possible solution:

Implement a component in auto-files-gateway that would be responsible for avoiding this double fetching by deduplicating or caching the mapping requests for the same chunk/object ranges during retries.

Metadata

Metadata

Assignees

Labels

bugSomething isn't workingenhancementNew feature or request

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions