Skip to content

Batched partial reading #37

Open
Open
@mkrd

Description

@mkrd

Currently, a partial read still has to load the entire file into memory if the key was not found by the indexer

Proposal:

  • utils.find_outermost_key_in_json_bytes and seek_index_through_value_bytes should be changed to operate on batches of file bytes, going through at most x megabytes of data at once.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions