Skip to content

Consider introducing a write method to facilitate data reprocessing #10

@glopesdev

Description

@glopesdev

Currently the low-level API focuses on loading data for ingestion and plotting given specified data schemas. However, current reprocessing bottlenecks have made us realize that writing data from the API might also be a critical requirement to allow post-processing or filtering of data directly in python.

To allow full reuse of existing data schemas, the easiest approach would be to expand the Reader interface to include a write method that outputs an existing data frame into the reader-native storage format.

A generic aeon.save method could then be introduced to take an existing data frame, chunk it into pieces, and call write for each chunk. It could probably be made to take in the same parameters as aeon.load, with root specifying the output folder. The reader object could similarly provide naming conventions for the device folder and data file paths to ensure consistency with the original dataset.

If this proposal is adopted we may also want to reconsider the name Reader.

Metadata

Metadata

Assignees

No one assigned

    Labels

    proposalRequest for a new feature

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions