-
Notifications
You must be signed in to change notification settings - Fork 1
Description
Currently the low-level API focuses on loading data for ingestion and plotting given specified data schemas. However, current reprocessing bottlenecks have made us realize that writing data from the API might also be a critical requirement to allow post-processing or filtering of data directly in python.
To allow full reuse of existing data schemas, the easiest approach would be to expand the Reader
interface to include a write
method that outputs an existing data frame into the reader-native storage format.
A generic aeon.save
method could then be introduced to take an existing data frame, chunk it into pieces, and call write
for each chunk. It could probably be made to take in the same parameters as aeon.load
, with root
specifying the output folder. The reader
object could similarly provide naming conventions for the device folder and data file paths to ensure consistency with the original dataset.
If this proposal is adopted we may also want to reconsider the name Reader
.