Skip to content

Conversation

@zRafaF
Copy link

@zRafaF zRafaF commented Oct 15, 2024

Implements:

  • find_df_transitions: Finds transitions in the dataset that go over a treshold;
  • split_df_percentages: Function to split the data in its percentages.

QoL

  • Added instructions for testing the code with pytest
  • Added testing requierementes
  • Created vscode settings file: This will allow to set standard formatting to the code base, i suggest using black.

@zRafaF
Copy link
Author

zRafaF commented Oct 15, 2024

  • Imported 3w test dataset to the codebase

    • Loading the 3W dataset is possible with bibmon.load_3w() it returns the df and the config file parsed with ConfigParser
  • Added three_w.tools, a file that stores tooling specifically for the 3w dataset

three_w.tools

  • Created load_dataset_ini, a function to load the config file
  • Created split_dataset, a function to prepare the dataset for training, it allows the user to split the data when a transition state happnes, it also allows a subdivision of the dataset in training and validation dfs

QoL

  • Added unity tests to the new functions

@zRafaF zRafaF changed the title Implements two new function tools for DataFrame processing Tools for DataFrame processing and importing 3W dataset Oct 15, 2024
zRafaF and others added 2 commits October 15, 2024 19:17
* fixed relative imports

* fixed relative imports

* removed debbug prints
@zRafaF
Copy link
Author

zRafaF commented Oct 16, 2024

User can now pick another dataset from 3w to download. It will be downloaded and loaded using BytesIO so no file is written to the user's file system,

The download progress is displayed with tqdm

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant