Skip to content

Conversation

@joshuabmoore
Copy link
Collaborator

This PR fixes a bug in which time-series datasets were incorrectly normalised before de-trending. The following changes have been made:

  • The order in which pre-processing is performed has been corrected to first de-trend the data (if specified by the user) before normalising. Documentation has been updated to reflect these changes.
  • The de-trending and z-score normalising operations can be individually enabled by setting the detrend and normalise flags when instantiating the Calculator() object. Both pre-processing steps are enabled by default (as per previous versions of pyspi).
  • Summary statements are printed out to the user when instantiating the calculator object verifying the pre-processing steps that have been applied (detrending and/or normalising if enabled).
  • Coloured outputs have been added to enhance clarity via. a new colorama dependency.

While the reversal of pre-processing operations will not introduce breaking changes for existing users, a discrepancy between results obtained for pyspi < 2.0.0 and pyspi >= 2.0.0 is expected.

@joshuabmoore joshuabmoore added bug Something isn't working documentation Improvements or additions to documentation dependencies Pull requests that update a dependency file labels Apr 13, 2025
@joshuabmoore joshuabmoore changed the title Jmoo2880 norm detrend fix norm detrend fix Apr 13, 2025
Copy link
Member

@anniegbryant anniegbryant left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is great, Josh! The green-colored print messages explicitly stating whether detrending and/or normalizing were applied are a nice touch, and the docstring for the Data class summarizes the detrending + normalization steps well.

@joshuabmoore joshuabmoore merged commit 8d42f97 into main Apr 13, 2025
5 checks passed
@joshuabmoore
Copy link
Collaborator Author

This is great, Josh! The green-colored print messages explicitly stating whether detrending and/or normalizing were applied are a nice touch, and the docstring for the Data class summarizes the detrending + normalization steps well.

Thanks Annie!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working dependencies Pull requests that update a dependency file documentation Improvements or additions to documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants