feature: write YAML configuration files from other packages #1221
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Successes... - Don't attempt to order traces that do not have a disordered trace - `OrderedTrace` class with attributes and methods - `MatchedBranch` class Very messy at the moment, some thoughts... - noticing a number of places where vectorisation could be used instead of loops and some nesting that seems redundant. - Dictionaries aren't currently mapped to the classes and their structure, many attributes are themselves dictionaries. - 2025-10-09 - Currently need to get ordered_branches passing around correctly, they are meant to be attributes of `MatchedBranch`. - `tests/resources/tracing/ordered_tracing/catenane_post_nodestats.topostats` is currently 304.4MB which is too big, - need to do something about this. It has been renamed for now to `catenane_post_nodestats_20251013.topostats` because of a conflict when rebasing.
Closes #1220 (and possibly others but I can't find them at the moment!) TopoStats modular design, which is being improved in current refactoring, means that it should be easy to extend the analysis pipelines by developing other packages such as [AFMSlicer](https://github.com/AFM-SPM/AFMSlicer) where work is under way. One of the things that will be important is to allow developers of such packages, and in turn users, to generate sample configuration files which they can change as they desire. Rather than have the same code duplicated across packages we can use the `io.write_config_with_comments()` function from TopoStats to load a `<pkg_name>/default_config.yaml` from a package and write that to disk which is what this Pull Request achieves. I've included an early version of `docs/advanced/extending.md` to document how to develop extension packages, it _will_ change dramatically as this takes shape as this is new territory for me, but felt it important to document what I'm doing now so that I can expand and improve on it as things change and lesson are learnt. **NB** This branch will deliberately target `ns-rse/1102-switching-to-TopoStats-class` as that will be the basis on which other packages are built.
|
Tests will fail as that is still a work in progress on the branch being targeted. Pre-commit is complaining because of the Marimo notebooks, this will need addressing, see #1215. |
ns-rse
force-pushed
the
ns-rse/1102-switching-to-TopoStats-class
branch
from
54e4b94 to
23ca66c
Compare
ns-rse
merged commit 8028f84
into
ns-rse/1102-switching-to-TopoStats-class
0 of 7 checks passed
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Closes #1220 (and possibly others but I can't find them at the moment!)
TopoStats modular design, which is being improved in current refactoring, means that it should be easy to extend the analysis pipelines by developing other packages such as AFMSlicer where work is under way.
One of the things that will be important is to allow developers of such packages, and in turn users, to generate sample configuration files which they can change as they desire.
Rather than have the same code duplicated across packages we can use the
io.write_config_with_comments()function from TopoStats to load a<pkg_name>/default_config.yamlfrom a package and write that to disk which is what this Pull Request achieves.I've included an early version of
docs/advanced/extending.mdto document how to develop extension packages, it will change dramatically as this takes shape as this is new territory for me, but felt it important to document what I'm doing now so that I can expand and improve on it as things change and lesson are learnt.NB This branch deliberately targets
ns-rse/1102-switching-to-TopoStats-classas that will be the basis on which other packages are built.