[Bug]: Nodestats raises warning calculating angles sometimes #1210
Description
Checklist
- Find the offending file in the output. If processing halts, re-run analysis with
topostats --core 1 process. - Describe the bug.
- Include the configuration file.
- Copy of the log-file from running with
topostats --log-level debug <command>. - The exact command that failed. This is what you typed at the command line, including any options.
- TopoStats version, this is reported by
topostats --version - Operating System and Python Version
Describe the bug
I found I was having trouble testing equality of the Nodes.unmatched_branch_stats.angles as some in the catenanes image were reported as np.foat64(nan) and testing whether they were equal to either np.float64(float("nan")) or np.float64(np.nan) didn't work. Digging back I found the source of this lay in toposats.tracing.nodestats.nodeStats.calc_angles() where the following warning is raised.
It arises when -1 < cos_angles > 1 but realised it was going to take too long to dig this out so am noting the issue here.
As well as noting this inline in tracing/topostats/nodestats.py I've also left in place a clause in tests/test_processing.py::test_run_nodestats which skips the three affected nodes (out of 41 in catenanes) so that and the comments left in-line will also need removing once this has been fixed (although note these are yet to be merged into main as it is work in progress).
Copy of the log-file from running with topostats --log-level debug <command>
/home/neil/work/git/hub/AFM-SPM/TopoStats/topostats/tracing/nodestats.py:1129:
RuntimeWarning: invalid value encountered in arccos
return abs(np.arccos(cos_angles) / np.pi * 180) # angles in degrees Include the configuration file
This is using the default configuration file...
# Config file generated 2025-09-24 14:53:18
# # For more information on configuration and how to use it:
# https://afm-spm.github.io/TopoStats/main/configuration.html
base_dir: ./ # Directory in which to search for data files
output_dir: ./output # Directory to output results to
log_level: info # Verbosity of output. Options: warning, error, info, debug
cores: 2 # Number of CPU cores to utilise for processing multiple files simultaneously.
file_ext: .spm # File extension of the data files.
loading:
channel: Height # Channel to pull data from in the data files.
extract: raw # Array to extract when loading .topostats files.
filter:
run: true # Options : true, false
row_alignment_quantile: 0.5 # lower values may improve flattening of larger features
threshold_method: std_dev # Options : otsu, std_dev, absolute
otsu_threshold_multiplier: 1.0
threshold_std_dev:
below: 10.0 # Threshold for data below the image background
above: 1.0 # Threshold for data above the image background
threshold_absolute:
below: -1.0 # Threshold for data below the image background
above: 1.0 # Threshold for data above the image background
gaussian_size: 1.0121397464510862 # Gaussian blur intensity in px
gaussian_mode: nearest # Mode for Gaussian blurring. Options : nearest, reflect, constant, mirror, wrap
# Scar remvoal parameters. Be careful with editing these as making the algorithm too sensitive may
# result in ruining legitimate data.
remove_scars:
run: false
removal_iterations: 2 # Number of times to run scar removal.
threshold_low: 0.250 # lower values make scar removal more sensitive
threshold_high: 0.666 # lower values make scar removal more sensitive
max_scar_width: 4 # Maximum thickness of scars in pixels.
min_scar_length: 16 # Minimum length of scars in pixels.
grains:
run: true # Options : true, false
# Thresholding by height
grain_crop_padding: 1 # Padding to apply to grains. Needs to be at least 1, more padding may help with unets.
threshold_method: std_dev # Options : std_dev, otsu, absolute, unet
otsu_threshold_multiplier: 1.0
threshold_std_dev:
below: [10.0] # Thresholds for grains below the image background. List[float].
above: [1.0] # Thresholds for grains above the image background. List[float].
threshold_absolute:
below: [-1.0] # Thresholds for grains below the image background. List[float].
above: [1.0] # Thresholds for grains above the image background. List[float].
direction: above # Options: above, below, both (defines whether to look for grains above or below thresholds or both)
area_thresholds:
above: [300, 3000] # above surface [Low, High] in nm^2 (also takes null)
below: [null, null] # below surface [Low, High] in nm^2 (also takes null)
remove_edge_intersecting_grains: true # Whether or not to remove grains that touch the image border
unet_config:
model_path: null # Path to a trained U-Net model
upper_norm_bound: 5.0 # Upper bound for normalisation of input data. This should be slightly higher than the maximum desired / expected height of grains.
lower_norm_bound: -1.0 # Lower bound for normalisation of input data. This should be slightly lower than the minimum desired / expected height of the background.
remove_disconnected_grains: false # Whether to remove grains in the crop that don't touch the original grain mask.
confidence: 0.5 # Confidence threshold for the UNet model. Smaller is more generous, larger is more strict.
vetting:
whole_grain_size_thresholds: null # Size thresholds for whole grains in nanometres squared, ie all classes combined. Tuple of 2 floats, ie [low, high] eg [100, 1000] for grains to be between 100 and 1000 nm^2. Can use None to not set an upper/lower bound.
class_conversion_size_thresholds: null # Class conversion size thresholds, list of tuples of 3 integers and 2 integers, ie list[tuple[tuple[int, int, int], tuple[int, int]]] eg [[[1, 2, 3], [5, 10]]] for each region of class 1 to convert to 2 if smaller than 5 nm^2 and to class 3 if larger than 10 nm^2.
class_region_number_thresholds: null # Class region number thresholds, list of lists, ie [[class, low, high],] eg [[1, 2, 4], [2, 1, 1]] for class 1 to have 2-4 regions and class 2 to have 1 region. Can use None to not set an upper/lower bound.
class_size_thresholds: null # Class size thresholds (nm^2), list of tuples of 3 integers, ie [[class, low, high],] eg [[1, 100, 1000], [2, 1000, None]] for class 1 to have 100-1000 nm^2 and class 2 to have 1000-any nm^2. Can use None to not set an upper/lower bound.
nearby_conversion_classes_to_convert: null # Class conversion for nearby regions, list of tuples of two-integer tuples, eg [[[1, 2], [3, 4]]] to convert class 1 to 2 and 3 to 4 for small touching regions
class_touching_threshold: 5 # Number of dilation steps to use for detecting touching regions
keep_largest_labelled_regions_classes: null # Classes to keep the only largest regions for, list of integers eg [1, 2] to keep only the largest regions of class 1 and 2
class_connection_point_thresholds: null # Class connection point thresholds, [[[class_1, class_2], [min, max]]] eg [[[1, 2], [1, 1]]] for class 1 to have 1 connection point with class 2
classes_to_merge: null # Classes to merge into a single combined class. List of lists, eg [[1, 2]] to merge classes 1 and 2. New classes will be appended to the tensor.
grainstats:
run: true # Options : true, false
edge_detection_method: binary_erosion # Options: canny, binary erosion. Do not change this unless you are sure of what this will do.
extract_height_profile: true # Extract height profiles along maximum feret of molecules
class_names: ["DNA", "Protein"] # The names corresponding to each class of a object identified, please specify merged classes after.
disordered_tracing:
run: true # Options : true, false
class_index: 1 # The class index to trace. This is the class index of the grains.
min_skeleton_size: 10 # Minimum number of pixels in a skeleton for it to be retained.
mask_smoothing_params:
gaussian_sigma: 2 # Gaussian smoothing parameter 'sigma' in pixels.
dilation_iterations: 2 # Number of dilation iterations to use for grain smoothing.
holearea_min_max: [0, null] # Range (min, max) of a hole area in nm to refill in the smoothed masks.
skeletonisation_params:
method: topostats # Options : zhang | lee | thin | topostats
height_bias: 0.6 # Percentage of lowest pixels to remove each skeletonisation iteration. 1 equates to zhang.
pruning_params:
method: topostats # Method to clean branches of the skeleton. Options : topostats
max_length: 10.0 # Maximum length in nm to remove a branch containing an endpoint.
height_threshold: # The height to remove branches below.
method_values: mid # The method to obtain a branch's height for pruning. Options : min | median | mid.
method_outlier: mean_abs # The method to prune branches based on height. Options : abs | mean_abs | iqr.
only_height_prune_endpoints: False # Whether to restrict height-based pruning to skeleton segments containing an endpoint or not.
nodestats:
run: true # Options : true, false
node_joining_length: 7.0 # The distance in nanometres over which to join nearby crossing points.
node_extend_dist: 14.0 # The distance in nanometres over which to join nearby odd-branched nodes.
branch_pairing_length: 20.0 # The length in nanometres from the crossing point to pair and trace, obtaining FWHM's.
pair_odd_branches: false # Whether to try and pair odd-branched nodes. Options: true and false.
ordered_tracing:
run: true
ordering_method: nodestats # The method of ordering the disordered traces.
splining:
run: true # Options : true, false
method: "rolling_window" # Options : "spline", "rolling_window"
rolling_window_size: 20.0e-9 # size in nm of the rolling window.
rolling_window_resampling: true # Whether to resample the trace or not.
rolling_window_resample_regular_spatial_interval: 0.5e-9 # The spatial interval to resample the trace to in nm.
spline_step_size: 7.0e-9 # The sampling rate of the spline in metres.
spline_linear_smoothing: 5.0 # The amount of smoothing to apply to linear features.
spline_circular_smoothing: 5.0 # The amount of smoothing to apply to circular features.
spline_degree: 3 # The polynomial degree of the spline.
curvature:
run: true # Options : true, false
colourmap_normalisation_bounds: [-0.5, 0.5] # Radians per nm to normalise the colourmap to.
plotting:
run: true # Options : true, false
style: topostats.mplstyle # Options : topostats.mplstyle or path to a matplotlibrc params file
savefig_format: null # Options : null, png, svg or pdf. tif is also available although no metadata will be saved. (defaults to png) See https://matplotlib.org/stable/api/_as_gen/matplotlib.pyplot.savefig.html
savefig_dpi: 100 # Options : null (defaults to the value in topostats/plotting_dictionary.yaml), see https://afm-spm.github.io/TopoStats/main/configuration.html#further-customisation and https://matplotlib.org/stable/api/_as_gen/matplotlib.pyplot.savefig.html
pixel_interpolation: null # Options : https://matplotlib.org/stable/gallery/images_contours_and_fields/interpolation_methods.html
grain_crop_plot_size_nm: -1 # Size in nm of the square cropped grain images if using the grains image set. If -1, will use the grain's default bounding box size.
image_set: # Options : all, core, filters, grains, grain_crops, disordered_tracing, nodestats, ordered_tracing, splining. Uncomment to include
# - all
- core
# - filters
# - grains
# - grain_crops
# - disordered_tracing
# - nodestats
# - ordered_tracing
# - splining
zrange: [null, null] # low and high height range for core images (can take [null, null]). low <= high
colorbar: true # Options : true, false
axes: true # Options : true, false (due to off being a bool when parsed)
num_ticks: [null, null] # Number of ticks to have along the x and y axes. Options : null (auto) or integer > 1
cmap: null # Colormap/colourmap to use (default is 'nanoscope' which is used if null, other options are 'afmhot', 'viridis' etc.)
mask_cmap: blue_purple_green # Options : blu, jet_r and any in matplotlib
histogram_log_axis: false # Options : true, false
summary_stats:
run: true # Whether to make summary plots for output data
config: null
To Reproduce
Uses the catenane image from tests/resources/tracing/nodestats/catenane_post_disordered_tracing.pkl.
TopoStats Version
2.1.2
Python Version
3.11
Operating System
GNU/Linux
Python Packages
❱ topostats --version
Installed version of TopoStats: 2.3.2.dev491+g1e465a98d5.d20250804
absl-py==2.1.0
accessible-pygments==0.0.5
AFMReader @ git+https://github.com/AFM-SPM/AFMReader@9a8ae1beb9c9d9432032fd7ac48f0b721eae3c6b
alabaster==1.0.0
annotated-types==0.7.0
anyio==4.9.0
app-model==0.3.1
appdirs==1.4.4
argon2-cffi==23.1.0
argon2-cffi-bindings==21.2.0
arrow==1.3.0
art==6.4
astroid==3.3.8
asttokens==3.0.0
astunparse==1.6.3
async-lru==2.0.5
attrs==25.3.0
autopep8==2.0.4
babel==2.17.0
beautifulsoup4==4.13.3
bermuda==0.1.4
biopython==1.85
black==25.1.0
bleach==6.2.0
build==1.2.2.post1
cachetools==6.0.0
cachey==0.2.1
cattrs==24.1.3
certifi==2025.1.31
cffi==1.17.1
cfgv==3.4.0
chardet==5.2.0
charset-normalizer==3.4.1
cheap_repr==0.5.2
click==8.1.8
cloudpickle==3.1.1
colorama==0.4.6
comm==0.2.2
contourpy==1.3.1
coverage==7.6.12
crc32c==2.7.1
cycler==0.12.1
dask==2025.5.1
debugpy==1.8.13
decorator==5.1.1
defusedxml==0.7.1
dill==0.3.9
distlib==0.3.9
docstring-to-markdown==0.16
docstring_parser==0.16
docutils==0.21.2
donfig==0.8.1.post1
EditorConfig==0.17.0
et_xmlfile==2.0.0
execnet==2.1.1
executing==2.2.0
fastjsonschema==2.21.1
filelock==3.17.0
filetype==1.2.0
flake8==7.1.2
flatbuffers==25.2.10
flexcache==0.3
flexparser==0.4
fonttools==4.56.0
fqdn==1.5.1
freetype-py==2.5.1
fsspec==2025.5.1
gast==0.6.0
ghp-import==2.1.0
google-pasta==0.2.0
gprof2dot==2025.4.14
griffe==1.5.7
grpcio==1.70.0
h11==0.14.0
h5glance==0.9.0
h5py==3.13.0
HeapDict==1.0.1
hsluv==5.0.4
htmlgen==2.0.0
httpcore==1.0.7
httpx==0.28.1
icdiff==2.0.7
identify==2.6.7
idna==3.10
igor2==0.5.9
imageio==2.37.0
imagesize==1.4.1
importlib_metadata==8.6.1
importlib_resources==6.5.2
in-n-out==0.2.1
iniconfig==2.0.0
ipykernel==6.29.5
ipython==8.34.0
ipython-genutils==0.2.0
ipython_pygments_lexers==1.1.1
ipywidgets==8.1.5
isoduration==20.11.0
isort==6.0.0
itsdangerous==2.2.0
jedi==0.19.2
jedi-language-server==0.45.1
Jinja2==3.1.5
joblib==1.4.2
jsbeautifier==1.15.3
json5==0.10.0
jsonpointer==3.0.0
jsonschema==4.23.0
jsonschema-specifications==2024.10.1
jupyter==1.1.1
jupyter-console==6.6.3
jupyter-events==0.12.0
jupyter-highlight-selected-word==0.2.0
jupyter-lsp==2.2.5
jupyter_client==8.6.3
jupyter_contrib_core==0.4.2
jupyter_contrib_nbextensions==0.7.0
jupyter_core==5.7.2
jupyter_nbextensions_configurator==0.6.4
jupyter_server==2.15.0
jupyter_server_terminals==0.5.3
jupyterlab==4.3.6
jupyterlab_pygments==0.3.0
jupyterlab_server==2.27.3
jupyterlab_widgets==3.0.13
jupyterthemes==0.20.0
jupytext==1.16.7
keras==3.8.0
kiwisolver==1.4.8
lazy_loader==0.4
lesscpy==0.15.1
libclang==18.1.1
llvmlite==0.44.0
locket==1.0.0
loguru==0.7.3
lsprotocol==2023.0.1
lxml==5.3.1
magicgui==0.10.0
marimo==0.11.29
Markdown==3.7
markdown-it-py==3.0.0
MarkupSafe==3.0.2
matplotlib==3.10.5
matplotlib-inline==0.1.7
mccabe==0.7.0
mdit-py-plugins==0.4.2
mdurl==0.1.2
mergedeep==1.3.4
mike==2.1.3
mistune==3.1.3
mkdocs==1.6.1
mkdocs-autorefs==1.3.1
mkdocs-get-deps==0.2.0
mkdocs-material==9.6.5
mkdocs-material-extensions==1.3.1
mkdocs-mermaid2-plugin==1.2.1
mkdocs-terminal==4.7.0
mkdocstrings==0.28.1
mkdocstrings-python==1.16.1
ml-dtypes==0.4.1
mypy==1.15.0
mypy-extensions==1.0.0
myst-parser==4.0.1
namex==0.0.8
napari==0.6.1
-e git+ssh://[email protected]/AFM-SPM/napari-AFMReader.git@f1305f3ab2b4471665e550ddaaeccf8d66798171#egg=napari_afmreader
napari-console==0.1.3
napari-plugin-engine==0.2.0
napari-plugin-manager==0.1.6
napari-svg==0.2.1
narwhals==1.32.0
nbclient==0.10.2
nbconvert==7.16.6
nbformat==5.10.4
nbmultitask==0.1.0
nest-asyncio==1.6.0
networkx==3.5
nodeenv==1.9.1
notebook==7.3.3
notebook_shim==0.2.4
npe2==0.7.8
numba==0.61.0
numcodecs==0.16.1
numpy==2.0.2
numpydoc==1.8.0
numpyencoder @ git+https://github.com/AFM-SPM/numpyencoder@cd84c7cbb7e59386ed7c0a7e03e03c4134b2b84c
openpyxl==3.1.5
opt_einsum==3.4.0
optree==0.14.0
overrides==7.7.0
packaging==25.0
paginate==0.5.7
pandas==2.2.3
pandocfilters==1.5.1
parso==0.8.4
partd==1.4.2
pathspec==0.12.1
pexpect==4.9.0
pickleshare==0.7.5
pillow==11.3.0
Pint==0.24.4
platformdirs==4.3.6
pluggy==1.5.0
ply==3.11
pockets==0.9.1
polars==1.33.1
pooch==1.8.2
pprintpp==0.4.0
pre_commit==4.1.0
prometheus_client==0.21.1
prompt_toolkit==3.0.50
protobuf==5.29.3
psutil==5.9.8
psygnal==0.12.0
ptyprocess==0.7.0
pure_eval==0.2.3
pycodestyle==2.12.1
pyconify==0.2.1
pycparser==2.22
pycrdt==0.11.1
pydantic==2.11.5
pydantic-compat==0.1.2
pydantic_core==2.33.2
pydata-sphinx-theme==0.16.1
pydocstyle==6.3.0
pyflakes==3.2.0
pygls==1.3.1
Pygments==2.19.1
pylint==3.3.7
pylsp-mypy==0.7.0
pymdown-extensions==10.14.3
PyOpenGL==3.1.9
pyparsing==3.2.1
pyproject-api==1.9.1
pyproject_hooks==1.2.0
PyQt5==5.15.11
PyQt5-Qt5==5.15.17
PyQt5_sip==12.17.0
pyright==1.1.398
pyspm==0.6.3
pytest==8.4.0
pytest-cov==6.1.1
pytest-durations==1.3.1
pytest-github-actions-annotate-failures==0.3.0
pytest-icdiff==0.9
pytest-loguru==0.4.0
pytest-mock==3.14.0
pytest-mpl==0.17.0
pytest-profiling==1.8.1
pytest-pylint==0.21.0
pytest-regtest==2.3.1
pytest-testmon==2.1.3
pytest-xdist==3.6.1
python-dateutil==2.9.0.post0
python-json-logger==3.3.0
python-jsonrpc-server==0.4.0
python-language-server==0.36.2
python-lsp-jsonrpc==1.1.2
python-lsp-ruff==2.2.2
python-lsp-server==1.12.2
pytoolconfig==1.3.1
pytz==2025.1
pyupgrade==3.19.1
PyYAML==6.0.2
pyyaml_env_tag==0.1
pyzmq==26.3.0
qtconsole==5.6.1
QtPy==2.4.3
referencing==0.36.2
regex==2024.11.6
requests==2.32.3
rfc3339-validator==0.1.4
rfc3986-validator==0.1.1
rich==13.9.4
roman-numerals-py==3.0.0
rope==1.13.0
rpds-py==0.23.1
ruamel.yaml==0.18.10
ruamel.yaml.clib==0.2.12
ruff==0.11.5
ruff-lsp==0.0.62
schema==0.7.7
scikit-image==0.25.2
scikit-learn==1.6.1
scipy==1.15.3
seaborn==0.13.2
Send2Trash==1.8.3
shellingham==1.5.4
six==1.17.0
skan==0.12.2
sniffio==1.3.1
snoop==0.6.0
snowballstemmer==2.2.0
soupsieve==2.6
Sphinx==8.2.0
sphinx-autoapi==3.6.0
sphinx-markdown-tables==0.0.17
sphinx-multiversion==0.2.4
sphinx-rtd-theme==3.0.2
sphinxcontrib-applehelp==2.0.0
sphinxcontrib-devhelp==2.0.0
sphinxcontrib-htmlhelp==2.1.0
sphinxcontrib-jquery==4.1
sphinxcontrib-jsmath==1.0.1
sphinxcontrib-mermaid==1.0.0
sphinxcontrib-napoleon==0.7
sphinxcontrib-qthelp==2.0.0
sphinxcontrib-serializinghtml==2.0.0
stack-data==0.6.3
starlette==0.46.1
superqt==0.7.1
syrupy==4.9.1
tabulate==0.9.0
tensorboard==2.18.0
tensorboard-data-server==0.7.2
tensorflow==2.18.0
tensorflow-io-gcs-filesystem==0.37.1
termcolor==2.5.0
terminado==0.18.1
threadpoolctl==3.5.0
tifffile==2025.2.18
tinycss2==1.4.0
tokenize_rt==6.1.0
tomli_w==1.2.0
tomlkit==0.13.2
toolz==1.0.0
topoly==1.1.0
-e git+ssh://[email protected]/AFM-SPM/TopoStats.git@f95364da521012495b334c2ed5eaa6a864aaa0af#egg=topostats
tornado==6.4.2
tox==4.26.0
tqdm==4.67.1
traitlets==5.14.3
triangle==20250106
typer==0.16.0
types-python-dateutil==2.9.0.20241206
typing-inspection==0.4.1
typing_extensions==4.12.2
tzdata==2025.1
ujson==5.10.0
uri-template==1.3.0
urllib3==2.3.0
uvicorn==0.34.0
verspec==0.1.0
virtualenv==20.31.2
vispy==0.15.2
watchdog==6.0.0
wcwidth==0.2.13
webcolors==24.11.1
webencodings==0.5.1
websocket-client==1.8.0
websockets==15.0.1
Werkzeug==3.1.3
whatthepatch==1.0.7
widgetsnbextension==4.0.13
wrapt==1.17.2
yapf==0.43.0
zarr==3.0.8
zipp==3.21.0