Performance improvements in glue and echo related to subsets #3618
Description
I've been investigating performance issues in jdaviz, starting from the massive slowdown in cubeviz after making just a few subsets. This is a meta-issue to keep track of the work on this.
Changes
I found several places in echo where it was possible to avoid repeated/duplicate calls:
- Fixed a bug that caused unnecessary global callbacks even if value did not change glue-viz/echo#46
- Avoid storing duplicate functions/methods in CallbackContainer glue-viz/echo#48
- Fixed bug that occurred if a callback attribute used a property which didn't exist yet glue-viz/echo#50
With these changes, I sped up cubeviz with 10 subsets by a factor of 4.5x. I have now released echo 0.11 which contains these changes. The changes are pretty generic and will likely result in speedup in other jdaviz applications too.
I have also made some improvements in glue-core for a further factor of 2x speedup:
- Allow initial values to be customized for state classes glue-viz/glue#2544 (speeds cubeviz up by ~30%)
- Delay callbacks related to layers when adding datasets to viewers glue-viz/glue#2545 (speeds cubeviz up by ~25%)
And I've also found some bottlenecks in jdaviz:
- Don't add actual subsets to CubevizProfileView #3626
- Avoid circular calls to _on_layers_update #3628
Test code
I've been testing performance using:
from regions import PixCoord, CirclePixelRegion
from jdaviz import Cubeviz
cubeviz = Cubeviz()
uri = "mast:JWST/product/jw02732-c1001_t004_miri_ch1-short_s3d.fits"
cubeviz.load_data(uri, cache=True)
N=12
for iter in range(N):
my_reg = CirclePixelRegion(center=PixCoord(x=35, y=35), radius=10)
cubeviz.plugins['Subset Tools'].import_region(my_reg, combination_mode='new')Results
Here is the time to add N subsets by varying N above with the original code, with echo 0.11, and with echo 0.11 and the jdaviz changes above. This doesn't yet include improvements related to glue-core PRs:
and in log:
and here is the final speedup:
