Significantly improve fit-grains performance #816
Conversation
Profiling showed that the list comprehension in `objFuncFitGrain` was taking a significant amount of time. This is because that function is called repeatedly during the least squares fit, and repeatedly performing the list comprehension would take a substantial amount of time. Performing the list comprehension outside of `objFuncFitGrain` was shown to provide a substantial speedup. Signed-off-by: Patrick Avery <[email protected]>
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #816 +/- ##
==========================================
+ Coverage 48.61% 48.66% +0.04%
==========================================
Files 143 143
Lines 22825 22870 +45
==========================================
+ Hits 11097 11129 +32
- Misses 11728 11741 +13 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
|
The second commit on this PR adds support for indexing into an imageseries like this: This means that instead of creating a whole dense array, and then performing fancy indexing into it, we can perform fancy indexing on the sparse array directly, and then create the array from that. This has the potential to speed up |
Rather than expanding the full csr_matrix into a numpy array, then performing fancy indexing on that, do the reverse: perform the fancy indexing on the csr_matrix, and then expand it into a numpy array. This has the potential to save huge amounts of computing time, since we do not have to create the entire dense array before performing the indexing. For the multiruby test case, this did not show any change to the performance. However, for a monolithic Eiger example from Josh Ward, it showed a 5x speed improvement. Signed-off-by: Patrick Avery <[email protected]>
This extracts the image data from the sparse array even faster, because it skips the creation of the unnecessary extra `csr_matrix()` (and its subsequent conversion back to a numpy array). Signed-off-by: Patrick Avery <[email protected]>
Signed-off-by: Patrick Avery <[email protected]>
Profiling showed that the list comprehension in
objFuncFitGrainwas taking a significant amount of time. This is because that function is called repeatedly during the least squares fit, and repeatedly performing the list comprehension would take a substantial amount of time.Performing the list comprehension outside of
objFuncFitGrainwas shown to provide a substantial speedup.This is something I discovered when I was working on the spot tracking code. I'm simply cherry-picking that commit here.
This only affects the fitting part, and not the
pull_spots()part.