Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PyGRB reweighted SNR cut bug #5023

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

Conversation

pannarale
Copy link
Contributor

@Thomas-JACQUOT uncovered a bug when running PyGRB without vetoes.

One of the plotting scripts was not applying the reweighted SNR cut to the reweighted SNR values of found injections. This would then lead to indexing errors such as

IndexError: boolean index did not match indexed array along dimension 0; dimension is 6080 but corresponding boolean dimension is 5827

This PR only affects PyGRB and fixes this problem.

I encourage @Thomas-JACQUOT to use this branch to ensure that his run goes to completion.

  • The author of this pull request confirms they will adhere to the code of conduct

Applying reweighted SNR cut to found injections also to the key storing reweighted SNR itself!
@pannarale pannarale added bug PyGRB PyGRB development labels Jan 24, 2025
@pannarale pannarale self-assigned this Jan 24, 2025
@pannarale
Copy link
Contributor Author

pannarale commented Jan 24, 2025

For reference, I amp posting the output of two jobs, with and without the veto-file

(o4_clean_master) [francesco.pannarale@ldas-osg2 post_processing]$ pycbc_pygrb_plot_injs_results --verbose  --newsnr-threshold 6. --trig-file H1L1V1-PYGRB_GRB170817A_OFFSOURCE_CLUSTERED-1187006058-5648.h5 --seg-files bufferSeg.txt offSourceSeg.txt onSourceSeg.txt  --trigger-time 1187008884 --found-missed-file HL-INJECTIONS_SPLIT_INSPINJ_NSNS_FOUNDMISSED-1187006058-5648.h5 --output-file H1L1V1-PYGRB_PLOT_INJS_RESULTS_MCHIRP_INCL_FOUND-ON-TOP_NSNS_GRB170817A-1187006058-5648.png --y-variable mchirp --x-variable incl
2025-01-24T08:01:33.613-08:00: INFO: Setting output directory.
2025-01-24T08:01:33.614-08:00: INFO: Loading timeslides.
2025-01-24T08:01:33.619-08:00: INFO: Loading segments.
2025-01-24T08:01:33.620-08:00: INFO: Constructing trials.
2025-01-24T08:01:33.671-08:00: INFO: 9834 trials generated.
2025-01-24T08:01:33.674-08:00: INFO: 17980 triggers loaded.
2025-01-24T08:01:33.679-08:00: INFO: 15455 triggers surviving reweighted SNR cut at 6.0.
2025-01-24T08:01:34.491-08:00: INFO: 15273 triggers found within the trials dictionary and sorted.
2025-01-24T08:01:34.499-08:00: INFO: Triggers information extracted.
2025-01-24T08:01:34.560-08:00: INFO: Background bestNR calculated.
2025-01-24T08:01:34.561-08:00: INFO: 1103 injections loaded.
2025-01-24T08:01:34.562-08:00: INFO: 1103 injections surviving reweighted SNR cut at None.
2025-01-24T08:01:34.622-08:00: INFO: 1397 missed injections analysed.
2025-01-24T08:01:34.622-08:00: INFO: 1103 found injections analysed.
2025-01-24T08:01:34.623-08:00: INFO: After applying reweighted SNR cut at 6.0: 1098 found injections and 1402 missed injections
2025-01-24T08:01:35.032-08:00: INFO: Plot complete.
(o4_clean_master) [francesco.pannarale@ldas-osg2 post_processing]$ pycbc_pygrb_plot_injs_results --verbose  --newsnr-threshold 6. --trig-file H1L1V1-PYGRB_GRB170817A_OFFSOURCE_CLUSTERED-1187006058-5648.h5 --seg-files bufferSeg.txt offSourceSeg.txt onSourceSeg.txt  --trigger-time 1187008884 --found-missed-file HL-INJECTIONS_SPLIT_INSPINJ_NSNS_FOUNDMISSED-1187006058-5648.h5 --output-file H1L1V1-PYGRB_PLOT_INJS_RESULTS_MCHIRP_INCL_FOUND-ON-TOP_NSNS_GRB170817A-1187006058-5648.png --y-variable mchirp --x-variable incl --veto-file H1L1V1-VETOES-1187006179-5502.xml
2025-01-24T08:01:44.487-08:00: INFO: Setting output directory.
2025-01-24T08:01:44.488-08:00: INFO: Loading timeslides.
2025-01-24T08:01:44.493-08:00: INFO: Loading segments.
2025-01-24T08:01:44.494-08:00: INFO: Constructing trials.
2025-01-24T08:01:44.506-08:00: INFO: 121s of data vetoed at GPS time 1187006058
2025-01-24T08:01:44.506-08:00: INFO: 2s of data vetoed at GPS time 1187009326
2025-01-24T08:01:44.506-08:00: INFO: 2s of data vetoed at GPS time 1187010566
2025-01-24T08:01:44.506-08:00: INFO: 25s of data vetoed at GPS time 1187011681
2025-01-24T08:01:44.506-08:00: INFO: 121s of data vetoed at GPS time 1187006058
2025-01-24T08:01:44.507-08:00: INFO: 25s of data vetoed at GPS time 1187011681
2025-01-24T08:01:44.507-08:00: INFO: 121s of data vetoed at GPS time 1187006058
2025-01-24T08:01:44.507-08:00: INFO: 25s of data vetoed at GPS time 1187011681
2025-01-24T08:01:44.572-08:00: INFO: 9490 trials generated.
2025-01-24T08:01:44.575-08:00: INFO: 17980 triggers loaded.
2025-01-24T08:01:44.578-08:00: INFO: 15455 triggers surviving reweighted SNR cut at 6.0.
2025-01-24T08:01:45.397-08:00: INFO: 15171 triggers found within the trials dictionary and sorted.
2025-01-24T08:01:45.405-08:00: INFO: Triggers information extracted.
2025-01-24T08:01:45.464-08:00: INFO: Background bestNR calculated.
2025-01-24T08:01:45.465-08:00: INFO: 1103 injections loaded.
2025-01-24T08:01:45.466-08:00: INFO: 1103 injections surviving reweighted SNR cut at None.
2025-01-24T08:01:45.526-08:00: INFO: 1397 missed injections analysed.
2025-01-24T08:01:45.526-08:00: INFO: 1103 found injections analysed.
2025-01-24T08:01:45.526-08:00: INFO: After applying reweighted SNR cut at 6.0: 1098 found injections and 1402 missed injections
2025-01-24T08:01:45.526-08:00: INFO: Applying data vetoes to found injections...
2025-01-24T08:01:45.530-08:00: INFO: 0 injections vetoed due to H1.
2025-01-24T08:01:45.538-08:00: INFO: 0 injections vetoed due to L1.
2025-01-24T08:01:45.545-08:00: INFO: 0 injections vetoed due to V1.
2025-01-24T08:01:45.549-08:00: INFO: 0 injections vetoed.
2025-01-24T08:01:45.549-08:00: INFO: 1098 injections surviving vetoes.
2025-01-24T08:01:45.940-08:00: INFO: Plot complete.

Copy link
Contributor

@Thomas-JACQUOT Thomas-JACQUOT left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've tested this modification on my jobs and it worked ! 🎉

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug PyGRB PyGRB development
Projects
Status: In Progress
Development

Successfully merging this pull request may close these issues.

3 participants