Add a component to reduce waveform readout window

[maxnoe]

The question is now where to best apply this.

I see a couple of options:

• Add directly to ctapipe-process, this would have the advantage that you could even just apply the reducer and write out reduced waveforms.
• Add to CameraCalibrator.dl0_to_dl1 step.

I would lean towards directly adding it in process.

[mdpunch]

Thanks Max: A couple of questions / comments from my side:

• If this is called with no config values, then start and end default to None so tel_container.waveform = tel_container.waveform[start:end] = tel_container.waveform[None:None] which means no change. This is the correct default, I agree.
• This looks for r0, r1 and dl0 waveforms in the event, and applies the reduction if these exist. Also good!

Minor comment on the test routine:

• For LST and MST-NectarCAM at least, which use the same EventBuilder, it seems to be feasible only to process an integer fraction of the full waveform (at least without major modifications)
  • so LST 40->20 (or ->10 probably too short for TTS reasons), and from the MCs as produced it looks like [10:30] is best.
  • for MST-Nectarcam, 60->30 (too long) -> 20 (too long still) -> 15 probably best, so from the MC [12:27] is best.
• ... but for a test routine the values aren't really critical (they would be in real life, though).
• Also, I have no idea what the constraints are on the MST-FlashCam side, or for the CHEC camera (but that's probably not at all critical even for GRBs).

So, generally looks fine on my side.

[kosack]

Just as a note: the reason we called it "Data Volume Reducer" and not "Zero Suppressor" was that we expected to potentially cut data not just spatially (per pixel), but also temporally (e.g. shorten waveforms). The "volume" here was not just "size of the data", but really the 3D volume formed by the waveform info (2D space + 1D time). So implementing this as a type of DataVolumeReducer makes a lot of sense. The notion of DVR comes from the original requirements elicitation process in 2016/17.

If it is a DataVolumeReducer, it's already a step in ctapipe-process. However, right now we only support one, so it might be useful to support a series of them if you want to truncate the waveforms and also zero-suppress pixels.

[maxnoe]

I would have implemented it as a volume reducer, but the current API of the DVR base class does not allow for that, as it returns a mask of pixels to keep.

However, I think it would make sense to generalize the API so that DVR can be something else then zero supressing whole pixels.

[mdpunch]

I guess it is a Data Volume Reducer in the sense that indeed it does reduce the volume of the data, but it's a bit special in that it's "pretending" that the camera produces shorter waveforms. So it would be "upstream" of DVR and Zero Suppression. But it doesn't matter so much, as long as it does what it says on the tin!

[mdpunch]

Still seems fine!

(Next step, to add a "GRB HaST" component in another pull request?)

[ctao-dpps-sonarqube]

Analysis Details

0 Issues

• 0 Bugs
• 0 Vulnerabilities
• 0 Code Smells

Coverage and Duplications

• 100.00% Coverage (93.80% Estimated after merge)
• 0.00% Duplicated Code (0.70% Estimated after merge)

Project ID: cta-observatory_ctapipe_AY52EYhuvuGcMFidNyUs

View in SonarQube

[maxnoe]

(Next step, to add a "GRB HaST" component in another pull request?)

We don't simulate dead time / busy telescopes, so what would this component do actually do?

[mdpunch]

(Next step, to add a "GRB HaST" component in another pull request?)

We don't simulate dead time / busy telescopes, so what would this component do actually do?

That's true (for now... I'd like to add that with my Multiplicity-Energy Look-up table (which is actually a TelescopeTigggerPattern-Energy LUT).

So, if there are no busy telescopes for now, there can be no change to the HaST simulation, you're right!

So... I should make a pull request for having the busy behaviour / included?

But, this is a step which should happen after the main ctapipe production, requiring at a minimum inputs of the standard background simulations (maybe protons scaled-up to the CR rate is enough).

But for now, maybe doing a ctapipe production with only the NectarCAMs having a reduced window is enough (and I'll apply the "GRB HaST" downstream (but I'd like some feedback from LST/TIB/DPPS/ASWG if my results and proposal seem reasonable) .