Saving Photometric priors as Sacc Tracers

[JaimeRZP]

Problem

Some priors for systematic parameters will depend on the measured data in the same way the measurement and covariance will. Therefore, I expect that the calibration of said parameters will become a TXPIPE stage. After this calibration, we will have to figure out how to save said priors inside sacc such that it can be passed to the likelihood builder and inference frameworks. In this particular issue I am focusing on the photometric uncertainties generated using the RAIL ecosystem and calibrated by nz_prior.

Proposed solution

From a conversation with Firecrown, it seems like the most intuitive way to achieve this would be by proposing a new sacc tracer that replaces the QPTracer generated by RAIL and that that contains:

• z
• nz
• photoz_params_mean
• photoz_params_cholesky
  
  I tried loading these on the NZTracer extra_columns fields but the dimensionalities are not compatible.

Known problems

• In the above solution, the user will have to remember what parameters they have calibrated although maybe this could be specified in the metadata of the arrays.
• Cross-tomographic bin correlations cannot be implemented using this approach.
• Joe: Do we really want to carry particular photoz uncertainty models information in the sacc?

Alternative solutions

• Instead of completely substituting the NZTracers by another tracer which also contains the prior information, we could add a second tracer per tomographic bin which only contains the mean and covariance.
• Alternatively, we could save the mean and covariance to the sacc file the same way we save the measurement and the covariance. On the one hand, this might make keeping track of what parameters belong to what tomographic bin more complicated. On the other hand, it will allow us to include cross-tomographic bin correlations in the near future more easily once they are needed.

[JaimeRZP]

thoughts @elisachisari, @joezuntz @damonge (feel free to ping others who might have an interest)?

[joezuntz]

This is part of a more generic class of problems of saving information about systematic error models in with the system. We could also imagine saving PSF templates (like in HSC Y3) or shear calibration uncertainties.

We should be a little careful here because there is an overlap between what is part of the model and what is part of the (SACC) data. We wouldn't want to get into saving IA-related information here, for example.

Another possibility, especially in the light of your point that cross-tomo tracer information may come up, would be an additional general class of information to be stored in the sacc files, something like SystematicInfo or SomethingMetadata for storing information that is too structured to go in the metadata but not quite so directly attached to the Tracer or DataPoint objects.

[JaimeRZP]

We should be a little careful here because there is an overlap between what is part of the model and what is part of the (SACC) data. We wouldn't want to get into saving IA-related information here, for example.

Ok, here we have a design disagreement. The priors for the photoz uncertainties will depend on the model they refer to (shifts, shifts and widths). Even worse, in the case of linear models, it is best to fold the weights in the model into the prior, meaning that I would like to actually store a matrix: W = M * L, where M is the model weights and L is the cholesky of the prior.

This would seem to suggest that we want to run nz_prior as a stage but not safe the input in sacc what so ever. The question then is, where? Does txpipe produce other files that we could pass to firecrown? Is firecrown aware of this files?

[joezuntz]

We discussed this on a telecon and agreed that sacc could include a general class of tracer uncertainty models, which this would be an example of. I will put together a prototype.

[damonge]

Just a thought, to potentially keep things simple for the time being: would it be worth trying to simply squeeze this into the existing NZTracers? E.g. in the simpler scenario where the N(z) uncertainties do not couple N(z)s from different bins, you just need to store a set of PCA vectors (each with the same size as that tracer's N(z)), and the covariance of the corresponding linear coefficients. I would have thought the former can just be stored in the extra_columns part of the tracer (and IIRC, this was in fact the original motivation for having that). For the latter, I am not sure what is the best solution, but I'm guessing a simple extension of the NZTracer would be enough...

[joezuntz]

That would certainly work as a stop-gap, I think without even any changes?

[JaimeRZP]

i tried overloading extra_columns but the dimensions are not correct. If you make the demands on the dimensions more permissive I can do this now.

[joezuntz]

That wouldn't be straightforward as right now all the columns are saved in the same FITS table, so have to be the same length.

(I really want to provide other options than FITS but have never got around to it).