Semantic Entropy Probes

[dylanbouchard]

Is your feature request related to a problem? Please describe.
We would like to offer a scorer for Semantic Entropy Probes. This tuner offers the high-performance of Semantic Entropy without the high cost and latency at generation time required by black-box scorers. For more on this scorer, refer to Kossen et al., 2024.

Describe the solution you'd like
A new scorer class SemanticEntropyProbes

Describe alternatives you've considered
Current suite of scorers

Additional context
This method is complicated and nuanced. Please ensure you have read and fully understand the paper before asking to work on this issue.

[dross20]

Hey @dylanbouchard! If I may, I'd love to help out with this feature!

One note about this method: it involves training a logistic regression model on the hidden state of a language model, meaning we will need a separate pretrained logistic regressor for each LM we want to support. Are there any particular models that you definitely want to support for this scorer?

[dylanbouchard]

We'd love to have your help @dross20 ! think we can choose based on model popularity and feasibility. So among models that this is feasible for, which is most popular? I'll leave it up to you to decide where to start. Thanks again!

[dylanbouchard]

@dross20 a couple of notes:

1. The dataset used for training will likely be very influential here. What do you think about using TriviaQA, since it is not domain-specific and appears to have good results in the original paper?
2. Eventually, we will want to enable users to train their own Semantic Entropy probes model. This can be a later PR if you do not want to include in this one.