Fix Evidence threshold bug

[vkakerbeck]

There was a bug in the _get_evidence_update_threshold function in the EvidenceLM where instead of setting the threshold for upding a hypothesis to x_percent_threshold it was set to 1 - x_percent_threshold.

Results are as expected:

• After the change is applied, accuracy and run time go down (green vs. grey) because we test way fewer hypotheses at every step.
• If we set the evidence update threshold manually back to 80% instead of 20%, we get exactly the same results as before the fix (turquoise vs. grey). This is just a sanity check.

Given that the accuracy decreases so much after the fix, it seems like it would be good to create a separate variable for the evidence update threshold (if it is a percentage) instead of using the x_percent_threshold variable defined for the terminal condition. This PR includes a proposed way to add an option to specify evidence_update_threshold as a percentage.

I ran some hyperparameter tests for different values:

Remaining tasks before this can become a full PR:

• Test this parameter on 77 object experiment
• Test whether there is a different optimal x_percent_threshold value, given that these two parameters as disentangled now
• Rerun all benchmarks & update results with the new values (if we set new values. Currently results should be exactly the same).

[nielsleadholm]

The change looks good! Will review properly when it's a full PR but don't see any issue with the updated code.

[tristanls]

suggestion (non-blocking): Perhaps rename "x_percent_threshold" and the variable to something else? Now that evidence_update_threshold can take an "x percent" threshold value (e.g., "80%"), this name becomes quite confusing to parse.

note: I'm generally confused what x means.

[vkakerbeck]

I was planning to do this in a separate PR (as it would touch a lot of the other code). Is that ok?

[tristanls]

Sorry, I should have mentioned that it was a non-blocking suggestion. I have no issues with that being a separate PR, just wanted to highlight my confusion.

[vkakerbeck]

Just turned this into a real PR as it doesn’t affect the benchmark results. It just fixes the bug. Then we can do a follow-up PR with updating to optimal parameters and updating the benchmark results with those.

[nielsleadholm]

Looks great, thanks for fixing this so quickly!

[hlee9212]

LGTM.