Reduced focal loss implementation is incorrect for binary case #102
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Fixed Binary focal loss when reduced threshold is not None. Focal term should be continuous and equal to 1 at pt equal to reduced threshold
|
Hmmm, indeed! Thanks for spotting this bug. I've never get any decent results with reduced focal loss, and this may explain why :)
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The current implementation is same as the one given in original paper . Although, it doesn't work as intended when value of reduced threshold is not 0.5. As per the paper, the focal term should be 1 when pt is less than reduced threshold and then exponentially decrease from 1 to 0 as pt goes from reduced threshold to 1. By this logic, the value of focal term should be 1 at pt equal to reduced threshold from both sides of the curve. From RHS the value is given as (1-pt/thresh) to power gamma multiplied by log pt. This will only be 1 at pt=thresh, when thresh=0.5, otherwise not.
So the correct formula focal term for pt>thresh should be ((1-pt)/(1-thresh) to the power gamma multiplied by log pt.
The corrected implementation goes hand in hand with the one done for softmax_focal_loss_with_logits which works as intended in the paper.