Skip to content

Commit 14b4fb3

Browse files
committed
Formatting
1 parent 9cf623d commit 14b4fb3

File tree

1 file changed

+10
-9
lines changed

1 file changed

+10
-9
lines changed

README.md

+10-9
Original file line numberDiff line numberDiff line change
@@ -1,14 +1,13 @@
11
## Model Description
2-
The model is finetuned on the [WavLM base plus](https://arxiv.org/abs/2110.13900) with 2,374 hours of audio clips from
3-
voice chat for multilabel classification.
4-
The audio clips are automatically labeled using a synthetic data pipeline described in [our blog post](link to blog post here).
5-
A single output can have multiple labels.
6-
The model outputs a n by 6 output tensor where the inferred labels are `Profanity`, `DatingAndSexting`, `Racist`,
7-
`Bullying`, `Other`, `NoViolation`. `Other` consists of policy violation categories with low prevalence such as drugs
8-
and alcohol or self-harm that are combined into a single category.
2+
The model is fine-tuned on the [WavLM base plus](https://arxiv.org/abs/2110.13900) with 2,374 hours of audio clips from
3+
voice chat for multilabel classification. The audio clips are automatically labeled using a synthetic data pipeline
4+
described in [our blog post](link to blog post here). A single output can have multiple labels. The model outputs a
5+
n by 6 output tensor where the inferred labels are `Profanity`, `DatingAndSexting`, `Racist`, `Bullying`, `Other`,
6+
`NoViolation`. `Other` consists of policy violation categories with low prevalence such as drugs and alcohol or
7+
self-harm that are combined into a single category.
98

10-
We evaluated this model on a dataset with human annotated labels that contained a total of 9795 samples with the class
11-
distribution shown below. Note that we did not include the "other" category in this evaluation dataset.
9+
We evaluated this model on a data set with human annotated labels that contained a total of 9,795 samples with the class
10+
distribution shown below. Note that we did not include the "other" category in this evaluation data set.
1211

1312
|Class|Number of examples| Duration (hours)|% of dataset|
1413
|---|---|---|---|
@@ -20,6 +19,8 @@ distribution shown below. Note that we did not include the "other" category in t
2019

2120

2221
If we set the same threshold across all classes and treat the model as a binary classifier across all 4 toxicity classes (`Profanity`, `DatingAndSexting`, `Racist`, `Bullying`), we get a binarized average precision of 94.48%. The precision recall curve is as shown below.
22+
23+
2324
<p align="center">
2425
<img src="images/human_eval_pr_curve.png" alt="PR Curve" width="500"/>
2526
</p>

0 commit comments

Comments
 (0)