Skip to content

Commit

Permalink
Update README.md
Browse files Browse the repository at this point in the history
  • Loading branch information
kelpabc123 committed Jun 28, 2024
1 parent 14b4fb3 commit 0eebdff
Showing 1 changed file with 14 additions and 8 deletions.
22 changes: 14 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,13 @@
## Model Description
The model is fine-tuned on the [WavLM base plus](https://arxiv.org/abs/2110.13900) with 2,374 hours of audio clips from
We present a large classification model trained on a manually curated real-world dataset that can be used as a new
benchmark for advancing research in toxicity detection and classification.
Our model is fine-tuned on the [WavLM base plus](https://arxiv.org/abs/2110.13900) with 2,374 hours of audio clips from
voice chat for multilabel classification. The audio clips are automatically labeled using a synthetic data pipeline
described in [our blog post](link to blog post here). A single output can have multiple labels. The model outputs a
n by 6 output tensor where the inferred labels are `Profanity`, `DatingAndSexting`, `Racist`, `Bullying`, `Other`,
`NoViolation`. `Other` consists of policy violation categories with low prevalence such as drugs and alcohol or
self-harm that are combined into a single category.
described in [our blog post](link to blog post here). A single output can have multiple labels.
The model outputs a n by 6 output tensor where the inferred labels are `Profanity`, `DatingAndSexting`, `Racist`,
`Bullying`, `Other`, `NoViolation`. `Other` consists of policy violation categories with low prevalence such as drugs
and alcohol or self-harm that are combined into a single category.


We evaluated this model on a data set with human annotated labels that contained a total of 9,795 samples with the class
distribution shown below. Note that we did not include the "other" category in this evaluation data set.
Expand All @@ -18,7 +21,9 @@ distribution shown below. Note that we did not include the "other" category in t
|NoViolation | 4185 | 9.93 | 42.73% |


If we set the same threshold across all classes and treat the model as a binary classifier across all 4 toxicity classes (`Profanity`, `DatingAndSexting`, `Racist`, `Bullying`), we get a binarized average precision of 94.48%. The precision recall curve is as shown below.
If we set the same threshold across all classes and treat the model as a binary classifier across all 4 toxicity classes
(`Profanity`, `DatingAndSexting`, `Racist`, `Bullying`), we get a binarized average precision of 94.48%. The precision
recall curve is as shown below.


<p align="center">
Expand All @@ -35,5 +40,6 @@ To run the inference file, please run the following command:
```
python inference.py --audio_file <your audio file path> --model_path <path to Huggingface model>
```
You can get the model weights either by downloading from the model releases page [here](https://github.com/Roblox/voice-safety-classifier/releases/tag/vs-classifier-v1), or from HuggingFace under `roblox/voice-safety-classifier`.
If `model_path` isn’t specified, the model will be loaded directly from HuggingFace.
You can get the model weights either by downloading from the model releases page [here](https://github.com/Roblox/voice-safety-classifier/releases/tag/vs-classifier-v1), or from HuggingFace under
[`roblox/voice-safety-classifier`](https://huggingface.co/Roblox/voice-safety-classifier). If `model_path` isn’t
specified, the model will be loaded directly from HuggingFace.

0 comments on commit 0eebdff

Please sign in to comment.