Model function/purpose?

[stajdini]

Hello @jimth001 . Can this model be used to evaluate the formality level of short sentences? (3-6 words) I don't need any style transfer. I'm looking for an output like this:
"I wanna go" --> formality score : 0
"I want to go" --> formality score : 100

[jimth001]

Hello @stajdini ,
Thank you for your interest in our work and for reaching out. I apologize for the delay in my response.

The model proposed in our paper is designed for style transfer, specifically for transforming text between different formality levels. It does not directly evaluate the formality score of short sentences. However, the paper does mention a model that was trained on formal/informal data specifically for evaluating formality scores. This model might meet your requirements, as it was designed to assess the formality score of text.

It's important to note that this evaluation model was developed around five years ago, so it might be outdated by current standards. Given the rapid advancements in language models, I would recommend considering more recent approaches. Using a powerful Large Language Model (LLM) for this task or fine-tuning a state-of-the-art base model on formal/informal data could yield better results.

I hope this helps, and please feel free to reach out if you have any further questions!