models qna groundedness eval

qna-groundedness-eval

Overview

The "QnA Groundedness Evaluation" is a model to evaluate the Q&A Retrieval Augmented Generation systems by leveraging the state-of-the-art Large Language Models (LLM) to measure the quality and safety of your responses. Utilizing GPT-3.5 as the Language Model to assist with measurements aims to achieve a high agreement with human evaluations compared to traditional mathematical measurements.

Inference samples

Inference type	CLI	VS Code Extension
Real time	deploy-promptflow-model-cli-example	deploy-promptflow-model-vscode-extension-example
Batch	N/A	N/A

Sample inputs and outputs (for real-time inference)

Sample input

{
    "inputs": {
        "question": "What feeds all the fixtures in low voltage tracks instead of each light having a line-to-low voltage transformer?",
        "context": "Track lighting, invented by Lightolier, was popular at one period of time because it was much easier to install than recessed lighting, and individual fixtures are decorative and can be easily aimed at a wall. It has regained some popularity recently in low-voltage tracks, which often look nothing like their predecessors because they do not have the safety issues that line-voltage systems have, and are therefore less bulky and more ornamental in themselves. A master transformer feeds all of the fixtures on the track or rod with 12 or 24 volts, instead of each light fixture having its own line-to-low voltage transformer. There are traditional spots and floods, as well as other small hanging fixtures. A modified version of this is cable lighting, where lights are hung from or clipped to bare metal cables under tension",
        "answer": "The main transformer is the object that feeds all the fixtures in low voltage tracks."
    }
}

Sample output

{
    "outputs": {
        "gpt_groundedness": 5
    }
}

Version: 3

View in Studio: https://ml.azure.com/registries/azureml/models/qna-groundedness-eval/version/3

Properties

is-promptflow: True

azureml.promptflow.section: gallery

azureml.promptflow.type: evaluate

azureml.promptflow.name: QnA Groundedness Evaluation

azureml.promptflow.description: Compute the groundedness of the answer for the given question based on the context.

inference-min-sku-spec: 2|0|14|28

inference-recommended-sku: Standard_DS3_v2