components generation_safety_quality_signal_monitor

Generation Safety & Quality - Signal Monitor

generation_safety_quality_signal_monitor

Overview

Computes the content generation safety metrics over LLM outputs.

Version: 0.5.22

View in Studio: https://ml.azure.com/registries/azureml/components/generation_safety_quality_signal_monitor/version/0.5.22

Inputs

Name	Description	Type	Default	Optional	Enum
monitor_name		string
signal_name		string
production_data		mltable
metric_names	a comma-separated list of metric names to compute	string		False
model_deployment_name	model name, e.g., name of the deployment for Azure OpenAI	string		False
sample_rate	sample rate for the input dataset, should be greater than 0 and at most 1	number		True
prompt_column_name		string	prompt	True
completion_column_name		string	completion	True
ground_truth_column_name		string	ground_truth	True
context_column_name		string	context	True
groundedness_rating_threshold		integer	4	True
similarity_rating_threshold		integer	4	True
relevance_rating_threshold		integer	4	True
fluency_rating_threshold		integer	4	True
coherence_rating_threshold		integer	4	True
thresholds	a comma-separated dictionary of threshold:value for each metric	string		True
groundedness_passrate_threshold		number	0.7	True
similarity_passrate_threshold		number	0.7	True
relevance_passrate_threshold		number	0.7	True
fluency_passrate_threshold		number	0.7	True
coherence_passrate_threshold		number	0.7	True
workspace_connection_arm_id		string		False
monitor_current_time		string
notification_emails		string		True
instance_type		string	standard_e4s_v3	True

Outputs

Name	Description	Type
signal_output		uri_folder
signal_scored_data		mltable