Skip to content

components model_prediction_with_container

github-actions[bot] edited this page Sep 27, 2024 · 4 revisions

Distributed Model Prediction

model_prediction_with_container

Overview

Optimized Distributed inference component for LLMs.

Version: 0.0.4

Tags

type : evaluation sub_type : subgraph Preview

View in Studio: https://ml.azure.com/registries/azureml/components/model_prediction_with_container/version/0.0.4

Inputs

model prediction

Name Description Type Default Optional Enum
task Task type string text-classification ['text-classification', 'text-classification-multilabel', 'text-named-entity-recognition', 'text-summarization', 'question-answering', 'text-translation', 'text-generation', 'fill-mask', 'chat-completion']
test_data Test Data uri_folder False
mlflow_model Mlflow model mlflow_model False
label_column_name Label column name in provided test dataset (Ex: label) string
input_column_names Input column names in provided test dataset (Ex : column1). Add comma delimited values in case of multiple input columns (Ex : column1,column2) string True
mini_batch_size integer 100 True
evaluation_config_params JSON Serialized string of evaluation_config string True

Outputs

Name Description Type
predictions uri_file
ground_truth uri_file
prediction_probabilities uri_file

TODO: Not implemented yet in this component

Name Description Type
performance_metadata Json File containing metadata for perf output uri_file

Environment

azureml://registries/azureml/environments/foundation-model-inference/labels/latest

Clone this wiki locally