-
Notifications
You must be signed in to change notification settings - Fork 124
components benchmark_embedding_model
github-actions[bot] edited this page Oct 23, 2024
·
4 revisions
Component for benchmarking an embedding model via MTEB.
Version: 0.0.2
View in Studio: https://ml.azure.com/registries/azureml/components/benchmark_embedding_model/version/0.0.2
| Name | Description | Type | Default | Optional | Enum |
|---|---|---|---|---|---|
| endpoint_url | For AOAI, the base endpoint url. For OAI, this will be ignored. For OSS_MaaS, the target url. | string | True | ||
| deployment_type | The deployment type. | string | False | ['AOAI', 'OAI', 'OSS_MaaS'] | |
| deployment_name | For AOAI, the deployment name. For OAI, the model name. For OSS_MaaS, this wil be ignored. | string | True | ||
| connections_name | Used for authenticating endpoint. | string | False | ||
| tasks | Comma separated string denoting the tasks to benchmark the model on. | string | True | ||
| task_types | Comma separated string denoting the task type to benchmark the model on. Choose from the following task types: BitextMining, Classification, Clustering, PairClassification, Reranking, Retrieval, STS, Summarization. | string | True | ||
| task_langs | Comma separated string denoting the task languages to benchmark the model on. | string | True | ||
| preset | Choose from one of the presets for benchmarking. | string | None | False | ['None', 'mteb_main_en'] |
| Name | Description | Type |
|---|---|---|
| metrics | Directory where the benchmark metrics will be saved. | uri_folder |
azureml://registries/azureml/environments/model-evaluation/labels/latest