Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upgrade NLP and Vision AutoML components to fix the notebooks #3701

Merged
merged 6 commits into from
Dec 24, 2024
Merged

Upgrade NLP and Vision AutoML components to fix the notebooks #3701

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
3a3c4e2
upgrade components
iamrk04 Dec 23, 2024
68970ae
Merge branch 'main' into rk/automl_nb_fix
iamrk04 Dec 23, 2024
b765aca
update component version
iamrk04 Dec 23, 2024
dd62a80
Merge branch 'main' into rk/automl_nb_fix
iamrk04 Dec 23, 2024
a77ed12
update pipeline components
iamrk04 Dec 23, 2024
8cad85d
Merge branch 'rk/automl_nb_fix' of https://github.com/Azure/azureml-a…
iamrk04 Dec 23, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions assets/training/finetune_acft_common/components/validation/spec.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
$schema: https://azuremlschemas.azureedge.net/latest/commandComponent.schema.json
type: command

version: 0.0.7
version: 0.0.8
name: finetune_common_validation
display_name: Common Validation Component
description: Component to validate the finetune job against Validation Service

is_deterministic: True

environment: azureml://registries/azureml/environments/acpt-pytorch-2.2-cuda12.1/versions/23
environment: azureml://registries/azureml/environments/acpt-pytorch-2.2-cuda12.1/labels/latest

code: ../../src/validation

Expand Down
4 changes: 2 additions & 2 deletions assets/training/finetune_acft_hf_nlp/components/finetune/text_classification/spec.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
$schema: https://azuremlschemas.azureedge.net/latest/commandComponent.schema.json
name: text_classification_finetune
version: 0.0.65
version: 0.0.66
type: command

is_deterministic: false

display_name: Text Classification Finetune
description: Component to finetune Hugging Face pretrained models for text classification task. The component supports optimizations such as LoRA, Deepspeed and ONNXRuntime for performance enhancement. See [docs](https://aka.ms/azureml/components/text_classification_finetune) to learn more.

environment: azureml://registries/azureml/environments/acft-hf-nlp-gpu/versions/80
environment: azureml://registries/azureml/environments/acft-hf-nlp-gpu/labels/latest

code: ../../../src/finetune

Expand Down
4 changes: 2 additions & 2 deletions assets/training/finetune_acft_hf_nlp/components/finetune/token_classification/spec.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
$schema: https://azuremlschemas.azureedge.net/latest/commandComponent.schema.json
name: token_classification_finetune
version: 0.0.65
version: 0.0.66
type: command

is_deterministic: false

display_name: Token Classification Finetune
description: Component to finetune Hugging Face pretrained models for token classification task. The component supports optimizations such as LoRA, Deepspeed and ONNXRuntime for performance enhancement. See [docs](https://aka.ms/azureml/components/token_classification_finetune) to learn more.

environment: azureml://registries/azureml/environments/acft-hf-nlp-gpu/versions/80
environment: azureml://registries/azureml/environments/acft-hf-nlp-gpu/labels/latest

code: ../../../src/finetune

Expand Down
4 changes: 2 additions & 2 deletions assets/training/finetune_acft_hf_nlp/components/model_import/text_classification/spec.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
$schema: https://azuremlschemas.azureedge.net/latest/commandComponent.schema.json
name: text_classification_model_import
version: 0.0.65
version: 0.0.66
type: command

is_deterministic: True

display_name: Text Classification Model Import
description: Component to import PyTorch / MLFlow model. See [docs](https://aka.ms/azureml/components/text_classification_model_import) to learn more.

environment: azureml://registries/azureml/environments/acft-hf-nlp-gpu/versions/80
environment: azureml://registries/azureml/environments/acft-hf-nlp-gpu/labels/latest

code: ../../../src/model_selector

Expand Down
4 changes: 2 additions & 2 deletions assets/training/finetune_acft_hf_nlp/components/model_import/token_classification/spec.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
$schema: https://azuremlschemas.azureedge.net/latest/commandComponent.schema.json
name: token_classification_model_import
version: 0.0.65
version: 0.0.66
type: command

is_deterministic: True

display_name: Token Classification Model Import
description: Component to import PyTorch / MLFlow model. See [docs](https://aka.ms/azureml/components/token_classification_model_import) to learn more.

environment: azureml://registries/azureml/environments/acft-hf-nlp-gpu/versions/80
environment: azureml://registries/azureml/environments/acft-hf-nlp-gpu/labels/latest

code: ../../../src/model_selector

Expand Down
8 changes: 4 additions & 4 deletions assets/training/finetune_acft_hf_nlp/components/pipeline_components/nlp_multiclass/spec.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
$schema: https://azuremlschemas.azureedge.net/latest/pipelineComponent.schema.json
name: nlp_textclassification_multiclass
version: 0.0.2
version: 0.0.3
type: pipeline
display_name: PipelineComponent for AutoML NLP Multiclass TextClassification
description: Pipeline component for AutoML NLP Multiclass Text classification
Expand Down Expand Up @@ -224,13 +224,13 @@ outputs:
jobs:
model_import:
type: command
component: azureml:text_classification_model_import:0.0.12_nlp
component: azureml:text_classification_model_import:0.0.66
compute: ${{parent.inputs.compute_model_import}}
inputs:
huggingface_id: ${{parent.inputs.model_name}}
preprocess:
type: command
component: azureml:nlp_multiclass_datapreprocessing:0.0.1
component: azureml:nlp_multiclass_datapreprocessing:0.0.2
compute: ${{parent.inputs.compute_preprocess}}
inputs:
label_column_name: ${{parent.inputs.label_column_name}}
Expand All @@ -240,7 +240,7 @@ jobs:
enable_long_range_text: ${{parent.inputs.enable_long_range_text}}
finetune:
type: command
component: azureml:text_classification_finetune:0.0.14_nlp
component: azureml:text_classification_finetune:0.0.66
compute: ${{parent.inputs.compute_finetune}}
distribution:
type: pytorch
Expand Down
8 changes: 4 additions & 4 deletions assets/training/finetune_acft_hf_nlp/components/pipeline_components/nlp_multilabel/spec.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
$schema: https://azuremlschemas.azureedge.net/latest/pipelineComponent.schema.json
name: nlp_textclassification_multilabel
version: 0.0.2
version: 0.0.3
type: pipeline
display_name: PipelineComponent for AutoML NLP Multilabel TextClassification
description: Pipeline component for AutoML NLP Multilabel Text classification
Expand Down Expand Up @@ -224,13 +224,13 @@ outputs:
jobs:
model_import:
type: command
component: azureml:text_classification_model_import:0.0.12_nlp
component: azureml:text_classification_model_import:0.0.66
compute: ${{parent.inputs.compute_model_import}}
inputs:
huggingface_id: ${{parent.inputs.model_name}}
preprocess:
type: command
component: azureml:nlp_multilabel_datapreprocessing:0.0.1
component: azureml:nlp_multilabel_datapreprocessing:0.0.2
compute: ${{parent.inputs.compute_preprocess}}
inputs:
label_column_name: ${{parent.inputs.label_column_name}}
Expand All @@ -240,7 +240,7 @@ jobs:
enable_long_range_text: ${{parent.inputs.enable_long_range_text}}
finetune:
type: command
component: azureml:text_classification_finetune:0.0.14_nlp
component: azureml:text_classification_finetune:0.0.66
compute: ${{parent.inputs.compute_finetune}}
distribution:
type: pytorch
Expand Down
8 changes: 4 additions & 4 deletions assets/training/finetune_acft_hf_nlp/components/pipeline_components/nlp_ner/spec.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
$schema: https://azuremlschemas.azureedge.net/latest/pipelineComponent.schema.json
name: nlp_textclassification_ner
version: 0.0.2
version: 0.0.3
type: pipeline
display_name: PipelineComponent for AutoML NLP NER
description: Pipeline component for AutoML NLP NER
Expand Down Expand Up @@ -216,21 +216,21 @@ outputs:
jobs:
model_import:
type: command
component: azureml:token_classification_model_import:0.0.12_nlp
component: azureml:token_classification_model_import:0.0.66
compute: ${{parent.inputs.compute_model_import}}
inputs:
huggingface_id: ${{parent.inputs.model_name}}
preprocess:
type: command
component: azureml:nlp_ner_datapreprocessing:0.0.1
component: azureml:nlp_ner_datapreprocessing:0.0.2
compute: ${{parent.inputs.compute_preprocess}}
inputs:
train_file_path: ${{parent.inputs.training_data}}
valid_file_path: ${{parent.inputs.validation_data}}
model_selector_output: ${{parent.jobs.model_import.outputs.output_dir}}
finetune:
type: command
component: azureml:token_classification_finetune:0.0.14_nlp
component: azureml:token_classification_finetune:0.0.66
compute: ${{parent.inputs.compute_finetune}}
distribution:
type: pytorch
Expand Down
4 changes: 2 additions & 2 deletions assets/training/finetune_acft_hf_nlp/components/preprocess/nlp_multiclass/spec.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
$schema: https://azuremlschemas.azureedge.net/latest/commandComponent.schema.json
name: nlp_multiclass_datapreprocessing
version: 0.0.1
version: 0.0.2
type: command

is_deterministic: True

display_name: DataPreProcessing for AutoMLNLPMulticlass
description: Component to preprocess data for automl nlp multiclass classification task

environment: azureml://registries/azureml/environments/acft-hf-nlp-gpu/versions/16
environment: azureml://registries/azureml/environments/acft-hf-nlp-gpu/labels/latest

code: ../../../src/preprocess

Expand Down
4 changes: 2 additions & 2 deletions assets/training/finetune_acft_hf_nlp/components/preprocess/nlp_multilabel/spec.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
$schema: https://azuremlschemas.azureedge.net/latest/commandComponent.schema.json
name: nlp_multilabel_datapreprocessing
version: 0.0.1
version: 0.0.2
type: command

is_deterministic: True

display_name: DataPreProcessing for AutoMLNLPMultilabel
description: Component to preprocess data for automl nlp multilabel classification task

environment: azureml://registries/azureml/environments/acft-hf-nlp-gpu/versions/16
environment: azureml://registries/azureml/environments/acft-hf-nlp-gpu/labels/latest

code: ../../../src/preprocess

Expand Down
4 changes: 2 additions & 2 deletions assets/training/finetune_acft_hf_nlp/components/preprocess/nlp_ner/spec.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
$schema: https://azuremlschemas.azureedge.net/latest/commandComponent.schema.json
name: nlp_ner_datapreprocessing
version: 0.0.1
version: 0.0.2
type: command

is_deterministic: True

display_name: DataPreProcessing for AutoMLNLPNER
description: Component to preprocess data for automl nlp ner task

environment: azureml://registries/azureml/environments/acft-hf-nlp-gpu/versions/16
environment: azureml://registries/azureml/environments/acft-hf-nlp-gpu/labels/latest

code: ../../../src/preprocess

Expand Down
4 changes: 2 additions & 2 deletions assets/training/finetune_acft_image/components/finetune/hf_classification/spec.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
$schema: https://azuremlschemas.azureedge.net/latest/commandComponent.schema.json
type: command

version: 0.0.19
version: 0.0.20
name: transformers_image_classification_finetune
display_name: Image Classification HuggingFace Transformers Model Finetune
description: Component to finetune HuggingFace transformers models for image classification.

is_deterministic: false

environment: azureml://registries/azureml/environments/acft-transformers-image-gpu/versions/46
environment: azureml://registries/azureml/environments/acft-transformers-image-gpu/labels/latest

code: ../../../src/finetune

Expand Down
4 changes: 2 additions & 2 deletions assets/training/finetune_acft_image/components/finetune/mmd_od_is/spec.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
$schema: https://azuremlschemas.azureedge.net/latest/commandComponent.schema.json
type: command

version: 0.0.20
version: 0.0.21
name: mmdetection_image_objectdetection_instancesegmentation_finetune
display_name: Image Object Detection and Instance Segmentation MMDetection Model Finetune
description: Component to finetune MMDetection models for image object detection and instance segmentation.

is_deterministic: false

environment: azureml://registries/azureml/environments/acft-mmdetection-image-gpu/versions/48
environment: azureml://registries/azureml/environments/acft-mmdetection-image-gpu/labels/latest

code: ../../../src/finetune

Expand Down
4 changes: 2 additions & 2 deletions assets/training/finetune_acft_image/components/framework_selector/spec.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
$schema: https://azuremlschemas.azureedge.net/development/commandComponent.schema.json
type: command

version: 0.0.19
version: 0.0.20
name: image_framework_selector
display_name: Framework Selector for Image Tasks
description: Framework selector control flow component for image tasks

is_deterministic: true

environment: azureml://registries/azureml/environments/acpt-automl-image-framework-selector-gpu/versions/44
environment: azureml://registries/azureml/environments/acpt-automl-image-framework-selector-gpu/labels/latest

code: ../../src/framework_selector

Expand Down
4 changes: 2 additions & 2 deletions assets/training/finetune_acft_image/components/model_import/hf_classification/spec.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
$schema: https://azuremlschemas.azureedge.net/latest/commandComponent.schema.json
type: command

version: 0.0.19
version: 0.0.20
name: transformers_image_classification_model_import
display_name: Image Classification HuggingFace Transformers Model Import
description: Import PyTorch / MLflow model

is_deterministic: True

environment: azureml://registries/azureml/environments/acft-transformers-image-gpu/versions/46
environment: azureml://registries/azureml/environments/acft-transformers-image-gpu/labels/latest

code: ../../../src/model_selector

Expand Down
4 changes: 2 additions & 2 deletions assets/training/finetune_acft_image/components/model_import/mmd_od_is/spec.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
$schema: https://azuremlschemas.azureedge.net/latest/commandComponent.schema.json
type: command

version: 0.0.19
version: 0.0.20
name: mmdetection_image_objectdetection_instancesegmentation_model_import
display_name: Image Object Detection and Instance Segmentation MMDetection Model Import
description: Import PyTorch / MLflow model

is_deterministic: True

environment: azureml://registries/azureml/environments/acft-mmdetection-image-gpu/versions/46
environment: azureml://registries/azureml/environments/acft-mmdetection-image-gpu/labels/latest

code: ../../../src/model_selector

Expand Down
4 changes: 2 additions & 2 deletions assets/training/finetune_acft_image/components/model_output_selector/spec.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
$schema: https://azuremlschemas.azureedge.net/latest/commandComponent.schema.json
type: command

version: 0.0.18
version: 0.0.19
name: image_model_output_selector
display_name: Model output selector for image components
description: Model output selector control flow component for image tasks

is_deterministic: true

environment: azureml://registries/azureml/environments/acpt-automl-image-framework-selector-gpu/versions/44
environment: azureml://registries/azureml/environments/acpt-automl-image-framework-selector-gpu/labels/latest

code: ../../src/model_output_selector

Expand Down
14 changes: 7 additions & 7 deletions assets/training/finetune_acft_image/components/pipeline_components/classification/spec.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
$schema: https://azuremlschemas.azureedge.net/latest/pipelineComponent.schema.json
type: pipeline

version: 0.0.22
version: 0.0.23
name: image_classification_pipeline
display_name: Image Classification Pipeline
description: Pipeline component for image classification.
Expand Down Expand Up @@ -255,7 +255,7 @@ jobs:

finetune_common_validation:
type: command
component: azureml:finetune_common_validation:0.0.7
component: azureml:finetune_common_validation:0.0.8
compute: ${{parent.inputs.compute_model_import}}
inputs:
train_mltable_path: ${{parent.inputs.training_data}}
Expand All @@ -269,7 +269,7 @@ jobs:

framework_selector:
type: command
component: azureml:image_framework_selector:0.0.19
component: azureml:image_framework_selector:0.0.20
compute: ${{parent.inputs.compute_model_import}}
inputs:
task_type: 'image-classification'
Expand All @@ -278,7 +278,7 @@ jobs:

image_classification_runtime_component:
type: command
component: azureml:train_image_classification_model:0.0.12
component: azureml:train_image_classification_model:0.0.13
compute: ${{parent.inputs.compute_finetune}}
resources:
shm_size: '16g'
Expand Down Expand Up @@ -320,7 +320,7 @@ jobs:

hugging_face_model_import:
type: command
component: azureml:transformers_image_classification_model_import:0.0.19
component: azureml:transformers_image_classification_model_import:0.0.20
compute: ${{parent.inputs.compute_model_import}}
inputs:
model_family: 'HuggingFaceImage'
Expand All @@ -330,7 +330,7 @@ jobs:

hugging_face_finetune:
type: command
component: azureml:transformers_image_classification_finetune:0.0.19
component: azureml:transformers_image_classification_finetune:0.0.20
compute: ${{parent.inputs.compute_finetune}}
distribution:
type: pytorch
Expand Down Expand Up @@ -372,7 +372,7 @@ jobs:

output_selector:
type: command
component: azureml:image_model_output_selector:0.0.18
component: azureml:image_model_output_selector:0.0.19
compute: ${{parent.inputs.compute_model_import}}
inputs:
mlflow_model_t: ${{parent.jobs.image_classification_runtime_component.outputs.mlflow_model_folder}}
Expand Down
Loading
Loading