Skip to content

Commit

Permalink
Update quickstart.ipynb (#2457)
Browse files Browse the repository at this point in the history
* Update quickstart.ipynb

* Update quickstart.ipynb

* Update quickstart.ipynb

* Update quickstart.ipynb

* Update quickstart.ipynb

* Update train-model.ipynb

* Update train-model.ipynb

* Update train-model.ipynb

* Update train-model.ipynb

* Update quickstart.ipynb

* Update train-model.ipynb

* Update pipeline.ipynb

* Update pipeline.ipynb

* Update pipeline.ipynb

* Update quickstart.ipynb

* Update train-model.ipynb

* Update quickstart.ipynb

* Update pipeline.ipynb

* Update train-model.ipynb

* Update train-model.ipynb

* Update quickstart.ipynb

* Update train-model.ipynb

* Update quickstart.ipynb

* Update train-model.ipynb

* Update e2e-object-classification-distributed-pytorch.ipynb

* Update e2e-object-classification-distributed-pytorch.ipynb

* Update e2e-object-classification-distributed-pytorch.ipynb

* Update e2e-object-classification-distributed-pytorch.ipynb

* Update e2e-object-classification-distributed-pytorch.ipynb

* Update e2e-object-classification-distributed-pytorch.ipynb

* Update e2e-object-classification-distributed-pytorch.ipynb

* Update e2e-object-classification-distributed-pytorch.ipynb

* Update e2e-object-classification-distributed-pytorch.ipynb

* Update e2e-object-classification-distributed-pytorch.ipynb

* Update e2e-object-classification-distributed-pytorch.ipynb

* Update e2e-object-classification-distributed-pytorch.ipynb

* Update e2e-object-classification-distributed-pytorch.ipynb

* Update e2e-object-classification-distributed-pytorch.ipynb

* Update e2e-object-classification-distributed-pytorch.ipynb

* Update pipeline.ipynb

* Update sklearn-diabetes.ipynb

* Update sklearn-diabetes.ipynb

* Update sklearn-diabetes.ipynb

* Update iris-scikit-learn.ipynb

* Update iris-scikit-learn.ipynb

* Update sklearn-diabetes.ipynb

* Update sklearn-mnist.ipynb

* Update debug-and-monitor.ipynb

* Update distributed-cifar10.ipynb

* Update distributed-cifar10.ipynb

* Update distributed-cifar10.ipynb

* Update distributed-cifar10.ipynb

* Update distributed-cifar10.ipynb

* Update objectdetectionAzureML.ipynb

* Update distributed-cifar10.ipynb

* Update pytorch-iris.ipynb

* Update tensorflow-mnist.ipynb

* Update tensorflow-mnist.ipynb

* Update tensorflow-mnist.ipynb

* Update debug-and-monitor.ipynb

* Update objectdetectionAzureML.ipynb

* Update distributed-cifar10.ipynb

* Update pytorch-iris.ipynb

* Update sklearn-diabetes.ipynb

* Update iris-scikit-learn.ipynb

* Update sklearn-mnist.ipynb

* Update tensorflow-mnist.ipynb

* Update distributed-cifar10.ipynb

* Update objectdetectionAzureML.ipynb

* Update tensorflow-mnist.ipynb

* Update tensorflow-mnist.ipynb

* Update e2e-object-classification-distributed-pytorch.ipynb

* Update distributed-cifar10.ipynb

* Update objectdetectionAzureML.ipynb

* Update tensorflow-mnist.ipynb

* Update automl-forecasting-recipe-univariate-run.ipynb

* Update automl-forecasting-recipe-univariate-run.ipynb

* Update automl-forecasting-recipe-univariate-run.ipynb

* Update automl-forecasting-recipe-univariate-run.ipynb

* Update automl-forecasting-recipe-univariate-run.ipynb

* Update automl-forecasting-recipe-univariate-run.ipynb

* Update automl-forecasting-recipe-univariate-run.ipynb

* Update automl-forecasting-recipe-univariate-run.ipynb

* Update automl-forecasting-recipe-univariate-run.ipynb

* Update automl-forecasting-recipe-univariate-run.ipynb

* Update tensorflow-mnist.ipynb

* Update e2e-object-classification-distributed-pytorch.ipynb

* Update auto-ml-forecasting-bike-share.ipynb

* Update auto-ml-forecasting-github-dau.ipynb

* Update auto-ml-forecasting-github-dau.ipynb

* Update automl-classification-task-bankmarketing-serverless.ipynb

* Update automl-forecasting-orange-juice-sales-mlflow.ipynb

* Update azureml-getting-started-studio.ipynb

* Update automl-regression-task-hardware-performance.ipynb

* Update automl-regression-task-hardware-performance.ipynb

* Update automl-nlp-text-ner-task.ipynb

* Update automl-nlp-text-ner-task.ipynb

* Update automl-nlp-text-ner-task.ipynb

* Update automl-nlp-text-ner-task.ipynb

* Update automl-nlp-multiclass-sentiment-mlflow.ipynb

* Update automl-nlp-multiclass-sentiment-mlflow.ipynb

* Update automl-nlp-multiclass-sentiment-mlflow.ipynb

* Update automl-nlp-multiclass-sentiment-mlflow.ipynb

* Update automl-nlp-multiclass-sentiment.ipynb

* Update automl-nlp-multilabel-paper-cat.ipynb

* Update automl-forecasting-task-energy-demand-advanced.ipynb

* Update automl-nlp-multiclass-sentiment-mlflow.ipynb

* Update automl-nlp-multilabel-paper-cat.ipynb

* Update automl-nlp-text-ner-task.ipynb

* Update automl-nlp-multiclass-sentiment-mlflow.ipynb

* Update automl-nlp-multiclass-sentiment.ipynb

* Update automl-nlp-multilabel-paper-cat.ipynb

* Update automl-nlp-text-ner-task.ipynb
  • Loading branch information
vijetajo authored Jul 22, 2023
1 parent 4ccfdf4 commit 67777f0
Show file tree
Hide file tree
Showing 24 changed files with 199 additions and 367 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -9,16 +9,15 @@
"**Requirements** - In order to benefit from this tutorial, you will need:\n",
"- A basic understanding of Machine Learning\n",
"- An Azure account with an active subscription. [Create an account for free](https://azure.microsoft.com/free/?WT.mc_id=A261C142F)\n",
"- An Azure ML workspace. [Check this notebook for creating a workspace](../../../resources/workspace/workspace.ipynb) \n",
"- This notebook leverages **serverless compute** to run the job. There is no need for user to create and manage compute. \n",
"- An Azure ML workspace. [Check this notebook for creating a workspace](../../../resources/workspace/workspace.ipynb) \n",
"- A python environment\n",
"- Installed Azure Machine Learning Python SDK v2 - [install instructions](../../../README.md) - check the getting started section\n",
"\n",
"\n",
"**Learning Objectives** - By the end of this tutorial, you should be able to:\n",
"- Connect to your AML workspace from the Python SDK\n",
"- Create an `AutoML classification Job` with the 'classification()' factory-fuction.\n",
"- Train the model using AmlCompute by submitting/running the AutoML training job\n",
"- Train the model using [serverless compute (preview)](https://learn.microsoft.com/azure/machine-learning/how-to-use-serverless-compute?view=azureml-api-2&tabs=python) by submitting/running the AutoML training job\n",
"- Obtaing the model and score predictions with it\n",
"- Leverage the auto generated training code and use it for retraining on an updated dataset\n",
"\n",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -37,15 +37,14 @@
"**Requirements** - In order to benefit from this tutorial, you will need:\n",
"- A basic understanding of Machine Learning\n",
"- An Azure account with an active subscription. [Create an account for free](https://azure.microsoft.com/free/?WT.mc_id=A261C142F)\n",
"- An Azure ML workspace. [Check this notebook for creating a workspace](../../../resources/workspace/workspace.ipynb) \n",
"- Serverless compute to run the job\n",
"- An Azure ML workspace. [Check this notebook for creating a workspace](../../../resources/workspace/workspace.ipynb) \n",
"- A python environment\n",
"- Installation instructions - [install instructions](../../../README.md)\n",
"\n",
"**Learning Objectives** - By the end of this tutorial, you should be able to:\n",
"- Connect to your AML workspace from the Python SDK\n",
"- Create an `AutoML time-series forecasting Job` with the 'forecasting()' factory-fuction\n",
"- Train the model using serverless compute by submitting/running the AutoML forecasting training job\n",
"- Train the model using [serverless compute (preview)](https://learn.microsoft.com/azure/machine-learning/how-to-use-serverless-compute?view=azureml-api-2&tabs=python) by submitting/running the AutoML forecasting training job\n",
"- Obtain the model and use it to generate forecast\n",
"\n",
"**Motivations** - This notebook explains how to setup and run an AutoML forecasting job. This is one of the nine ML-tasks supported by AutoML. Other ML-tasks are 'regression', 'classification', 'image classification', 'image object detection', 'nlp text classification', etc.\n",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@
"**Learning Objectives** - By the end of this tutorial, you should be able to:\n",
"- Connect to your AML workspace from the Python SDK\n",
"- Create an `AutoML time-series forecasting Job` with the 'forecasting()' factory-fuction.\n",
"- Train the model using AmlCompute by submitting/running the AutoML forecasting training job\n",
"- Train the model using [serverless compute (preview)](https://learn.microsoft.com/azure/machine-learning/how-to-use-serverless-compute?view=azureml-api-2&tabs=python) by submitting/running the AutoML forecasting training job\n",
"- Obtaing the model and score predictions with it\n",
"\n",
"**Motivations** - This notebook explains how to setup and run an AutoML forecasting job. This is one of the nine ML-tasks supported by AutoML. Other ML-tasks are 'regression', 'classification', 'image classification', 'image object detection', 'nlp text classification', etc.\n",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@
"**Learning Objectives** - By the end of this tutorial, you should be able to:\n",
"- Connect to your AML workspace from the Python SDK\n",
"- Create an `AutoML time-series forecasting Job` with the 'forecasting()' factory-fuction\n",
"- Train the model using AmlCompute by submitting/running the AutoML forecasting training job\n",
"- Train the model using [serverless compute (preview)](https://learn.microsoft.com/azure/machine-learning/how-to-use-serverless-compute?view=azureml-api-2&tabs=python) by submitting/running the AutoML forecasting training job\n",
"- Obtain the model and use it to generate forecast\n",
"\n"
]
Expand Down Expand Up @@ -95,6 +95,7 @@
"from azure.ai.ml.constants import AssetTypes, InputOutputModes\n",
"from azure.ai.ml import automl\n",
"from azure.ai.ml import Input\n",
"from azure.ai.ml.entities import ResourceConfiguration\n",
"\n",
"import json\n",
"import pandas as pd\n",
Expand All @@ -112,9 +113,8 @@
"\n",
"As part of the setup you have already created a Workspace. To connect to a workspace, we need identifier parameters - a subscription, resource group and workspace name. We will use these details in the `MLClient` from `azure.ai.ml` to get a handle to the required Azure Machine Learning workspace. We use the default [default azure authentication](https://docs.microsoft.com/en-us/python/api/azure-identity/azure.identity.defaultazurecredential?view=azure-python) for this tutorial. Check the [configuration notebook](../../configuration.ipynb) for more details on how to configure credentials and connect to a workspace.\n",
"\n",
" You will also need to create a compute target for your AutoML run. In this tutorial, you create AmlCompute as your training compute resource.\n",
"\n",
"<b>Note </b>that if you have an AzureML Data Scientist role, you will not have permission to create compute resources. Talk to your workspace or IT admin to create the compute targets described in this section, if they do not already exist."
" You will also need to create a compute target for your AutoML run. In this tutorial, you will use serverless compute (preview) as your training compute resource.\n",
"\n"
]
},
{
Expand Down Expand Up @@ -563,52 +563,6 @@
"\"\"\""
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"## 3. Create or Attach existing AmlCompute\n",
"\n",
"[Azure Machine Learning Compute](https://docs.microsoft.com/en-us/azure/machine-learning/service/how-to-set-up-training-targets#amlcompute) is a managed-compute infrastructure that allows the user to easily create a single or multi-node compute. In this tutorial, you will create and an AmlCompute cluster as your training compute resource.\n",
"\n",
"<b>Creation of AmlCompute takes approximately 5 minutes.</b>\n",
"\n",
"If the AmlCompute with that name is already in your workspace this code will skip the creation process.\n",
"As with other Azure services, there are limits on certain resources (e.g. AmlCompute) associated with the Azure Machine Learning service. Please read [this article](https://docs.microsoft.com/en-us/azure/machine-learning/service/how-to-manage-quotas) on the default limits and how to request more quota."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"gather": {
"logged": 1684500104600
}
},
"outputs": [],
"source": [
"from azure.core.exceptions import ResourceNotFoundError\n",
"from azure.ai.ml.entities import AmlCompute\n",
"\n",
"cluster_name = \"recipe-cluster\"\n",
"\n",
"try:\n",
" # Retrieve an already attached Azure Machine Learning Compute.\n",
" compute = ml_client.compute.get(cluster_name)\n",
"except ResourceNotFoundError as e:\n",
" compute = AmlCompute(\n",
" name=cluster_name,\n",
" size=\"STANDARD_DS12_V2\",\n",
" type=\"amlcompute\",\n",
" min_instances=0,\n",
" max_instances=4,\n",
" idle_time_before_scale_down=120,\n",
" )\n",
" poller = ml_client.begin_create_or_update(compute)\n",
" poller.wait()"
]
},
{
"attachments": {},
"cell_type": "markdown",
Expand All @@ -629,7 +583,7 @@
"|**target_column_name**|The name of the label column.|\n",
"|**primary_metric**|This is the metric that you want to optimize.<br> Forecasting supports the following primary metrics <br><i>spearman_correlation</i><br><i>normalized_root_mean_squared_error</i><br><i>r2_score</i><br><i>normalized_mean_absolute_error</i>|\n",
"|**training_data**|The training data to be used for this experiment. You can use a registered MLTable in the workspace using the format `<mltable_name>:<version>` OR you can use a local file or folder as a MLTable. For e.g `Input(mltable='my_mltable:1')` OR `Input(mltable=MLTable(local_path=\"./data\"))` The parameter 'training_data' must always be provided.|\n",
"|**compute**|The compute on which the AutoML job will run. In this example we are using a compute called 'cpu-cluster' present in the workspace. You can replace it with any other compute in the workspace.|\n",
"|**compute**|The compute on which the AutoML job will run. In this example we are using serverless compute so compute does not need to be specified. You can replace it with a compute cluster in the workspace.|\n",
"|**n_cross_validations**|Number of cross-validation folds to use for model/pipeline selection. The default value is \"auto\", in which case AutoMl determines the number of cross-validations automatically, if a validation set is not provided. Or, users could specify an integer value.|\n",
"|**name**|The name of the Job/Run. This is an optional property. If not specified, a random name will be generated.\n",
"|**experiment_name**|The name of the Experiment. An Experiment is like a folder with multiple runs in Azure ML Workspace that should be related to the same logical machine learning experiment. For example, if a user runs this notebook multiple times, there will be multiple runs associated with the same Experiment name.|\n",
Expand Down Expand Up @@ -715,7 +669,6 @@
"# Create the AutoML forecasting job with the related factory-function.\n",
"\n",
"forecasting_job = automl.forecasting(\n",
" compute=cluster_name,\n",
" experiment_name=exp_name,\n",
" training_data=my_training_data_input,\n",
" target_column_name=TARGET_COLNAME,\n",
Expand Down Expand Up @@ -753,7 +706,8 @@
")\n",
"\n",
"# Training properties are optional\n",
"forecasting_job.set_training(blocked_training_algorithms=BLOCKED_MODELS)"
"forecasting_job.set_training(blocked_training_algorithms=BLOCKED_MODELS)\n",
"forecasting_job.resources = ResourceConfiguration(instance_count=4)"
]
},
{
Expand Down Expand Up @@ -1089,7 +1043,38 @@
"source": [
"Now that we have retrieved the best pipeline/model, it can be used to make predictions on test data. We will do batch inferencing on the test dataset which must have the same schema as training dataset.\n",
"\n",
"The inference will run on a remote compute. In this example, it will re-use the training compute."
"The inference will run on a remote compute. In this example, we will create compute cluster."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"gather": {
"logged": 1684500104600
}
},
"outputs": [],
"source": [
"from azure.core.exceptions import ResourceNotFoundError\n",
"from azure.ai.ml.entities import AmlCompute\n",
"\n",
"cluster_name = \"recipe-cluster\"\n",
"\n",
"try:\n",
" # Retrieve an already attached Azure Machine Learning Compute.\n",
" compute = ml_client.compute.get(cluster_name)\n",
"except ResourceNotFoundError as e:\n",
" compute = AmlCompute(\n",
" name=cluster_name,\n",
" size=\"STANDARD_DS12_V2\",\n",
" type=\"amlcompute\",\n",
" min_instances=0,\n",
" max_instances=4,\n",
" idle_time_before_scale_down=120,\n",
" )\n",
" poller = ml_client.begin_create_or_update(compute)\n",
" poller.wait()"
]
},
{
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@
"**Learning Objectives** - By the end of this tutorial, you should be able to:\n",
"- Connect to your AML workspace from the Python SDK\n",
"- Create an `AutoML time-series forecasting Job` with the 'forecasting()' factory-fuction\n",
"- Train the model using AmlCompute by submitting/running the AutoML forecasting training job\n",
"- Train the model using [serverless compute (preview)](https://learn.microsoft.com/azure/machine-learning/how-to-use-serverless-compute?view=azureml-api-2&tabs=python) by submitting/running the AutoML forecasting training job\n",
"- Obtain the model and use it to generate forecast\n",
"\n",
"**Motivations** - This notebook explains how to setup and run an AutoML forecasting job. This is one of the nine ML-tasks supported by AutoML. Other ML-tasks are 'regression', 'classification', 'image classification', 'image object detection', 'nlp text classification', etc.\n",
Expand Down Expand Up @@ -312,46 +312,6 @@
"- https://learn.microsoft.com/en-us/azure/machine-learning/how-to-create-data-assets?tabs=Python-SDK covers how to work with them in the v2 CLI/SDK."
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"# 3 Create or Attach existing AmlCompute.\n",
"[Azure Machine Learning Compute](https://docs.microsoft.com/en-us/azure/machine-learning/service/how-to-set-up-training-targets#amlcompute) is a managed-compute infrastructure that allows the user to easily create a single or multi-node compute. In this tutorial, you will create and an AmlCompute cluster as your training compute resource.\n",
"\n",
"#### Creation of AmlCompute takes approximately 5 minutes. \n",
"If the AmlCompute with that name is already in your workspace this code will skip the creation process.\n",
"As with other Azure services, there are limits on certain resources (e.g. AmlCompute) associated with the Azure Machine Learning service. Please read [this article](https://docs.microsoft.com/en-us/azure/machine-learning/service/how-to-manage-quotas) on the default limits and how to request more quota."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from azure.core.exceptions import ResourceNotFoundError\n",
"from azure.ai.ml.entities import AmlCompute\n",
"\n",
"cluster_name = \"bike-share-v2\"\n",
"\n",
"try:\n",
" # Retrieve an already attached Azure Machine Learning Compute.\n",
" compute = ml_client.compute.get(cluster_name)\n",
"except ResourceNotFoundError as e:\n",
" compute = AmlCompute(\n",
" name=cluster_name,\n",
" size=\"STANDARD_DS12_V2\",\n",
" type=\"amlcompute\",\n",
" min_instances=0,\n",
" max_instances=4,\n",
" idle_time_before_scale_down=120,\n",
" )\n",
" poller = ml_client.begin_create_or_update(compute)\n",
" poller.wait()"
]
},
{
"attachments": {},
"cell_type": "markdown",
Expand All @@ -371,7 +331,7 @@
"|**target_column_name**|The name of the label column.|\n",
"|**primary_metric**|This is the metric that you want to optimize.<br> Forecasting supports the following primary metrics <br><i>spearman_correlation</i><br><i>normalized_root_mean_squared_error</i><br><i>r2_score</i><br><i>normalized_mean_absolute_error</i>|\n",
"|**training_data**|The training data to be used for this experiment. You can use a registered MLTable in the workspace using the format `<mltable_name>:<version>` OR you can use a local file or folder as a MLTable. For e.g `Input(mltable='my_mltable:1')` OR `Input(mltable=MLTable(local_path=\"./data\"))` The parameter 'training_data' must always be provided.|\n",
"|**compute**|The compute on which the AutoML job will run. In this example we are using a compute called 'cpu-cluster' present in the workspace. You can replace it with any other compute in the workspace.|\n",
"|**compute**|The compute on which the AutoML job will run. In this example we are using serverless compute. You can replace it with a compute cluster in the workspace.|\n",
"|**n_cross_validations**|Number of cross-validation folds to use for model/pipeline selection. This can be set to \"auto\", in which case AutoMl determines the number of cross-validations automatically, if a validation set is not provided. Or, users could specify an integer value.|\n",
"|**name**|The name of the Job/Run. This is an optional property. If not specified, a random name will be generated.\n",
"|**experiment_name**|The name of the Experiment. An Experiment is like a folder with multiple runs in Azure ML Workspace that should be related to the same logical machine learning experiment. For example, if a user runs this notebook multiple times, there will be multiple runs associated with the same Experiment name.|\n",
Expand Down Expand Up @@ -462,8 +422,7 @@
"outputs": [],
"source": [
"# Create the AutoML forecasting job with the related factory-function. Force the target column, to be integer type (To be added in phase 2)\n",
"forecasting_job = automl.forecasting(\n",
" compute=\"bike-share-v2\",\n",
"forecasting_job = automl.forecasting(\n",
" experiment_name=exp_name,\n",
" training_data=my_training_data_input,\n",
" target_column_name=target_column_name,\n",
Expand Down Expand Up @@ -797,7 +756,34 @@
"\n",
"Now that we have retrieved the best pipeline/model, it can be used to make predictions on test data. We will do batch inferencing on the test dataset which must have the same schema as training dataset.\n",
"\n",
"The inference will run on a remote compute. In this example, it will re-use the training compute."
"The inference will run on a remote compute. In this example, it will create compute cluster."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from azure.core.exceptions import ResourceNotFoundError\n",
"from azure.ai.ml.entities import AmlCompute\n",
"\n",
"cluster_name = \"bike-share-v2\"\n",
"\n",
"try:\n",
" # Retrieve an already attached Azure Machine Learning Compute.\n",
" compute = ml_client.compute.get(cluster_name)\n",
"except ResourceNotFoundError as e:\n",
" compute = AmlCompute(\n",
" name=cluster_name,\n",
" size=\"STANDARD_DS12_V2\",\n",
" type=\"amlcompute\",\n",
" min_instances=0,\n",
" max_instances=4,\n",
" idle_time_before_scale_down=120,\n",
" )\n",
" poller = ml_client.begin_create_or_update(compute)\n",
" poller.wait()"
]
},
{
Expand Down
Loading

0 comments on commit 67777f0

Please sign in to comment.