models AutoML Image Instance Segmentation
Automated Machine Learning, or AutoML, is a process that automates the repetitive and time-consuming tasks involved in developing machine learning models. This helps data scientists, analysts, and developers to create models more efficiently and with higher quality, resulting in increased productivity and scalability. AutoML Image Instance Segmentation enables you to train machine learning models to identify and separate individual objects within an image, including detecting the boundaries of each object and assigning a unique label to each object. The goal of instance segmentation is to produce a pixel-wise segmentation map of the image, where each pixel is assigned to a specific object instance.
With this functionality, you can:
- Directly use datasets coming from Azure Machine Learning data labeling
- Utilize labeled data to create image models without any training code.
- Enhance model performance by selecting the appropriate algorithm and fine-tuning the hyperparameters selecting the appropriate algorithm from a large selection of models or let AutoML find the best model for you.
- Either download or deploy the resulting model as a endpoint in Azure Machine Learning.
- Scale the operationalization process with the help of Azure Machine Learning's MLOps and ML Pipelines capabilities.
See How to train image models for more information.
To create computer vision models, it is necessary to provide labeled image data as input for model training. This data needs to be in the form of an MLTable, which can be created from training data in JSONL format. Please see documentation for JSONL Schema and consuming the same in MLTable.
You can initiate individual trials, manual sweeps, or automatic sweeps. It is suggested to begin with an automatic sweep to establish a baseline model. Afterward, you can experiment with individual trials using specific models and hyperparameter configurations. Lastly, manual sweeps can be used to explore multiple hyperparameter values near the more promising models and hyperparameter configurations. This three-step process (automatic sweep, individual trials, manual sweeps) helps avoid searching the entirety of the hyperparameter space, which grows exponentially with the number of hyperparameters.
For more information, see how to configure experiments
| Task | Use case | Dataset | Python sample (Notebook) | CLI with YAML |
|---|---|---|---|---|
| Image Instance Segmentation | Image instance segmentation | fridgeObjects | automl-image-instance-segmentation-task-fridge-items.ipynb | cli-automl-image-instance-segmentation-task-fridge-items.yml |
{
"input_data": {
"columns": [
"image"
],
"index": [0, 1],
"data": ["image1", "image2"]
}
}Note:
- "image1" and "image2" should be strings in
base64format.
[
{
"boxes": [
{
"box": {
"topX": 0.679,
"topY": 0.491,
"bottomX": 0.926,
"bottomY": 0.810
},
"label": "can",
"score": 0.992,
"polygon": [
[
0.82, 0.811, 0.771, 0.810, 0.758, 0.805, 0.741, 0.797, 0.735, 0.791, 0.718, 0.785, 0.715, 0.778, 0.706, 0.775, 0.696, 0.758, 0.695, 0.717, 0.698, 0.567, 0.705, 0.552, 0.706, 0.540, 0.725, 0.520, 0.735, 0.505, 0.745, 0.502, 0.755, 0.493
]
]
},
{
"box": {
"topX": 0.220,
"topY": 0.298,
"bottomX": 0.397,
"bottomY": 0.601
},
"label": "milk_bottle",
"score": 0.989,
"polygon": [
[
0.365, 0.602, 0.273, 0.602, 0.26, 0.595, 0.263, 0.588, 0.251, 0.546, 0.248, 0.501, 0.25, 0.485, 0.246, 0.478, 0.245, 0.463, 0.233, 0.442, 0.231, 0.43, 0.226, 0.423, 0.226, 0.408, 0.234, 0.385, 0.241, 0.371, 0.238, 0.345, 0.234, 0.335, 0.233, 0.325, 0.24, 0.305, 0.586, 0.38, 0.592, 0.375, 0.598, 0.365
]
]
},
{
"box": {
"topX": 0.433,
"topY": 0.280,
"bottomX": 0.621,
"bottomY": 0.679
},
"label": "water_bottle",
"score": 0.988,
"polygon": [
[
0.576, 0.680, 0.501, 0.680, 0.475, 0.675, 0.460, 0.625, 0.445, 0.630, 0.443, 0.572, 0.440, 0.560, 0.435, 0.515, 0.431, 0.501, 0.431, 0.433, 0.433, 0.426, 0.445, 0.417, 0.456, 0.407, 0.465, 0.381, 0.468, 0.327, 0.471, 0.318
]
]
}
]
}
]Note: Please refer to instance segmentation output data schema for more detail.
/markrcnn_resnet50_fpn_tao_vis.png)
Version: 1
license : bsd-3-clause task : image-segmentation
View in Studio: https://ml.azure.com/registries/azureml/models/AutoML-Image-Instance-Segmentation/version/1
License: bsd-3-clause
datasets: COCO
finetune-min-sku-spec: 4|1|28|176
finetune-recommended-sku: Standard_NC6s_v3
finetuning-tasks: image-object-detection
inference-min-sku-spec: 4|0|14|28
inference-recommended-sku: Standard_DS3_v2, Standard_D4a_v4, Standard_D4as_v4, Standard_DS4_v2, Standard_D8a_v4, Standard_D8as_v4, Standard_DS5_v2, Standard_D16a_v4, Standard_D16as_v4, Standard_D32a_v4, Standard_D32as_v4, Standard_D48a_v4, Standard_D48as_v4, Standard_D64a_v4, Standard_D64as_v4, Standard_D96a_v4, Standard_D96as_v4, Standard_FX4mds, Standard_F8s_v2, Standard_FX12mds, Standard_F16s_v2, Standard_F32s_v2, Standard_F48s_v2, Standard_F64s_v2, Standard_F72s_v2, Standard_FX24mds, Standard_FX36mds, Standard_FX48mds, Standard_E4s_v3, Standard_E8s_v3, Standard_E16s_v3, Standard_E32s_v3, Standard_E48s_v3, Standard_E64s_v3, Standard_NC4as_T4_v3, Standard_NC6s_v3, Standard_NC8as_T4_v3, Standard_NC12s_v3, Standard_NC16as_T4_v3, Standard_NC24s_v3, Standard_NC64as_T4_v3, Standard_NC24ads_A100_v4, Standard_NC48ads_A100_v4, Standard_NC96ads_A100_v4, Standard_ND96asr_v4, Standard_ND96amsr_A100_v4, Standard_ND40rs_v2