- 1. Introduction
- 2. Quick Start
- 3. Training, Evaluation and Inference
- 4. Model Compression
- 5. SHAS
- 6. Inference Deployment
This case provides a way for users to quickly build a lightweight, high-precision and practical classification model of vehicle attribute using PaddleClas PULC (Practical Ultra Lightweight image Classification). The model can be widely used in Vehicle identification, road monitoring and other scenarios.
The following table lists the relevant indicators of the model. The first three lines means that using Res2Net200_vd_26w_4s, ResNet50 and MobileNetV3_small_x0_35 as the backbone to training. The fourth to seventh lines means that the backbone is replaced by PPLCNet, additional use of EDA strategy and additional use of EDA strategy and SKL-UGI knowledge distillation strategy.
| Backbone | mA（%） | Latency(ms) | Size(M) | Training Strategy |
|---|---|---|---|---|
| Res2Net200_vd_26w_4s | 91.36 | 79.46 | 293 | using ImageNet pretrained |
| ResNet50 | 89.98 | 12.83 | 92 | using ImageNet pretrained |
| MobileNetV3_small_x0_35 | 87.41 | 2.91 | 2.8 | using ImageNet pretrained |
| PPLCNet_x1_0 | 89.57 | 2.36 | 7.2 | using ImageNet pretrained |
| PPLCNet_x1_0 | 90.07 | 2.36 | 7.2 | using SSLD pretrained |
| PPLCNet_x1_0 | 90.59 | 2.36 | 7.2 | using SSLD pretrained + EDA strategy |
| PPLCNet_x1_0 | 90.81 | 2.36 | 7.2 | using SSLD pretrained + EDA strategy + SKL-UGI knowledge distillation strategy |
It can be seen from the table that the ma metric is higher when the backbone is Res2Net200_vd_26w_4s, but the inference speed is slower. After replacing the backbone with the lightweight model MobileNetV3_small_x0_35, the speed can be greatly improved, but the ma metric drops significantly. When the backbone is replaced by PPLCNet_x1_0, the ma metric is increased by 2 percentage points, and the speed is also increased by about 23%. On this basis, after using the SSLD pre-training model, the ma metric can be improved by about 0.5 percentage points without changing the inference speed. Further, when the EDA strategy is integrated, the ma metric can be improved by another 0.52 percentage points. Finally, using After SKL-UGI knowledge distillation, the ma metric can continue to improve by 0.23 percentage points. At this time, the ma metric of PPLCNet_x1_0 is only 0.55 percentage points away from Res2Net200_vd_26w_4s, but it is 32 times faster. The training method and deployment instructions of PULC will be introduced in detail below.
Note:
- The Latency is tested on Intel(R) Xeon(R) Gold 6148 CPU @ 2.40GHz. The MKLDNN is enabled and the number of threads is 10.
- About PP-LCNet, please refer to PP-LCNet Introduction and PP-LCNet Paper.
- Run the following command to install if CUDA9 or CUDA10 is available.
python3 -m pip install paddlepaddle-gpu -i https://mirror.baidu.com/pypi/simple- Run the following command to install if GPU device is unavailable.
python3 -m pip install paddlepaddle -i https://mirror.baidu.com/pypi/simplePlease refer to PaddlePaddle Installation for more information about installation, for examples other versions.
The command of PaddleClas installation as bellow:
pip3 install paddleclasFirst, please click here to download and unzip to get the test demo images.
- Prediction with CLI
paddleclas --model_name=vehicle_attribute --infer_imgs=pulc_demo_imgs/vehicle_attribute/0002_c002_00030670_0.jpgResults:
>>> result
attributes: Color: (yellow, prob: 0.9893476963043213), Type: (hatchback, prob: 0.9734097719192505), output: [1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0], filename: pulc_demo_imgs/vehicle_attribute/0002_c002_00030670_0.jpg
Predict complete!
Note: If you want to test other images, only need to specify the --infer_imgs argument, and the directory containing images is also supported.
- Prediction in Python
import paddleclas
model = paddleclas.PaddleClas(model_name="vehicle_attribute")
result = model.predict(input_data="pulc_demo_imgs/vehicle_attribute/0002_c002_00030670_0.jpg")
print(next(result))Note: The result returned by model.predict() is a generator, so you need to use the next() function to call it or for loop to loop it. And it will predict with batch_size size batch and return the prediction results when called. The default batch_size is 1, and you also specify the batch_size when instantiating, such as model = paddleclas.PaddleClas(model_name="vehicle_attribute", batch_size=2). The result of demo above:
>>> result
[{'attributes': 'Color: (yellow, prob: 0.9893476963043213), Type: (hatchback, prob: 0.9734097719192505)', 'output': [1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0], 'filename': 'pulc_demo_imgs/vehicle_attribute/0002_c002_00030670_0.jpg'}]
Note: The value index of output is 0-9 to indicate the color attribute, and the corresponding colors are: yellow, orange, green, gray, red, blue, white, golden, brown, black; the index is 10-18 to represent the model attributes, the corresponding models are sedan, suv, van, hatchback, mpv, pickup, bus, truck, estate.
Please refer to Installation to get the description about installation.
The data used in this case is the pa100k dataset.
Part of the data visualization is shown below.
First, apply for and download data from VeRi dataset official website, put it in the dataset directory of PaddleClas, the dataset directory name is VeRi , use the following command to enter the folder.
cd PaddleClas/dataset/VeRi/Then use the following code to convert the label (you can execute the following command in the python terminal, or you can write it to a file and run the file using python3 convert.py).
import os
from xml.dom.minidom import parse
vehicleids = []
def convert_annotation(input_fp, output_fp):
in_file = open(input_fp)
list_file = open(output_fp, 'w')
tree = parse(in_file)
root = tree.documentElement
for item in root.getElementsByTagName("Item"):
label = ['0', '0', '0', '0', '0', '0', '0', '0', '0', '0', '0', '0', '0', '0', '0', '0', '0', '0', '0']
if item.hasAttribute("imageName"):
name = item.getAttribute("imageName")
if item.hasAttribute("vehicleID"):
vehicleid = item.getAttribute("vehicleID")
if vehicleid not in vehicleids :
vehicleids.append(vehicleid)
vid = vehicleids.index(vehicleid)
if item.hasAttribute("colorID"):
colorid = int (item.getAttribute("colorID"))
label[colorid-1] = '1'
if item.hasAttribute("typeID"):
typeid = int (item.getAttribute("typeID"))
label[typeid+9] = '1'
label = ','.join(label)
list_file.write(os.path.join('image_train', name) + "\t" + label + "\n")
list_file.close()
convert_annotation('train_label.xml', 'train_list.txt') #imagename vehiclenum colorid typeid
convert_annotation('test_label.xml', 'test_list.txt')After executing the above command, the VeRi directory has the following data:
VeRi
├── image_train
│ ├── 0001_c001_00016450_0.jpg
│ ├── 0001_c001_00016460_0.jpg
│ ├── 0001_c001_00016470_0.jpg
...
├── image_test
│ ├── 0002_c002_00030600_0.jpg
│ ├── 0002_c002_00030605_1.jpg
│ ├── 0002_c002_00030615_1.jpg
...
...
├── train_list.txt
├── test_list.txt
├── train_label.xml
├── test_label.xml
where train/ and test/ are the training set and validation set, respectively. train_list.txt and test_list.txt are the converted label files for training and validation sets, respectively.
The details of training config in ./ppcls/configs/PULC/vehicle_attribute/PPLCNet_x1_0.yaml. The command about training as follows:
export CUDA_VISIBLE_DEVICES=0,1,2,3
python3 -m paddle.distributed.launch \
--gpus="0,1,2,3" \
tools/train.py \
-c ./ppcls/configs/PULC/vehicle_attribute/PPLCNet_x1_0.yamlThe best metric for the validation set is around 90.59% (the dataset is small and generally fluctuates around 0.3%).
After training, you can use the following commands to evaluate the model.
python3 tools/eval.py \
-c ./ppcls/configs/PULC/vehicle_attribute/PPLCNet_x1_0.yaml \
-o Global.pretrained_model="output/PPLCNet_x1_0/best_model"Among the above command, the argument -o Global.pretrained_model="output/PPLCNet_x1_0/best_model" specify the path of the best model weight file. You can specify other path if needed.
After training, you can use the model that trained to infer. Command is as follow:
python3 tools/infer.py \
-c ./ppcls/configs/PULC/vehicle_attribute/PPLCNet_x1_0.yaml \
-o Global.pretrained_model=output/DistillationModel/best_modelThe results:
[{'attr': 'Color: (yellow, prob: 0.9893478155136108), Type: (hatchback, prob: 0.9734100103378296)', 'pred': [1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0], 'file_name': './deploy/images/PULC/vehicle_attribute/0002_c002_00030670_0.jpg'}]
Note:
- Among the above command, argument
-o Global.pretrained_model="output/PPLCNet_x1_0/best_model"specify the path of the best model weight file. You can specify other path if needed. - The default test image is
./deploy/images/PULC/vehicle_attribute/0002_c002_00030670_0.jpg. And you can test other image, only need to specify the argument-o Infer.infer_imgs=path_to_test_image.
SKL-UGI is a simple but effective knowledge distillation algrithem proposed by PaddleClas.
Training the teacher model with hyperparameters specified in ppcls/configs/PULC/vehicle_attribute/PPLCNet_x1_0.yaml. The command is as follow:
export CUDA_VISIBLE_DEVICES=0,1,2,3
python3 -m paddle.distributed.launch \
--gpus="0,1,2,3" \
tools/train.py \
-c ./ppcls/configs/PULC/vehicle_attribute/PPLCNet_x1_0.yaml \
-o Arch.name=ResNet101_vdThe best metric for the validation set is around 91.60%. The best teacher model weight would be saved in file output/ResNet101_vd/best_model.pdparams.
The training strategy, specified in training config file ppcls/configs/PULC/vehicle_attribute/PPLCNet_x1_0_distillation.yaml, the teacher model is ResNet101_vd, the student model is PPLCNet_x1_0. The command is as follow:
export CUDA_VISIBLE_DEVICES=0,1,2,3
python3 -m paddle.distributed.launch \
--gpus="0,1,2,3" \
tools/train.py \
-c ./ppcls/configs/PULC/vehicle_attribute/PPLCNet_x1_0_distillation.yaml \
-o Arch.models.0.Teacher.pretrained=output/ResNet101_vd/best_modelThe best metric for the validation set is around 90.81%. The best student model weight would be saved in file output/DistillationModel/best_model_student.pdparams.
The hyperparameters used by 3.2 section and 4.1 section are according by Hyperparameters Searching in PaddleClas. If you want to get better results on your own dataset, you can refer to Hyperparameters Searching to get better hyperparameters.
Note: This section is optional. Because the search process will take a long time, you can selectively run according to your specific. If not replace the dataset, you can ignore this section.
Paddle Inference is the original Inference Library of the PaddlePaddle, provides high-performance inference for server deployment. And compared with directly based on the pretrained model, Paddle Inference can use tools to accelerate prediction, so as to achieve better inference performance. Please refer to Paddle Inference for more information.
Paddle Inference need Paddle Inference Model to predict. Two process provided to get Paddle Inference Model. If want to use the provided by PaddleClas, you can download directly, click Downloading Inference Model.
The command about exporting Paddle Inference Model is as follow:
python3 tools/export_model.py \
-c ./ppcls/configs/PULC/vehicle_attribute/PPLCNet_x1_0.yaml \
-o Global.pretrained_model=output/DistillationModel/best_model_student \
-o Global.save_inference_dir=deploy/models/PPLCNet_x1_0_vehicle_attribute_inferAfter running above command, the inference model files would be saved in PPLCNet_x1_0_vehicle_attribute_infer, as shown below:
└── PPLCNet_x1_0_vehicle_attribute_infer
├── inference.pdiparams
├── inference.pdiparams.info
└── inference.pdmodel
Note: The best model is from knowledge distillation training. If knowledge distillation training is not used, the best model would be saved in output/PPLCNet_x1_0/best_model.pdparams.
You can also download directly.
cd deploy/models
# download the inference model and decompression
wget https://paddleclas.bj.bcebos.com/models/PULC/vehicle_attribute_infer.tar && tar -xf vehicle_attribute_infer.tar
After decompression, the directory models should be shown below.
├── vehicle_attribute_infer
│ ├── inference.pdiparams
│ ├── inference.pdiparams.info
│ └── inference.pdmodel
Return the directory deploy:
cd ../
Run the following command to classify whether there are human in the image ../images/PULC/vehicle_attribute/0002_c002_00030670_0.jpg.
# Use the following command to predict with GPU.
python3.7 python/predict_cls.py -c configs/PULC/vehicle_attribute/inference_vehicle_attribute.yaml -o Global.use_gpu=True
# Use the following command to predict with CPU.
python3.7 python/predict_cls.py -c configs/PULC/vehicle_attribute/inference_vehicle_attribute.yaml -o Global.use_gpu=FalseThe prediction results:
0002_c002_00030670_0.jpg: {'attributes': 'Color: (yellow, prob: 0.9893478155136108), Type: (hatchback, prob: 0.9734099507331848)', 'output': [1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0]}
If you want to predict images in directory, please specify the argument Global.infer_imgs as directory path by -o Global.infer_imgs. The command is as follow.
# Use the following command to predict with GPU. If want to replace with CPU, you can add argument -o Global.use_gpu=False
python3.7 python/predict_cls.py -c configs/PULC/vehicle_attribute/inference_vehicle_attribute.yaml -o Global.infer_imgs="./images/PULC/vehicle_attribute/"All prediction results will be printed, as shown below.
0002_c002_00030670_0.jpg: {'attributes': 'Color: (yellow, prob: 0.9893476963043213), Type: (hatchback, prob: 0.9734097719192505)', 'output': [1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0]}
0014_c012_00040750_0.jpg: {'attributes': 'Color: (red, prob: 0.999872088432312), Type: (sedan, prob: 0.999976634979248)', 'output': [0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0]}
Among the prediction results above, someone means that there is a human in the image, nobody means that there is no human in the image.
PaddleClas provides an example about how to deploy with C++. Please refer to Deployment with C++.
Paddle Serving is a flexible, high-performance carrier for machine learning models, and supports different protocol, such as RESTful, gRPC, bRPC and so on, which provides different deployment solutions for a variety of heterogeneous hardware and operating system environments. Please refer Paddle Serving for more information.
PaddleClas provides an example about how to deploy as service by Paddle Serving. Please refer to Paddle Serving Deployment.
Paddle-Lite is an open source deep learning framework that designed to make easy to perform inference on mobile, embeded, and IoT devices. Please refer to Paddle-Lite for more information.
PaddleClas provides an example of how to deploy on mobile by Paddle-Lite. Please refer to Paddle-Lite deployment.
Paddle2ONNX support convert Paddle Inference model to ONNX model. And you can deploy with ONNX model on different inference engine, such as TensorRT, OpenVINO, MNN/TNN, NCNN and so on. About Paddle2ONNX details, please refer to Paddle2ONNX.
PaddleClas provides an example of how to convert Paddle Inference model to ONNX model by paddle2onnx toolkit and predict by ONNX model. You can refer to paddle2onnx for deployment details.