Skip to content

Commit 59a3b68

Browse files
majianjiamajianjia
committed
add utils docs
1 parent d1afd89 commit 59a3b68

File tree

2 files changed

+232
-1
lines changed

2 files changed

+232
-1
lines changed

‎docs/api_nnom_utils.md

Lines changed: 231 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,231 @@
1+
2+
## NNoM Utils
3+
4+
NNoM Utils are Python scripts for deploying models. What makes NNoM easy to use is the models can be deployed to MCU automatically or manually with the help of NNoM utils. These functions are located in `scripts/nnom_utils.py`
5+
6+
7+
# API
8+
9+
---
10+
11+
## generate_model()
12+
13+
~~~python
14+
generate_model(model, x_test, name='weights.h')
15+
~~~
16+
17+
This method is the most frequently used function for deployment.
18+
1. It firsly scans the output range of each layer's output using `layers_output_ranges()`
19+
2. Then it quantised and write the weights & bias, fused the BatchNorm parameters using `generate_weights()`
20+
3. Finally, it generate the C functions ` nnom_model_t* nnom_model_create(void)` in `weights.h`
21+
**Arguments**
22+
23+
- **model:** the trained Keras model
24+
- **x_test:** the dataset used to check the output range of each layer.
25+
- **name:** the name of the automatically generated c file.
26+
27+
**Notes**
28+
29+
- This method might not be updated from time to time with new features in NNoM.
30+
- Currently, only support single input, single output models.
31+
32+
---
33+
34+
## layers_output_ranges()
35+
36+
~~~python
37+
layers_output_ranges(model, x_test):
38+
~~~
39+
40+
This function is to check the output range and generate the output shifting list of each layer. It will automatically distinguish whether a layer can change its output Q format or not.
41+
42+
**Arguments**
43+
44+
- **model:** the trained Keras model
45+
- **x_test:** the dataset used to check the output range of each layer.
46+
47+
**Return**
48+
49+
- The shifting list.
50+
51+
**Notes**
52+
53+
- Checking output range of each layer is essential in deploying. It is a part of the quantisation process.
54+
55+
---
56+
57+
## generate_weights()
58+
59+
~~~python
60+
generate_weights(model, name='weights.h', shift_list=None)
61+
~~~
62+
63+
Scans all the layer which includes weights, quantise the weights and put them into the c header.
64+
65+
**Arguments**
66+
67+
- **model:** the trained Keras model
68+
- **name:** the c file name to store weigths.
69+
- **shift_list:** the shift list returned by `layers_output_ranges(model, x_test)`
70+
71+
**Notes**
72+
73+
- Use function individually when willing to use none-supported operation by `generate_model()`
74+
75+
---
76+
77+
## generate_weights()
78+
79+
~~~python
80+
generate_weights(model, name='weights.h', shift_list=None)
81+
~~~
82+
83+
Scans all the layer which includes weights, quantise the weights and put them into the c header.
84+
85+
**Arguments**
86+
87+
- **model:** the trained Keras model
88+
- **name:** the c file name to store weigths.
89+
- **shift_list:** the shift list returned by `layers_output_ranges(model, x_test)`
90+
91+
92+
**Notes**
93+
94+
- Use function individually when willing to use none-supported operation by `generate_model()`
95+
96+
---
97+
98+
## evaluate_model()
99+
100+
~~~python
101+
def evaluate_model(model, x_test, y_test, running_time=False, to_file='evaluation.txt'):
102+
~~~
103+
104+
Evaluate the model after training. It do running time check, Top-k(k=1,2) accuracy, and confusion matrix.
105+
106+
**Arguments**
107+
108+
- **model:** the trained Keras model
109+
- **x_test:** the dataset for testing (one hot format)
110+
- **y_test:** the label for testing dataset
111+
- **running_time:** check running time for one prediction
112+
- **to_file:** the file to save the results.
113+
114+
---
115+
116+
## generate_test_bin()
117+
118+
~~~python
119+
generate_test_bin(x, y, name='test_data_with_label.bin')
120+
~~~
121+
122+
This is to generate a binary file for MCU side for model validation.
123+
The format of the file is shown below.
124+
Each batch size 128 started with 128 label, each label has converted from one-hot to number.
125+
The 'n' is the size of one data, such as 28x28=784 for mnist, or 32x32x3=3072 for cifar.
126+
127+
|Label(0~127)|Data0|Data1|...|Data127|Label(128)|Data128...|
128+
|---|||||||
129+
|128-byte|n-byte|n-byte|...|n-byte|128-byte|n-bytes...|
130+
131+
**Arguments**
132+
133+
- **x:** the quantised dataset for testing
134+
- **y:** the label for testing dataset(one hot format)
135+
- **name:** the label for testing dataset
136+
137+
**Output**
138+
- the binary file for testing on MCU.
139+
140+
**Notes**
141+
142+
The data must be quantised to the fixed-point range. For example,
143+
MNIST range `0~255` whcih can be converted into `0~1` using `((float)mnist/255)` for training.
144+
After training, it should be converted back to `0~127` for binary file because MCU only recognised q7 format.
145+
146+
147+
---
148+
149+
## fake_clip()
150+
151+
~~~python
152+
fake_clip(frac_bit=0, bit=8)
153+
~~~
154+
155+
(deprecated) This function is used for inseart a fake_quantitation using tensorflow's method `tf.fake_quant_with_min_max_vars()`, for similating quantised output of fixed-point model (the deployed model) during training. It should be inserted to the model after every 'conv' or 'dense' layer.
156+
157+
However, without simulating the quantisation, most of the models are still performing well. Manually insert these layers will take huge effort for training-configuring cycle result in only a slitly better accuracy.
158+
159+
**Arguments**
160+
161+
- **frac_bit:** the fraction bit in Q-format
162+
- **bit:** the quantisation bitwidth.
163+
164+
**Examples**
165+
166+
~~~python
167+
x = Conv2D(k, kernel_size=(3, 3), strides=(1,1), padding="same")(x)
168+
x = fake_clip(frac_bit=7, bit=8)(x) # quantise range to [-1~1) with 256 level.
169+
x = ReLU()(x)
170+
~~~
171+
172+
---
173+
174+
## fake_clip_min_max()
175+
176+
~~~python
177+
fake_clip_min_max(min=0, max=1, bit=8)
178+
~~~
179+
180+
(deprecated) a max-min version of `fake_clip()`, check `fake_clip()` for details.
181+
182+
**Arguments**
183+
184+
- **min:** the min value to be clipped.
185+
- **max:** the max value to be clipped.
186+
- **bit:** the quantisation bitwidth.
187+
188+
---
189+
190+
# Examples
191+
192+
This code snips shows training using above functions.
193+
194+
Please check [examples](https://github.com/majianjia/nnom/tree/master/examples) for real-life utilisation.
195+
196+
~~~python
197+
# load data
198+
(x_train, y_train), (x_test, y_test) = mnist.load_data()
199+
200+
# convert class vectors to one-hot
201+
y_train = to_categorical(y_train, num_classes)
202+
y_test = to_categorical(y_test, num_classes)
203+
204+
# quantize the range to 0~1
205+
x_test = x_test.astype('float32')/255
206+
x_train = x_train.astype('float32')/255
207+
208+
# (NNoM utils) generate the binary file for testing on MCU.
209+
generate_test_bin(x_test*127, y_test, name='test_data.bin')
210+
211+
# Train
212+
train(x_train,y_train, x_test, y_test, batch_size=128, epochs=epochs)
213+
214+
# (NNoM utils) evaluate
215+
evaluate_model(model, x_test, y_test)
216+
217+
# (NNoM utils) Automatically deploying, use 100 pices for output range
218+
generate_model(model, x_test[:100], name=weights)
219+
220+
~~~
221+
222+
223+
224+
225+
226+
227+
228+
229+
230+
231+

‎mkdocs.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,8 +2,8 @@ site_name: NNoM
22

33
nav:
44
- Overview: 'index.md'
5-
65
- APIs:
6+
- Utils: 'api_nnom_utils.md'
77
- Model: 'api_model.md'
88
- Construction: 'api_construction.md'
99
- Properties: 'api_properties.md'

0 commit comments

Comments
 (0)