-
Notifications
You must be signed in to change notification settings - Fork 79
Expand file tree
/
Copy pathGradientPTQConfig.html
More file actions
204 lines (190 loc) · 16 KB
/
GradientPTQConfig.html
File metadata and controls
204 lines (190 loc) · 16 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
<!DOCTYPE html>
<html lang="en" data-content_root="../../../">
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" /><meta name="viewport" content="width=device-width, initial-scale=1" />
<title>GradientPTQConfig Class — MCT Documentation: ver 2.6.0</title>
<link rel="stylesheet" type="text/css" href="../../../static/pygments.css?v=fa44fd50" />
<link rel="stylesheet" type="text/css" href="../../../static/bizstyle.css?v=5283bb3d" />
<link rel="stylesheet" type="text/css" href="../../../static/css/custom.css?v=01243f34" />
<script src="../../../static/documentation_options.js?v=ae39cb24"></script>
<script src="../../../static/doctools.js?v=9bcbadda"></script>
<script src="../../../static/sphinx_highlight.js?v=dc90522c"></script>
<script src="../../../static/bizstyle.js"></script>
<link rel="index" title="Index" href="../../../genindex.html" />
<link rel="search" title="Search" href="../../../search.html" />
<meta name="viewport" content="width=device-width,initial-scale=1.0" />
<!--[if lt IE 9]>
<script src="static/css3-mediaqueries.js"></script>
<![endif]-->
</head><body>
<div class="related" role="navigation" aria-label="Related">
<h3>Navigation</h3>
<ul>
<li class="right" style="margin-right: 10px">
<a href="../../../genindex.html" title="General Index"
accesskey="I">index</a></li>
<li class="nav-item nav-item-0"><a href="../../../index.html">MCT Documentation: ver 2.6.0</a> »</li>
<li class="nav-item nav-item-this"><a href="">GradientPTQConfig Class</a></li>
</ul>
</div>
<div class="document">
<div class="documentwrapper">
<div class="bodywrapper">
<div class="body" role="main">
<section id="gradientptqconfig-class">
<span id="ug-gradientptqconfig"></span><h1>GradientPTQConfig Class<a class="headerlink" href="#gradientptqconfig-class" title="Link to this heading">¶</a></h1>
<p><strong>The following API can be used to create a GradientPTQConfig instance which can be used for post training quantization using knowledge distillation from a teacher (float model) to a student (the quantized model)</strong></p>
<dl class="py class">
<dt class="sig sig-object py" id="model_compression_toolkit.gptq.GradientPTQConfig">
<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-prename descclassname"><span class="pre">model_compression_toolkit.gptq.</span></span><span class="sig-name descname"><span class="pre">GradientPTQConfig</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">n_epochs</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">loss</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">optimizer</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">optimizer_rest</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">train_bias</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">hessian_weights_config</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">gradual_activation_quantization_config</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">regularization_factor</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">rounding_type=RoundingType.SoftQuantizer</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">optimizer_quantization_parameter=None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">optimizer_bias=None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">log_function=None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">gptq_quantizer_params_override=<factory></span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#model_compression_toolkit.gptq.GradientPTQConfig" title="Link to this definition">¶</a></dt>
<dd><p>Configuration to use for quantization with GradientPTQ.</p>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>n_epochs</strong> – Number of representative dataset epochs to train.</p></li>
<li><p><strong>loss</strong> – The loss to use. See ‘multiple_tensors_mse_loss’ for the expected interface.</p></li>
<li><p><strong>optimizer</strong> – Optimizer to use.</p></li>
<li><p><strong>optimizer_rest</strong> – Default optimizer to use for bias and quantizer parameters.</p></li>
<li><p><strong>train_bias</strong> – Whether to update the bias during the training or not.</p></li>
<li><p><strong>hessian_weights_config</strong> – A configuration that include all necessary arguments to run a computation of
Hessian scores for the GPTQ loss.</p></li>
<li><p><strong>gradual_activation_quantization_config</strong> – A configuration for Gradual Activation Quantization.</p></li>
<li><p><strong>regularization_factor</strong> – A floating point number that defines the regularization factor.</p></li>
<li><p><strong>rounding_type</strong> – An enum that defines the rounding type.</p></li>
<li><p><strong>optimizer_quantization_parameter</strong> – Optimizer to override the rest optimizer for quantizer parameters.</p></li>
<li><p><strong>optimizer_bias</strong> – Optimizer to override the rest optimizer for bias.</p></li>
<li><p><strong>log_function</strong> – Function to log information about the GPTQ process.</p></li>
<li><p><strong>gptq_quantizer_params_override</strong> – A dictionary of parameters to override in GPTQ quantizer instantiation.</p></li>
</ul>
</dd>
</dl>
</dd></dl>
</section>
<section id="gptqhessianscoresconfig-class">
<h1>GPTQHessianScoresConfig Class<a class="headerlink" href="#gptqhessianscoresconfig-class" title="Link to this heading">¶</a></h1>
<p><strong>The following API can be used to create a GPTQHessianScoresConfig instance which can be used to define necessary parameters for computing Hessian scores for the GPTQ loss function.</strong></p>
<dl class="py class">
<dt class="sig sig-object py" id="model_compression_toolkit.gptq.GPTQHessianScoresConfig">
<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-prename descclassname"><span class="pre">model_compression_toolkit.gptq.</span></span><span class="sig-name descname"><span class="pre">GPTQHessianScoresConfig</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">per_sample</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">hessians_num_samples</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">norm_scores</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">log_norm</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">scale_log_norm</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">False</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">hessian_batch_size</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">32</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#model_compression_toolkit.gptq.GPTQHessianScoresConfig" title="Link to this definition">¶</a></dt>
<dd><p>Configuration to use for computing the Hessian-based scores for GPTQ loss metric.</p>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>per_sample</strong> (<em>bool</em>) – Whether to use per sample attention score.</p></li>
<li><p><strong>hessians_num_samples</strong> (<em>int</em><em>|</em><em>None</em>) – Number of samples to use for computing the Hessian-based scores.
If None, compute Hessian for all images.</p></li>
<li><p><strong>norm_scores</strong> (<em>bool</em>) – Whether to normalize the returned scores of the weighted loss function (to get values between 0 and 1).</p></li>
<li><p><strong>log_norm</strong> (<em>bool</em>) – Whether to use log normalization for the GPTQ Hessian-based scores.</p></li>
<li><p><strong>scale_log_norm</strong> (<em>bool</em>) – Whether to scale the final vector of the Hessian-based scores.</p></li>
<li><p><strong>hessian_batch_size</strong> (<em>int</em>) – The Hessian computation batch size. used only if using GPTQ with Hessian-based objective.</p></li>
</ul>
</dd>
</dl>
</dd></dl>
</section>
<section id="roundingtype">
<h1>RoundingType<a class="headerlink" href="#roundingtype" title="Link to this heading">¶</a></h1>
<dl class="py class">
<dt class="sig sig-object py" id="model_compression_toolkit.gptq.RoundingType">
<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-prename descclassname"><span class="pre">model_compression_toolkit.gptq.</span></span><span class="sig-name descname"><span class="pre">RoundingType</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">value</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#model_compression_toolkit.gptq.RoundingType" title="Link to this definition">¶</a></dt>
<dd><p>An enum for choosing the GPTQ rounding methods:</p>
<p>STE - STRAIGHT-THROUGH ESTIMATOR</p>
<p>SoftQuantizer - SoftQuantizer</p>
</dd></dl>
</section>
<section id="gradualactivationquantizationconfig">
<h1>GradualActivationQuantizationConfig<a class="headerlink" href="#gradualactivationquantizationconfig" title="Link to this heading">¶</a></h1>
<p><strong>The following API can be used to configure the gradual activation quantization when using GPTQ.</strong></p>
<dl class="py class">
<dt class="sig sig-object py" id="model_compression_toolkit.gptq.GradualActivationQuantizationConfig">
<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-prename descclassname"><span class="pre">model_compression_toolkit.gptq.</span></span><span class="sig-name descname"><span class="pre">GradualActivationQuantizationConfig</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">q_fraction_scheduler_policy=<factory></span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#model_compression_toolkit.gptq.GradualActivationQuantizationConfig" title="Link to this definition">¶</a></dt>
<dd><p>Configuration for Gradual Activation Quantization.</p>
<p>By default, the quantized fraction increases linearly from 0 to 1 throughout the training.</p>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><p><strong>q_fraction_scheduler_policy</strong> – config for the scheduling of the quantized fraction.
Only linear annealing is currently supported.</p>
</dd>
</dl>
</dd></dl>
</section>
<section id="qfractionlinearannealingconfig">
<h1>QFractionLinearAnnealingConfig<a class="headerlink" href="#qfractionlinearannealingconfig" title="Link to this heading">¶</a></h1>
<dl class="py class">
<dt class="sig sig-object py" id="model_compression_toolkit.gptq.QFractionLinearAnnealingConfig">
<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-prename descclassname"><span class="pre">model_compression_toolkit.gptq.</span></span><span class="sig-name descname"><span class="pre">QFractionLinearAnnealingConfig</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">initial_q_fraction</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">target_q_fraction</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">start_step</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">end_step</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#model_compression_toolkit.gptq.QFractionLinearAnnealingConfig" title="Link to this definition">¶</a></dt>
<dd><p>Config for the quantized fraction linear scheduler of Gradual Activation Quantization.</p>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>initial_q_fraction</strong> – initial quantized fraction</p></li>
<li><p><strong>target_q_fraction</strong> – target quantized fraction</p></li>
<li><p><strong>start_step</strong> – gradient step to begin annealing</p></li>
<li><p><strong>end_step</strong> – gradient step to complete annealing. None means last step.</p></li>
</ul>
</dd>
</dl>
</dd></dl>
</section>
<div class="clearer"></div>
</div>
</div>
</div>
<div class="sphinxsidebar" role="navigation" aria-label="Main">
<div class="sphinxsidebarwrapper">
<div>
<h3><a href="../../../index.html">Table of Contents</a></h3>
<ul>
<li><a class="reference internal" href="#">GradientPTQConfig Class</a><ul>
<li><a class="reference internal" href="#model_compression_toolkit.gptq.GradientPTQConfig"><code class="docutils literal notranslate"><span class="pre">GradientPTQConfig</span></code></a></li>
</ul>
</li>
<li><a class="reference internal" href="#gptqhessianscoresconfig-class">GPTQHessianScoresConfig Class</a><ul>
<li><a class="reference internal" href="#model_compression_toolkit.gptq.GPTQHessianScoresConfig"><code class="docutils literal notranslate"><span class="pre">GPTQHessianScoresConfig</span></code></a></li>
</ul>
</li>
<li><a class="reference internal" href="#roundingtype">RoundingType</a><ul>
<li><a class="reference internal" href="#model_compression_toolkit.gptq.RoundingType"><code class="docutils literal notranslate"><span class="pre">RoundingType</span></code></a></li>
</ul>
</li>
<li><a class="reference internal" href="#gradualactivationquantizationconfig">GradualActivationQuantizationConfig</a><ul>
<li><a class="reference internal" href="#model_compression_toolkit.gptq.GradualActivationQuantizationConfig"><code class="docutils literal notranslate"><span class="pre">GradualActivationQuantizationConfig</span></code></a></li>
</ul>
</li>
<li><a class="reference internal" href="#qfractionlinearannealingconfig">QFractionLinearAnnealingConfig</a><ul>
<li><a class="reference internal" href="#model_compression_toolkit.gptq.QFractionLinearAnnealingConfig"><code class="docutils literal notranslate"><span class="pre">QFractionLinearAnnealingConfig</span></code></a></li>
</ul>
</li>
</ul>
</div>
<search id="searchbox" style="display: none" role="search">
<h3 id="searchlabel">Quick search</h3>
<div class="searchformwrapper">
<form class="search" action="../../../search.html" method="get">
<input type="text" name="q" aria-labelledby="searchlabel" autocomplete="off" autocorrect="off" autocapitalize="off" spellcheck="false"/>
<input type="submit" value="Go" />
</form>
</div>
</search>
<script>document.getElementById('searchbox').style.display = "block"</script>
</div>
</div>
<div class="clearer"></div>
</div>
<div class="related" role="navigation" aria-label="Related">
<h3>Navigation</h3>
<ul>
<li class="right" style="margin-right: 10px">
<a href="../../../genindex.html" title="General Index"
>index</a></li>
<li class="nav-item nav-item-0"><a href="../../../index.html">MCT Documentation: ver 2.6.0</a> »</li>
<li class="nav-item nav-item-this"><a href="">GradientPTQConfig Class</a></li>
</ul>
</div>
<div class="footer" role="contentinfo">
© Copyright 2022, Sony Semiconductor Solutions.
Created using <a href="https://www.sphinx-doc.org/">Sphinx</a> 8.1.3.
</div>
</body>
</html>