-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy pathht_note.html
1967 lines (1727 loc) · 188 KB
/
ht_note.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
<!DOCTYPE html><html><head>
<title>ht_note</title>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<link rel="stylesheet" href="file:///D:\VSCode\Extensions\shd101wyy.markdown-preview-enhanced-0.6.1\node_modules\@shd101wyy\mume\dependencies\katex\katex.min.css">
<style>
/**
* prism.js Github theme based on GitHub's theme.
* @author Sam Clarke
*/
code[class*="language-"],
pre[class*="language-"] {
color: #333;
background: none;
font-family: Consolas, "Liberation Mono", Menlo, Courier, monospace;
text-align: left;
white-space: pre;
word-spacing: normal;
word-break: normal;
word-wrap: normal;
line-height: 1.4;
-moz-tab-size: 8;
-o-tab-size: 8;
tab-size: 8;
-webkit-hyphens: none;
-moz-hyphens: none;
-ms-hyphens: none;
hyphens: none;
}
/* Code blocks */
pre[class*="language-"] {
padding: .8em;
overflow: auto;
/* border: 1px solid #ddd; */
border-radius: 3px;
/* background: #fff; */
background: #f5f5f5;
}
/* Inline code */
:not(pre) > code[class*="language-"] {
padding: .1em;
border-radius: .3em;
white-space: normal;
background: #f5f5f5;
}
.token.comment,
.token.blockquote {
color: #969896;
}
.token.cdata {
color: #183691;
}
.token.doctype,
.token.punctuation,
.token.variable,
.token.macro.property {
color: #333;
}
.token.operator,
.token.important,
.token.keyword,
.token.rule,
.token.builtin {
color: #a71d5d;
}
.token.string,
.token.url,
.token.regex,
.token.attr-value {
color: #183691;
}
.token.property,
.token.number,
.token.boolean,
.token.entity,
.token.atrule,
.token.constant,
.token.symbol,
.token.command,
.token.code {
color: #0086b3;
}
.token.tag,
.token.selector,
.token.prolog {
color: #63a35c;
}
.token.function,
.token.namespace,
.token.pseudo-element,
.token.class,
.token.class-name,
.token.pseudo-class,
.token.id,
.token.url-reference .token.variable,
.token.attr-name {
color: #795da3;
}
.token.entity {
cursor: help;
}
.token.title,
.token.title .token.punctuation {
font-weight: bold;
color: #1d3e81;
}
.token.list {
color: #ed6a43;
}
.token.inserted {
background-color: #eaffea;
color: #55a532;
}
.token.deleted {
background-color: #ffecec;
color: #bd2c00;
}
.token.bold {
font-weight: bold;
}
.token.italic {
font-style: italic;
}
/* JSON */
.language-json .token.property {
color: #183691;
}
.language-markup .token.tag .token.punctuation {
color: #333;
}
/* CSS */
code.language-css,
.language-css .token.function {
color: #0086b3;
}
/* YAML */
.language-yaml .token.atrule {
color: #63a35c;
}
code.language-yaml {
color: #183691;
}
/* Ruby */
.language-ruby .token.function {
color: #333;
}
/* Markdown */
.language-markdown .token.url {
color: #795da3;
}
/* Makefile */
.language-makefile .token.symbol {
color: #795da3;
}
.language-makefile .token.variable {
color: #183691;
}
.language-makefile .token.builtin {
color: #0086b3;
}
/* Bash */
.language-bash .token.keyword {
color: #0086b3;
}
/* highlight */
pre[data-line] {
position: relative;
padding: 1em 0 1em 3em;
}
pre[data-line] .line-highlight-wrapper {
position: absolute;
top: 0;
left: 0;
background-color: transparent;
display: block;
width: 100%;
}
pre[data-line] .line-highlight {
position: absolute;
left: 0;
right: 0;
padding: inherit 0;
margin-top: 1em;
background: hsla(24, 20%, 50%,.08);
background: linear-gradient(to right, hsla(24, 20%, 50%,.1) 70%, hsla(24, 20%, 50%,0));
pointer-events: none;
line-height: inherit;
white-space: pre;
}
pre[data-line] .line-highlight:before,
pre[data-line] .line-highlight[data-end]:after {
content: attr(data-start);
position: absolute;
top: .4em;
left: .6em;
min-width: 1em;
padding: 0 .5em;
background-color: hsla(24, 20%, 50%,.4);
color: hsl(24, 20%, 95%);
font: bold 65%/1.5 sans-serif;
text-align: center;
vertical-align: .3em;
border-radius: 999px;
text-shadow: none;
box-shadow: 0 1px white;
}
pre[data-line] .line-highlight[data-end]:after {
content: attr(data-end);
top: auto;
bottom: .4em;
}html body{font-family:"Helvetica Neue",Helvetica,"Segoe UI",Arial,freesans,sans-serif;font-size:16px;line-height:1.6;color:#333;background-color:#fff;overflow:initial;box-sizing:border-box;word-wrap:break-word}html body>:first-child{margin-top:0}html body h1,html body h2,html body h3,html body h4,html body h5,html body h6{line-height:1.2;margin-top:1em;margin-bottom:16px;color:#000}html body h1{font-size:2.25em;font-weight:300;padding-bottom:.3em}html body h2{font-size:1.75em;font-weight:400;padding-bottom:.3em}html body h3{font-size:1.5em;font-weight:500}html body h4{font-size:1.25em;font-weight:600}html body h5{font-size:1.1em;font-weight:600}html body h6{font-size:1em;font-weight:600}html body h1,html body h2,html body h3,html body h4,html body h5{font-weight:600}html body h5{font-size:1em}html body h6{color:#5c5c5c}html body strong{color:#000}html body del{color:#5c5c5c}html body a:not([href]){color:inherit;text-decoration:none}html body a{color:#08c;text-decoration:none}html body a:hover{color:#00a3f5;text-decoration:none}html body img{max-width:100%}html body>p{margin-top:0;margin-bottom:16px;word-wrap:break-word}html body>ul,html body>ol{margin-bottom:16px}html body ul,html body ol{padding-left:2em}html body ul.no-list,html body ol.no-list{padding:0;list-style-type:none}html body ul ul,html body ul ol,html body ol ol,html body ol ul{margin-top:0;margin-bottom:0}html body li{margin-bottom:0}html body li.task-list-item{list-style:none}html body li>p{margin-top:0;margin-bottom:0}html body .task-list-item-checkbox{margin:0 .2em .25em -1.8em;vertical-align:middle}html body .task-list-item-checkbox:hover{cursor:pointer}html body blockquote{margin:16px 0;font-size:inherit;padding:0 15px;color:#5c5c5c;background-color:#f0f0f0;border-left:4px solid #d6d6d6}html body blockquote>:first-child{margin-top:0}html body blockquote>:last-child{margin-bottom:0}html body hr{height:4px;margin:32px 0;background-color:#d6d6d6;border:0 none}html body table{margin:10px 0 15px 0;border-collapse:collapse;border-spacing:0;display:block;width:100%;overflow:auto;word-break:normal;word-break:keep-all}html body table th{font-weight:bold;color:#000}html body table td,html body table th{border:1px solid #d6d6d6;padding:6px 13px}html body dl{padding:0}html body dl dt{padding:0;margin-top:16px;font-size:1em;font-style:italic;font-weight:bold}html body dl dd{padding:0 16px;margin-bottom:16px}html body code{font-family:Menlo,Monaco,Consolas,'Courier New',monospace;font-size:.85em !important;color:#000;background-color:#f0f0f0;border-radius:3px;padding:.2em 0}html body code::before,html body code::after{letter-spacing:-0.2em;content:"\00a0"}html body pre>code{padding:0;margin:0;font-size:.85em !important;word-break:normal;white-space:pre;background:transparent;border:0}html body .highlight{margin-bottom:16px}html body .highlight pre,html body pre{padding:1em;overflow:auto;font-size:.85em !important;line-height:1.45;border:#d6d6d6;border-radius:3px}html body .highlight pre{margin-bottom:0;word-break:normal}html body pre code,html body pre tt{display:inline;max-width:initial;padding:0;margin:0;overflow:initial;line-height:inherit;word-wrap:normal;background-color:transparent;border:0}html body pre code:before,html body pre tt:before,html body pre code:after,html body pre tt:after{content:normal}html body p,html body blockquote,html body ul,html body ol,html body dl,html body pre{margin-top:0;margin-bottom:16px}html body kbd{color:#000;border:1px solid #d6d6d6;border-bottom:2px solid #c7c7c7;padding:2px 4px;background-color:#f0f0f0;border-radius:3px}@media print{html body{background-color:#fff}html body h1,html body h2,html body h3,html body h4,html body h5,html body h6{color:#000;page-break-after:avoid}html body blockquote{color:#5c5c5c}html body pre{page-break-inside:avoid}html body table{display:table}html body img{display:block;max-width:100%;max-height:100%}html body pre,html body code{word-wrap:break-word;white-space:pre}}.markdown-preview{width:100%;height:100%;box-sizing:border-box}.markdown-preview .pagebreak,.markdown-preview .newpage{page-break-before:always}.markdown-preview pre.line-numbers{position:relative;padding-left:3.8em;counter-reset:linenumber}.markdown-preview pre.line-numbers>code{position:relative}.markdown-preview pre.line-numbers .line-numbers-rows{position:absolute;pointer-events:none;top:1em;font-size:100%;left:0;width:3em;letter-spacing:-1px;border-right:1px solid #999;-webkit-user-select:none;-moz-user-select:none;-ms-user-select:none;user-select:none}.markdown-preview pre.line-numbers .line-numbers-rows>span{pointer-events:none;display:block;counter-increment:linenumber}.markdown-preview pre.line-numbers .line-numbers-rows>span:before{content:counter(linenumber);color:#999;display:block;padding-right:.8em;text-align:right}.markdown-preview .mathjax-exps .MathJax_Display{text-align:center !important}.markdown-preview:not([for="preview"]) .code-chunk .btn-group{display:none}.markdown-preview:not([for="preview"]) .code-chunk .status{display:none}.markdown-preview:not([for="preview"]) .code-chunk .output-div{margin-bottom:16px}.scrollbar-style::-webkit-scrollbar{width:8px}.scrollbar-style::-webkit-scrollbar-track{border-radius:10px;background-color:transparent}.scrollbar-style::-webkit-scrollbar-thumb{border-radius:5px;background-color:rgba(150,150,150,0.66);border:4px solid rgba(150,150,150,0.66);background-clip:content-box}html body[for="html-export"]:not([data-presentation-mode]){position:relative;width:100%;height:100%;top:0;left:0;margin:0;padding:0;overflow:auto}html body[for="html-export"]:not([data-presentation-mode]) .markdown-preview{position:relative;top:0}@media screen and (min-width:914px){html body[for="html-export"]:not([data-presentation-mode]) .markdown-preview{padding:2em calc(50% - 457px + 2em)}}@media screen and (max-width:914px){html body[for="html-export"]:not([data-presentation-mode]) .markdown-preview{padding:2em}}@media screen and (max-width:450px){html body[for="html-export"]:not([data-presentation-mode]) .markdown-preview{font-size:14px !important;padding:1em}}@media print{html body[for="html-export"]:not([data-presentation-mode]) #sidebar-toc-btn{display:none}}html body[for="html-export"]:not([data-presentation-mode]) #sidebar-toc-btn{position:fixed;bottom:8px;left:8px;font-size:28px;cursor:pointer;color:inherit;z-index:99;width:32px;text-align:center;opacity:.4}html body[for="html-export"]:not([data-presentation-mode])[html-show-sidebar-toc] #sidebar-toc-btn{opacity:1}html body[for="html-export"]:not([data-presentation-mode])[html-show-sidebar-toc] .md-sidebar-toc{position:fixed;top:0;left:0;width:300px;height:100%;padding:32px 0 48px 0;font-size:14px;box-shadow:0 0 4px rgba(150,150,150,0.33);box-sizing:border-box;overflow:auto;background-color:inherit}html body[for="html-export"]:not([data-presentation-mode])[html-show-sidebar-toc] .md-sidebar-toc::-webkit-scrollbar{width:8px}html body[for="html-export"]:not([data-presentation-mode])[html-show-sidebar-toc] .md-sidebar-toc::-webkit-scrollbar-track{border-radius:10px;background-color:transparent}html body[for="html-export"]:not([data-presentation-mode])[html-show-sidebar-toc] .md-sidebar-toc::-webkit-scrollbar-thumb{border-radius:5px;background-color:rgba(150,150,150,0.66);border:4px solid rgba(150,150,150,0.66);background-clip:content-box}html body[for="html-export"]:not([data-presentation-mode])[html-show-sidebar-toc] .md-sidebar-toc a{text-decoration:none}html body[for="html-export"]:not([data-presentation-mode])[html-show-sidebar-toc] .md-sidebar-toc ul{padding:0 1.6em;margin-top:.8em}html body[for="html-export"]:not([data-presentation-mode])[html-show-sidebar-toc] .md-sidebar-toc li{margin-bottom:.8em}html body[for="html-export"]:not([data-presentation-mode])[html-show-sidebar-toc] .md-sidebar-toc ul{list-style-type:none}html body[for="html-export"]:not([data-presentation-mode])[html-show-sidebar-toc] .markdown-preview{left:300px;width:calc(100% - 300px);padding:2em calc(50% - 457px - 150px);margin:0;box-sizing:border-box}@media screen and (max-width:1274px){html body[for="html-export"]:not([data-presentation-mode])[html-show-sidebar-toc] .markdown-preview{padding:2em}}@media screen and (max-width:450px){html body[for="html-export"]:not([data-presentation-mode])[html-show-sidebar-toc] .markdown-preview{width:100%}}html body[for="html-export"]:not([data-presentation-mode]):not([html-show-sidebar-toc]) .markdown-preview{left:50%;transform:translateX(-50%)}html body[for="html-export"]:not([data-presentation-mode]):not([html-show-sidebar-toc]) .md-sidebar-toc{display:none}
/* Please visit the URL below for more information: */
/* https://shd101wyy.github.io/markdown-preview-enhanced/#/customize-css */
</style>
</head>
<body for="html-export">
<div class="mume markdown-preview ">
<p><em>搬运工：保罗史蒂芬乔治洪 ~ <a href="http://paul.ht">paul.ht</a></em></p>
<p><strong>布局结构</strong>：</p>
<ul>
<li>Python
<ul>
<li>Pytorch</li>
<li>Tensorflow</li>
<li>Python</li>
<li>opencv2</li>
</ul>
</li>
<li>MATLAB</li>
<li>Linux</li>
<li>Git</li>
<li>Mac</li>
<li>Markdown</li>
<li>LaTex</li>
<li>Others</li>
</ul>
<hr>
<div class="code-chunk" data-id="code-chunk-id-0" data-cmd="toc"><div class="input-div"><div class="btn-group"><div class="run-btn btn"><span>▶︎</span></div><div class="run-all-btn btn">all</div></div><div class="status">running...</div></div><div class="output-div"></div></div><ul>
<li><a href="#python">Python</a>
<ul>
<li><a href="#pytorch">Pytorch</a>
<ul>
<li><a href="#tensor">Tensor</a></li>
<li><a href="#item">item()</a></li>
<li><a href="#torchvisiontransforms">torchvision.transforms</a></li>
<li><a href="#torchvisionsave_image">torchvision.save_image</a></li>
<li><a href="#relu">Relu</a></li>
<li><a href="#nnconvtranspose2d">nn.ConvTranspose2d</a></li>
<li><a href="#torchvision">torchvision</a></li>
<li><a href="#cv2%E7%9A%84%E5%9D%91">cv2的坑</a></li>
<li><a href="#%E8%B0%83%E8%8A%82%E5%AD%A6%E4%B9%A0%E7%8E%87">调节学习率</a></li>
<li><a href="#torchnndataparallel">torch.nn.DataParallel()</a></li>
<li><a href="#torchnndistributeddataparallel">torch.nn.DistributedDataParallel</a></li>
<li><a href="#num_workers">num_workers</a></li>
<li><a href="#device">device</a></li>
<li><a href="#multinomial">multinomial</a></li>
<li><a href="#load_lua---torchfileload">load_lua -> torchfile.load</a></li>
<li><a href="#hook">hook</a></li>
<li><a href="#torchsummary">torchsummary</a></li>
<li><a href="#cprofile">cProfile</a></li>
</ul>
</li>
<li><a href="#tensorflow">Tensorflow</a>
<ul>
<li><a href="#tfsession">tf.Session()</a></li>
<li><a href="#tensorflow-%E4%B9%8B-checkpoint">tensorflow 之 checkpoint</a></li>
<li><a href="#%E9%80%89%E6%8B%A9gpu">选择GPU</a></li>
<li><a href="#tfnn">tf.nn</a></li>
<li><a href="#%E5%8D%B7%E7%A7%AF%E6%8E%A2%E8%AE%A8">卷积探讨</a></li>
<li><a href="#%E6%84%9F%E5%8F%97%E9%87%8E">感受野</a></li>
<li><a href="#resnet">ResNet</a></li>
<li><a href="#ncwh">(N,C,W,H)</a></li>
<li><a href="#%E4%BC%98%E5%8C%96%E5%99%A8">优化器</a></li>
<li><a href="#training-accuracy">training accuracy</a></li>
</ul>
</li>
<li><a href="#python-1">Python</a>
<ul>
<li><a href="#print">print</a></li>
<li><a href="#npclip">np.clip()</a></li>
<li><a href="#nprandomchoice">np.random.choice()</a></li>
<li><a href="#%E6%8E%92%E5%BA%8F">排序</a></li>
<li><a href="#zip">zip</a></li>
<li><a href="#eval">eval()</a></li>
<li><a href="#f-string">f-string</a></li>
<li><a href="#glob">glob</a></li>
<li><a href="#ospathdirname__file__">os.path.dirname(__file__)</a></li>
<li><a href="#argparse">argparse</a>
<ul>
<li><a href="#bool%E5%9E%8Bargparse-%E5%9D%91">bool型argparse 坑</a></li>
</ul>
</li>
<li><a href="#config">config</a></li>
<li><a href="#class">class</a></li>
<li><a href="#__call__">__call__()</a></li>
<li><a href="#__dir__">__dir__()</a></li>
<li><a href="#python%E5%87%BD%E6%95%B0%E4%BC%A0%E5%AF%B9%E8%B1%A1call-by-object">Python函数——传对象(call by object)</a></li>
<li><a href="#globals">globals()</a></li>
<li><a href="#zfill">zfill</a></li>
<li><a href="#ravel--flatten">ravel() & flatten()</a></li>
<li><a href="#%E4%B8%8D%E8%BF%9E%E7%BB%AD%E5%88%87%E7%89%87-take">不连续切片 take()</a></li>
<li><a href="#nprollaxis">np.rollaxis（）</a></li>
<li><a href="#matplotlib">matplotlib</a></li>
<li><a href="#pltplot">plt.plot()</a></li>
</ul>
</li>
<li><a href="#-opencv2">## opencv2</a>
<ul>
<li><a href="#opencv-%E5%AE%89%E8%A3%85">opencv 安装</a></li>
<li><a href="#resize">resize</a></li>
<li><a href="#%E9%A2%9C%E8%89%B2%E9%80%9A%E9%81%93">颜色通道</a></li>
<li><a href="#imshow-%E4%B8%8E-imwrite">imshow() 与 imwrite()</a></li>
<li><a href="#pilimage">PIL.Image</a></li>
</ul>
</li>
</ul>
</li>
<li><a href="#matlab">MATLAB</a><br>
- <a href="#matlab-bsxfun">MATLAB bsxfun</a>
<ul>
<li><a href="#python-%E4%B8%8E-matlab%E7%9A%84%E4%B8%80%E4%BA%9B%E5%87%BD%E6%95%B0%E5%8C%BA%E5%88%AB%E7%BB%86%E8%8A%82">Python 与 MATLAB的一些函数区别（细节）</a>
<ul>
<li><a href="#%E5%A4%8D%E6%95%B0%E5%9F%9F">复数域</a></li>
</ul>
</li>
</ul>
</li>
<li><a href="#linux">Linux</a><br>
- <a href="#bash">bash</a><br>
- <a href="#adduser-useradd">adduser useradd</a><br>
- <a href="#sudo">sudo</a><br>
- <a href="#chown">chown</a><br>
- <a href="#ls">ls</a><br>
- <a href="#%E8%BD%AF%E9%93%BE%E6%8E%A5">软链接</a><br>
- <a href="#ssh">ssh</a><br>
- <a href="#%E6%9F%A5%E7%9C%8Bcpu-gpu%E4%BD%BF%E7%94%A8%E6%83%85%E5%86%B5">查看CPU GPU使用情况</a><br>
- <a href="#%E8%BE%93%E5%87%BA%E6%9C%BA%E5%88%B6">输出机制</a><br>
- <a href="#export--echo">export & echo</a><br>
- <a href="#%E6%96%87%E4%BB%B6%E5%8E%8B%E7%BC%A9%E4%B8%8E%E8%A7%A3%E5%8E%8B">文件压缩与解压</a><br>
- <a href="#scp">scp</a><br>
- <a href="#ctrl%E7%B1%BB%E5%BF%AB%E6%8D%B7%E9%94%AE">Ctrl类快捷键</a><br>
- <a href="#%E6%9F%A5%E7%9C%8B%E4%BD%8D%E7%BD%AE">查看位置</a><br>
- <a href="#linux%E6%9F%A5%E7%9C%8B%E6%96%87%E4%BB%B6%E5%A4%A7%E5%B0%8F%E6%95%B0%E9%87%8F">Linux查看文件大小数量</a><br>
- <a href="#linux-%E6%9F%A5%E7%9C%8B%E7%A1%AC%E7%9B%98%E5%88%86%E5%8C%BA%E5%86%85%E5%AD%98">Linux 查看硬盘分区内存</a><br>
- <a href="#mkdir">mkdir</a><br>
- <a href="#%E6%9F%A5%E7%9C%8B%E6%9D%80%E6%AD%BB-%E8%BF%9B%E7%A8%8B">查看/杀死 进程</a><br>
- <a href="#ps-ax--grep-python">ps ax | grep python</a><br>
- <a href="#dos2unix">dos2unix</a><br>
- <a href="#%E5%AE%89%E8%A3%85matlab">安装MATLAB</a><br>
- <a href="#windows-%E8%BF%9C%E7%A8%8B%E8%BF%9E%E6%8E%A5-linux">windows 远程连接 linux</a><br>
- <a href="#rename">rename</a><br>
- <a href="#vim">vim</a><br>
- <a href="#vimrc">~/.vimrc</a><br>
- <a href="#%E6%88%91%E7%9A%84vimrc">我的~/.vimrc</a><br>
- <a href="#vim%E8%87%AA%E5%8A%A8%E8%A1%A5%E5%85%A8">vim自动补全</a><br>
- <a href="#vim-bundle">vim Bundle</a><br>
- <a href="#pip">pip</a><br>
- <a href="#conda">conda</a><br>
- <a href="#linux%E6%9B%B4%E6%94%B9%E9%BB%98%E8%AE%A4python%E7%89%88%E6%9C%AC">Linux更改默认Python版本</a><br>
- <a href="#%E6%9F%A5%E7%9C%8B%E7%BD%91%E5%85%B3">查看网关</a><br>
- <a href="#slurm%E9%9B%86%E7%BE%A4%E7%AE%A1%E7%90%86">slurm集群管理</a><br>
- <a href="#tmux">tmux</a><br>
- <a href="#path">$PATH</a><br>
- <a href="#cuda%E9%85%8D%E7%BD%AE">CUDA配置</a></li>
<li><a href="#git">Git</a><br>
- <a href="#push">push</a><br>
- <a href="#clone">clone</a><br>
- <a href="#pull">pull</a><br>
- <a href="#merge">merge</a><br>
- <a href="#diff">diff</a><br>
- <a href="#gitignore">.gitignore</a><br>
- <a href="#commit">commit</a><br>
- <a href="#reset">reset</a></li>
<li><a href="#mac">Mac</a><br>
- <a href="#%E5%B8%B8%E7%94%A8%E5%BF%AB%E6%8D%B7%E9%94%AE">常用快捷键</a><br>
- <a href="#%E6%96%B0%E5%BB%BA%E6%96%87%E4%BB%B6">新建文件</a><br>
- <a href="#%E9%9A%90%E8%97%8F%E6%96%87%E4%BB%B6">隐藏文件</a></li>
<li><a href="#markdown">Markdown</a><br>
- <a href="#markdown-%E8%B6%85%E9%93%BE%E6%8E%A5">Markdown 超链接</a><br>
- <a href="#markdown-%E7%A9%BA%E6%A0%BC">Markdown 空格</a><br>
- <a href="#markdown-%E4%BB%A3%E7%A0%81">Markdown 代码</a><br>
- <a href="#markdown-%E5%85%AC%E5%BC%8F">Markdown 公式</a><br>
- <a href="#markdown-%E5%9B%BE%E7%89%87">Markdown 图片</a><br>
- <a href="#markdown-%E7%9B%AE%E5%BD%95">Markdown 目录</a></li>
<li><a href="#latex">LaTex</a><br>
- <a href="#vscode-%E7%BC%96%E8%AF%91%E5%99%A8">VSCode 编译器</a><br>
- <a href="#%E4%B8%80%E4%BA%9B%E7%AC%A6%E5%8F%B7%E4%BB%A3%E7%A0%81">一些符号代码</a><br>
- <a href="#beamer-%E6%8F%92%E5%85%A5-logo">Beamer 插入 logo</a><br>
- <a href="#beamer-%E6%8F%92%E5%85%A5-gif">Beamer 插入 GIF</a><br>
- <a href="#%E6%97%A5%E6%9C%9F%E6%98%BE%E7%A4%BA%E4%B8%AD%E6%96%87">日期显示中文</a></li>
<li><a href="#others">Others</a><br>
- <a href="#paper-writing">paper writing</a><br>
- <a href="#%E6%8F%92%E5%85%A5%E5%9B%BE%E7%89%87">插入图片</a><br>
- <a href="#cmd-%E8%84%9A%E6%9C%AC">cmd 脚本</a><br>
- <a href="#windows-c%E7%9B%98%E7%A9%BA%E9%97%B4%E4%B8%8D%E8%B6%B3">Windows C盘空间不足</a><br>
- <a href="#jupyter-%E6%B5%8F%E8%A7%88%E5%99%A8%E8%AE%BF%E9%97%AE%E5%A4%B1%E8%B4%A5">Jupyter 浏览器访问失败</a><br>
- <a href="#usb%E5%A4%B1%E6%95%88">USB失效</a></li>
</ul>
<hr>
<h1 class="mume-header" id="python">Python</h1>
<h2 class="mume-header" id="pytorch">Pytorch</h2>
<hr>
<p>连接，返回tensor:</p>
<pre class="language-text">torch.cat(inputs, dimension=0)
</pre>
<p>分块：</p>
<pre class="language-text">torch.chunk()
</pre>
<hr>
<h3 class="mume-header" id="tensor">Tensor</h3>
<p><code>torch.tensor</code>会从data中的数据部分做拷贝（而不是直接引用），根据原始数据类型生成相应的torch.LongTensor、torch.FloatTensor和torch.DoubleTensor<br>
而 <code>torch.Tensor()</code>是python类，更明确地说，是默认张量类型<code>torch.FloatTensor()</code> 的别名，<code>torch.Tensor([1,2])</code>会调用Tensor类的构造函数<code>__init__</code>，生成单精度浮点类型的张量。</p>
<p>会改变tensor的函数操作会用一个下划线后缀来标示。比如，<code>torch.FloatTensor.abs_()</code>会在原地计算绝对值，并返回改变后的tensor，而<code>tensor.FloatTensor.abs()</code>将会在一个新的tensor中计算结果。</p>
<p>若将数据<code>a</code>由<code>torch.DoubleTensor()</code>转化为<code>torch.FloatTensor()</code>，可记为<br>
<code>a.float() 或 a.to(torch.float32)</code>。</p>
<p><img src="https://github.com/PaulTHong/SecretGarden/raw/master/images/torch_Tensor.png" alt="Tensor"></p>
<hr>
<h3 class="mume-header" id="item">item()</h3>
<p><code>item()</code>只针对仅含一个元素的张量，取出其值。若为多个元素的张量，可考虑<code>tolist()</code>。</p>
<hr>
<h3 class="mume-header" id="torchvisiontransforms">torchvision.transforms</h3>
<ul>
<li>
<p><code>torchvision.transforms.ToTensor()</code><br>
输入为<code>PIL.Image</code>类型 或<code>numpy.array</code>中的<code>numpy.uint8</code>类型时，才会对其归一化(scale)，即除以255。</p>
</li>
<li>
<p><code>transforms</code>中的一些变换如<code>Resize(), Crop()</code>等输入必为<code>Image</code>型，<code>numpy.array</code>会报错。</p>
</li>
<li>
<p>注：<code>pytorch</code>的图片接口为<code>PIL.Image</code>库，该库读取的图片类型为<code>Image.Image</code>，而<code>cv2</code>读取的图片则与<code>numpy</code>库一致，即<code>numpy.ndarray</code>型。</p>
</li>
</ul>
<hr>
<p>t 代指数据：<br>
cpu转gpu使用<code>t.cuda()</code><br>
gpu转cpu使用<code>t.cpu()</code><br>
tensor转numpy使用<code>t.numpy()</code><br>
numpy转tensor使用<code>torch.from_numpy()</code></p>
<p>cpu,gpu转variable使用<code>Variable(t)</code><br>
Variable转cpu，gpu使用<code>v.data</code><br>
注意：<code>y = Variable(t.cuda())</code>生成一个节点y，<code>y = Variable(t).cuda()</code>，生成两个计算图节点t和y</p>
<p>注：torch 0.4.0及其后续版本合并了Variable与Tensor，故Variable不再使用。<br>
GPU上的Tensor不能直接转为numpy，需先转为CPU上的Tensor再转为numpy。</p>
<hr>
<h3 class="mume-header" id="torchvisionsave_image">torchvision.save_image</h3>
<pre class="language-text">mul(255).add_(0.5).clamp(0, 255).permute(1, 2, 0).to('cpu', torch.uint8)
</pre>
<p>对于uint8类型，torch与numpy均为<strong>向下取整</strong>，故先+0.5再clamp。</p>
<pre class="language-text">torch.Tensor: a.to(torch.uint8)
numpy.array: a.astype(np.uint8)
</pre>
<hr>
<h3 class="mume-header" id="relu">Relu</h3>
<p><code>torch.nn.Relu(inplace=False)</code></p>
<p>inplace为True，将会改变输入的数据 ，否则不会改变原输入，只会产生新的输出。默认为False。</p>
<hr>
<h3 class="mume-header" id="nnconvtranspose2d">nn.ConvTranspose2d</h3>
<p><span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><msub><mi>H</mi><mrow><mi>o</mi><mi>u</mi><mi>t</mi></mrow></msub><mo>=</mo><mo stretchy="false">(</mo><msub><mi>H</mi><mrow><mi>i</mi><mi>n</mi></mrow></msub><mo>−</mo><mn>1</mn><mo stretchy="false">)</mo><mo>×</mo><mi>s</mi><mi>t</mi><mi>r</mi><mi>i</mi><mi>d</mi><mi>e</mi><mo>−</mo><mn>2</mn><mi>p</mi><mo>+</mo><mi>d</mi><mi>i</mi><mi>l</mi><mi>a</mi><mi>t</mi><mi>i</mi><mi>o</mi><mi>n</mi><mo>×</mo><mo stretchy="false">(</mo><mi>k</mi><mo>−</mo><mn>1</mn><mo stretchy="false">)</mo><mo>+</mo><mi>o</mi><mi>u</mi><mi>t</mi><mi>p</mi><mi>u</mi><mi>t</mi><mi mathvariant="normal">_</mi><mi>p</mi><mi>a</mi><mi>d</mi><mi>d</mi><mi>i</mi><mi>n</mi><mi>g</mi><mo>+</mo><mn>1</mn></mrow><annotation encoding="application/x-tex">H_{out} = (H_{in} - 1)\times stride - 2p + dilation\times (k-1) + output\_padding + 1</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:0.83333em;vertical-align:-0.15em;"></span><span class="mord"><span class="mord mathnormal" style="margin-right:0.08125em;">H</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.2805559999999999em;"><span style="top:-2.5500000000000003em;margin-left:-0.08125em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight">o</span><span class="mord mathnormal mtight">u</span><span class="mord mathnormal mtight">t</span></span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.15em;"><span></span></span></span></span></span></span><span class="mspace" style="margin-right:0.2777777777777778em;"></span><span class="mrel">=</span><span class="mspace" style="margin-right:0.2777777777777778em;"></span></span><span class="base"><span class="strut" style="height:1em;vertical-align:-0.25em;"></span><span class="mopen">(</span><span class="mord"><span class="mord mathnormal" style="margin-right:0.08125em;">H</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.31166399999999994em;"><span style="top:-2.5500000000000003em;margin-left:-0.08125em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight">in</span></span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.15em;"><span></span></span></span></span></span></span><span class="mspace" style="margin-right:0.2222222222222222em;"></span><span class="mbin">−</span><span class="mspace" style="margin-right:0.2222222222222222em;"></span></span><span class="base"><span class="strut" style="height:1em;vertical-align:-0.25em;"></span><span class="mord">1</span><span class="mclose">)</span><span class="mspace" style="margin-right:0.2222222222222222em;"></span><span class="mbin">×</span><span class="mspace" style="margin-right:0.2222222222222222em;"></span></span><span class="base"><span class="strut" style="height:0.77777em;vertical-align:-0.08333em;"></span><span class="mord mathnormal">s</span><span class="mord mathnormal">t</span><span class="mord mathnormal" style="margin-right:0.02778em;">r</span><span class="mord mathnormal">i</span><span class="mord mathnormal">d</span><span class="mord mathnormal">e</span><span class="mspace" style="margin-right:0.2222222222222222em;"></span><span class="mbin">−</span><span class="mspace" style="margin-right:0.2222222222222222em;"></span></span><span class="base"><span class="strut" style="height:0.8388800000000001em;vertical-align:-0.19444em;"></span><span class="mord">2</span><span class="mord mathnormal">p</span><span class="mspace" style="margin-right:0.2222222222222222em;"></span><span class="mbin">+</span><span class="mspace" style="margin-right:0.2222222222222222em;"></span></span><span class="base"><span class="strut" style="height:0.77777em;vertical-align:-0.08333em;"></span><span class="mord mathnormal">d</span><span class="mord mathnormal">i</span><span class="mord mathnormal" style="margin-right:0.01968em;">l</span><span class="mord mathnormal">a</span><span class="mord mathnormal">t</span><span class="mord mathnormal">i</span><span class="mord mathnormal">o</span><span class="mord mathnormal">n</span><span class="mspace" style="margin-right:0.2222222222222222em;"></span><span class="mbin">×</span><span class="mspace" style="margin-right:0.2222222222222222em;"></span></span><span class="base"><span class="strut" style="height:1em;vertical-align:-0.25em;"></span><span class="mopen">(</span><span class="mord mathnormal" style="margin-right:0.03148em;">k</span><span class="mspace" style="margin-right:0.2222222222222222em;"></span><span class="mbin">−</span><span class="mspace" style="margin-right:0.2222222222222222em;"></span></span><span class="base"><span class="strut" style="height:1em;vertical-align:-0.25em;"></span><span class="mord">1</span><span class="mclose">)</span><span class="mspace" style="margin-right:0.2222222222222222em;"></span><span class="mbin">+</span><span class="mspace" style="margin-right:0.2222222222222222em;"></span></span><span class="base"><span class="strut" style="height:1.00444em;vertical-align:-0.31em;"></span><span class="mord mathnormal">o</span><span class="mord mathnormal">u</span><span class="mord mathnormal">tp</span><span class="mord mathnormal">u</span><span class="mord mathnormal">t</span><span class="mord" style="margin-right:0.02778em;">_</span><span class="mord mathnormal">p</span><span class="mord mathnormal">a</span><span class="mord mathnormal">dd</span><span class="mord mathnormal">in</span><span class="mord mathnormal" style="margin-right:0.03588em;">g</span><span class="mspace" style="margin-right:0.2222222222222222em;"></span><span class="mbin">+</span><span class="mspace" style="margin-right:0.2222222222222222em;"></span></span><span class="base"><span class="strut" style="height:0.64444em;vertical-align:0em;"></span><span class="mord">1</span></span></span></span></p>
<p>反卷积即对应上采样过程，HW大小的计算对应卷积计算公式的逆过程。 nn.ConvTranspose2d函数默认参数为：dilation=1, output_padding=0。</p>
<hr>
<h3 class="mume-header" id="torchvision">torchvision</h3>
<p><code>toTensor()</code> 中有归一化，且若是二维会扩成三维 <code>[:, :, None]</code> 。<br>
<code>ImageFolder</code> 类中 <code>pil_loader</code>返回时会<code>img.convert('RGB')</code> ; classes 为 [name], class_to_idx 为 {name: idx}, samples 为 [(path, target)] 。</p>
<hr>
<h3 class="mume-header" id="cv2%E7%9A%84%E5%9D%91">cv2的坑</h3>
<p>因cv2读取图片为BGR格式，若想将其转为RGB格式后转为tensor,<br>
<code>img = img[:, :, -1]</code> 后使用 <code>torch.from_numpy(img)</code> 会报错。<br>
解决方案一：<br>
<code>img = img[:, :, -1].copy()</code><br>
解决方案二：<br>
<code>img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)</code></p>
<hr>
<h3 class="mume-header" id="%E8%B0%83%E8%8A%82%E5%AD%A6%E4%B9%A0%E7%8E%87">调节学习率</h3>
<p>在Pytorch 1.1.0及以后的版本中，应先更新优化器optimizer，再更新学习率，代码框架可如下所示：</p>
<pre class="language-text">scheduler = ...
for epoch in range(100):
train(...)
validate(...)
scheduler.step()
</pre>
<p><a href="https://pytorch.org/docs/stable/optim.html?highlight=lr#torch.optim.lr_scheduler.MultiStepLR" title="官方文档">How to adjust Learning Rate</a></p>
<blockquote>
<p>Prior to PyTorch 1.1.0, the learning rate scheduler was expected to be called before the optimizer’s update; 1.1.0 changed this behavior in a BC-breaking way. If you use the learning rate scheduler (calling scheduler.step()) before the optimizer’s update (calling optimizer.step()), this will skip the first value of the learning rate schedule. If you are unable to reproduce results after upgrading to PyTorch 1.1.0, please check if you are calling scheduler.step() at the wrong time.</p>
</blockquote>
<hr>
<h3 class="mume-header" id="torchnndataparallel">torch.nn.DataParallel()</h3>
<p>model, optimizer 等均可用DataParallel包裹，即表示用多块GPU训练。（CUDA_VISIBLE_DEVICES=0,1,2 此种方式仍为单卡训练，只是占用多块卡的显存。）</p>
<p>以model为例，其一旦被DataParallel包裹之后，其对应的参数state_dict的keys前会多七个字符，即<code>module.</code>。所以在读写checkpoint时需注意，单卡时不能有<code>module.</code>，所以读取一个多卡训练的checkpoint，中间需加入<code>.module</code>，即由<code>model.state_dict()</code> 变为<code>model.module.state_dict()</code>，其实就相当于把读取的参数字典的keys去掉了前七个字符<code>module.</code>。<br>
当然了，若果存储模型时就选择单卡型，即</p>
<pre class="language-text">torch.save({'model': model.module.state_dict()}, save_path)
</pre>
<p>则读取时就不需进行去module的操作。<br>
同理，读取单卡checkpoint进行多卡训练时，按单卡代码定义好model（注意此时从CPU转到GPU上，即末尾加个<code>.cuda()</code>或<code>.to(device)</code>，device需定义一下，可为<code>'cuda'</code>）,optimizer等，最后加一个</p>
<pre class="language-text">model = torch.nn.DataParallel(model)
</pre>
<p>即可，此句代码相当于在model的state_dict的keys前加了个<code>module.</code>。</p>
<p>总结:<code>torch.nn.DataParallel()</code>相当于在checkpoint的state_dict()的keys前加上了<code>module.</code>，意味着对应多卡；单卡的state_dict()则无<code>module</code>，<code>model.module.state_dict()</code>则为去掉了<code>module.</code>的state_dict()。</p>
<p>参考：<a href="https://blog.csdn.net/kaixinjiuxing666/article/details/85115077">Missing key(s) in state_dict: Unexpected key(s) in state_dict:</a></p>
<h3 class="mume-header" id="torchnndistributeddataparallel">torch.nn.DistributedDataParallel</h3>
<p>这才是真正的分布式训练，且为官方推荐，即使是在单节点多卡上使用时。简单说明一下分布式的常用参数：</p>
<pre data-role="codeBlock" data-info class="language-"><code>--dist_url 通信地址，rank为0的进程在这个地址对应的机器上运行。可取为127.0.0.1:8888，8888为任意可用端口，在同一节点上若运行了多个分布式程序，端口需不同，否则会报addres不可用的错误；另IP笔者写为实际IP时程序一直卡着不跑，改为127.0.0.1就好了。
--world_size 进程数量，一般为节点数x每个节点上的GPU数，一个GPU对应一个进程。
--rank 进程的编号，从0开始。因为多进程的重复输出等现象，建议输出日志、存储模型等时调加一句if语句就可以了，如当前rank为0才输出，if rank == 0:
</code></pre><p>当用<code>python -m torch.distributed.launch *py</code>训练时，<code>*.py</code>中需要一个args参数为<code>local_rank</code>。</p>
<p>笔者在单节点多卡上进行分布式训练时，习惯用<code>CUDA_VISIBLE_DEVICES=2,3 python *.py</code>的方式来运行，此时程序会找到编号为2，3的两块GPU，且给它们的rank即为0和1。</p>
<p>在argparse里有一个<code>local_rank</code>参数：</p>
<pre data-role="codeBlock" data-info class="language-"><code>parser.add_argument("--local_rank", type=int, default=-1, help="number of cpu threads to use during batch generation")
</code></pre><p>通过<code>torch.distributed.launch</code>来启动训练，<code>torch.distributed.launch</code> 会给模型分配一个args.local_rank的参数，所以在训练代码中要解析这个参数，也可以通过<code>torch.distributed.get_rank()</code>获取进程id。</p>
<p>关于ImageNet等的分布式训练，可参考Github代码: <a href="https://github.com/richardkxu/distributed-pytorch">distributed-pytorch</a></p>
<hr>
<h3 class="mume-header" id="num_workers">num_workers</h3>
<p><code>torch.utils.data.DataLoader</code>常以batch的方式读取数据，其参数<code>num_workers</code>表示所用核数（并行读取），<code>num_workers=0</code>表示只用一个主进程读取。</p>
<p>在笔者用风格迁移做数据增强的实验中，风格迁移本身需调用一个VGG网络，而分类采用的ResNet50网络。继承了<code>torch.utils.Dataset</code>类，在里面加入了style_transform操作，从而相当于在<code>DataLoader</code>阶段数据就需要放到cuda里。</p>
<p>采取<code>torch.nn.DataParallel</code>单节点多卡训练时，会报错<code>RuntimeError: CUDA error: initialization error</code>，解决办法一只要把<code>num_workers</code>设为0就解决了；但考虑到<code>num_workers=4</code>等可能会提升速度，笔者还是尝试着去解决，在主函数的<code>if __name__ =='__main__':</code>后加入</p>
<pre class="language-text">import multiprocessing as mp
mp.set_start_method('spawn')
</pre>
<p>便解决了，意为用多进程读取数据。（注意此时这两行代码必须得写在<code>if __name__ =='__main__':</code>后，不然也可能报错。）</p>
<p>而同样的代码改为<code>torch.nn.DistributedDataParallel</code>版本的分布式单节点多卡训练后，<code>num_workers=0</code>好像也报错了，猜测是分布式本身就是多进程，与只有一个主进程读取数据矛盾。然后改为<code>num_workers=4</code>报了个风格迁移的网络的模型参数weight和输入weight不在一张卡上，稍微修改了下vgg.cuda()和decoder.cuda()在代码中的位置，用了个<code>.to(torch.cuda.current_device())</code>，便解决了，因为分布式下每个进程会使用一块GPU，此时load数据时保证风格迁移网络和数据输入都在这张卡上，就没有问题了。</p>
<hr>
<h3 class="mume-header" id="device">device</h3>
<p><code>torch.cuda.current_device()</code>返回的是一个整数，如0 1，而获取某张量当前的device，用str()包裹后即为'cuda:0'的形式。</p>
<pre data-role="codeBlock" data-info class="language-"><code>a = torch.tensor([2, 3])
device = torch.cuda.current_device() # 0 int
b = a.to(device) # a: tensor([2, 3]) b: tensor([2, 3], device='cuda:0')
b.device # device(type='cuda', index=0)
str(b.device) # 'cuda:0'
</code></pre><hr>
<h3 class="mume-header" id="multinomial">multinomial</h3>
<pre class="language-text">torch.multinomial(input, num_samples,replacement=False, out=None) → LongTensor
</pre>
<p>按权重张量input的概率采样num_samples次。</p>
<p>参考：<a href="https://blog.csdn.net/monchin/article/details/79787621">torch.multinomial()理解</a></p>
<hr>
<h3 class="mume-header" id="load_lua-torchfileload">load_lua -> torchfile.load</h3>
<p>pytorch由0.4版本升级为1.0+版本后，一些函数会发生变化。<br>
对于训好的老式参数模型，读取函数由<code>load_lua</code>变为<code>torchfile.load</code>。<br>
在一次实际操作中，记读取的模型为<code>vgg</code>，则其第一层的权重调用方式由<code>vgg.get(0).weight</code> 变为<code>vgg.modules[0].weight</code>。</p>
<hr>
<h3 class="mume-header" id="hook">hook</h3>
<p>获取中间层feature map时经常利用hook函数，如<code>register_forward_hook(fn)</code>，而其参数<code>fn</code>为一函数，定义形式为<code>fn(module, input, output)</code>，其中的 input 和 output 均为 tuple 型，里面元素为 Tensor。</p>
<hr>
<h3 class="mume-header" id="torchsummary">torchsummary</h3>
<p>可借助<code>torchsummary</code>库查看网络结构，包括各层feature map的大小维度。</p>
<pre class="language-text">from torchsummary import summary
summary(net, (3, 32, 32), device='gpu')
</pre>
<hr>
<h3 class="mume-header" id="cprofile">cProfile</h3>
<p>借助cProfile库观测代码各部分消耗时间。<br>
如下第一行所示，在原有python运行脚本汇总加入 -m 和 -o 参数即可，其中 -o 表示输出文件名。之后在对自定义命名的 time.profile 运行第二行即可，<code>p.sort_stats('time').print_stats(50)</code> 表示按运行时间降序，只显示前五十项。</p>
<pre class="language-text">python -m cProfile -o time.profile tmp.py
python -c "import pstats; p=stats.Stats('time.profile'); p.sort_stats('time').print_stats(50)" > time.print
</pre>
<hr>
<h3 class="mume-header" id="%E5%A4%9A%E8%BF%9B%E7%A8%8B%E4%B9%8B%E9%97%B4%E5%85%B1%E4%BA%AB%E5%85%A8%E5%B1%80%E5%8F%98%E9%87%8F">多进程之间共享全局变量</h3>
<p>在做ISBI数据集的实验时，valid时会开个子进程来调用路径为script_path的java脚本，其会计算指标值。而想在主进程中获取指标值时，便可借助subprocess库了。</p>
<pre data-role="codeBlock" data-info class="language-"><code>import subprocess
# subprocess.call([Fiji_path, script_path, original_label, proposal_label, tmp_file])  # 不需要子进程返回信息时运行此行即可
return_info = subprocess.Popen([Fiji_path, script_path, original_label, proposal_label, tmp_file], shell=False, stdout=subprocess.PIPE)
while return_info.poll() is None: 
line = return_info.stdout.readline() 
    line = line.strip().decode('utf-8')
    ...
</code></pre><p>当然了，如果使用子进程只是为了获取一个值的话有些大材小用。比如在此实验的java脚本中把指标值保存在一个文件里，然后python代码里再读取该文件即可。</p>
<p>此外查到有一个python多进程共享变量Value：</p>
<pre data-role="codeBlock" data-info class="language-"><code>multiprocessing.Value(typecode_or_type, *args[, lock])
</code></pre><p>比如<code>Value("i", 0)</code>，i表示整型，两个字节。"d" 表示float?<br>
再比如<code>import ctypes; Value(ctypes.c_float, 1.0)</code>。</p>
<p>参考：<a href="https://blog.csdn.net/qq_36653505/article/details/85254909">Python中多进程间通信（multiprocessing.Manager)</a></p>
<hr>
<h2 class="mume-header" id="tensorflow">Tensorflow</h2>
<hr>
<h3 class="mume-header" id="tfsession">tf.Session()</h3>
<p><code>sess.run()</code> 返回的不是张量类型，为numpy类型，如<code>np.ndarray</code>, <code>np.float32</code>等。</p>
<p>tensorflow由Session.run()或eval()返回的任何张量都是numpy数组类型。</p>
<pre class="language-text">with tf.Session().as_default() as sess:
code A
code B
</pre>
<p>加了<code>as_default()</code>后会话结束仍可输出<code>run(), eval()</code>的值，即在code B对应的代码块仍可调用这些函数。</p>
<hr>
<h3 class="mume-header" id="tensorflow-%E4%B9%8B-checkpoint">tensorflow 之 checkpoint</h3>
<pre class="language-text">tf.train.get_checkpoint_state(checkpoint_dir,latest_filename=None)
</pre>
<p>该函数返回的是checkpoint文件CheckpointState proto类型的内容，其中有model_checkpoint_path和all_model_checkpoint_paths两个属性。其中model_checkpoint_path保存了最新的tensorflow模型文件的文件名，all_model_checkpoint_paths则有未被删除的所有tensorflow模型文件的文件名。</p>
<p>参考：<a href="https://blog.csdn.net/changeforeve/article/details/80268522">https://blog.csdn.net/changeforeve/article/details/80268522</a></p>
<hr>
<h3 class="mume-header" id="%E9%80%89%E6%8B%A9gpu">选择GPU</h3>
<ol>
<li>
<p><code>tf.device('/gpu:2')</code><br>
虽然指定了第2块GPU来训练，但是其它几个GPU也还是被占用，只是模型训练的时候，是在第2块GPU上进行。</p>
</li>
<li>
<p><code>os.environ['CUDA_VISIBLE_DEVICES']='2'</code><br>
在训练模型的时候，使用了第2块GPU，并且其它几块GPU也没有被占用，这种就相当于在我们运行程序的时候，将除第2块以外的GPU全部屏蔽了，只有第2块GPU对当前运行的程序是可见的。同样，如果要指定2，3块GPU来训练，则上面的代码中的<code>'2'</code>改成<code>'2, 3'</code>即可。</p>
</li>
<li>
<p><code>CUDA_VISIBLE_DEVICES=2 python train.py</code><br>
在终端中运行命令时选择GPU。</p>
</li>
</ol>
<hr>
<h3 class="mume-header" id="tfnn">tf.nn</h3>
<blockquote>
<ol>
<li>如果只是想快速了解一下大概，不建议使用<code>tf.nn.conv2d</code>类似的函数，可以使用<code>tf.layers</code>和<code>tf.contrib.layers</code>高级函数。</li>
<li>当有了一定的基础后，如果想在该领域进行深入学习，建议使用<code>tf.nn.conv2d</code>搭建神经网络，此时会帮助你深入理解网络中参数的具体功能与作用，而且对于loss函数需要进行正则化的时候很便于修改，能很清晰地知道修改的地方。而如果采用<code>tf.layers</code>和<code>tf.contrib.layers</code>高级函数，由于函数内部有正则项，不利于深入理解。而且如果编写者想自定义loss，此时比较困难，如果读者想共享参数，计算loss函数中的正则项时，应该只计算一次，如果采用高级函数可能不清楚到底如何计算的。</li>
</ol>
</blockquote>
<hr>
<h3 class="mume-header" id="%E5%8D%B7%E7%A7%AF%E6%8E%A2%E8%AE%A8">卷积探讨</h3>
<p>输出大小m与输入大小n的关系，其中p表示补丁padding大小，k表示卷积核kernel大小，s表示滑动步长stride：</p>
<p><span class="katex-display"><span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML" display="block"><semantics><mrow><mi>m</mi><mo>=</mo><mfrac><mrow><mi>n</mi><mo>+</mo><mn>2</mn><mo>∗</mo><mi>p</mi><mo>−</mo><mi>k</mi></mrow><mi>s</mi></mfrac><mo>+</mo><mn>1</mn></mrow><annotation encoding="application/x-tex">m = \frac{n + 2*p - k}{s} + 1</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:0.43056em;vertical-align:0em;"></span><span class="mord mathnormal">m</span><span class="mspace" style="margin-right:0.2777777777777778em;"></span><span class="mrel">=</span><span class="mspace" style="margin-right:0.2777777777777778em;"></span></span><span class="base"><span class="strut" style="height:2.0574399999999997em;vertical-align:-0.686em;"></span><span class="mord"><span class="mopen nulldelimiter"></span><span class="mfrac"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:1.3714399999999998em;"><span style="top:-2.314em;"><span class="pstrut" style="height:3em;"></span><span class="mord"><span class="mord mathnormal">s</span></span></span><span style="top:-3.23em;"><span class="pstrut" style="height:3em;"></span><span class="frac-line" style="border-bottom-width:0.04em;"></span></span><span style="top:-3.677em;"><span class="pstrut" style="height:3em;"></span><span class="mord"><span class="mord mathnormal">n</span><span class="mspace" style="margin-right:0.2222222222222222em;"></span><span class="mbin">+</span><span class="mspace" style="margin-right:0.2222222222222222em;"></span><span class="mord">2</span><span class="mspace" style="margin-right:0.2222222222222222em;"></span><span class="mbin">∗</span><span class="mspace" style="margin-right:0.2222222222222222em;"></span><span class="mord mathnormal">p</span><span class="mspace" style="margin-right:0.2222222222222222em;"></span><span class="mbin">−</span><span class="mspace" style="margin-right:0.2222222222222222em;"></span><span class="mord mathnormal" style="margin-right:0.03148em;">k</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.686em;"><span></span></span></span></span></span><span class="mclose nulldelimiter"></span></span><span class="mspace" style="margin-right:0.2222222222222222em;"></span><span class="mbin">+</span><span class="mspace" style="margin-right:0.2222222222222222em;"></span></span><span class="base"><span class="strut" style="height:0.64444em;vertical-align:0em;"></span><span class="mord">1</span></span></span></span></span></p>
<p>那么当不能整除时，各大框架如何处理呢？</p>
<p>先温故一下两大框架的2维卷积函数：</p>
<pre class="language-text">tf.nn.conv2d(input, filter, strides, padding)
</pre>
<p>其中input大小为[batch, height, width, channel_in]， filter大小为 [height_kernel, width_kernel, channel_in, channel_out], strides为 [1, stride, stride, 1], padding有'SAME'和'VALID'两个选项。</p>
<pre class="language-text">torch.nn.Conv2d(in_c, out_c, kernel_size, stride=1, padding=0, dilation=1, groups=1, bias=True)
</pre>
<p>其中参数一看就明白（论torch的可读性），就不赘述了。</p>
<p><strong>Tensorflow</strong>卷积池化均<strong>向上取整</strong>，简单粗暴。其有'SAME'和'VALID'两种补丁模式：前者超过原图边界处用0填充，当kernel为奇数时，padding可能只补一边；后者确保不超过边界，可能会丢失一些信息。<br>
要保持图片尺寸不变，看一个特例，即常用stride=1，且kernel为奇数，此时只需 k-2p=1 即可。</p>
<p><strong>PyTorch</strong>则<strong>向下取整</strong>。<br>
以Resnet经典一层（[1x1, 3x3, 1x1] + [1x1] shortcut）为例，padding为0输出大小为：<br>
<span class="katex-display"><span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML" display="block"><semantics><mrow><mi>m</mi><mo>=</mo><mo stretchy="false">⌊</mo><mfrac><mrow><mi>n</mi><mo>−</mo><mn>1</mn></mrow><mi>s</mi></mfrac><mo>+</mo><mn>1</mn><mo stretchy="false">⌋</mo></mrow><annotation encoding="application/x-tex">m = \lfloor \frac{n-1}{s} + 1 \rfloor</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:0.43056em;vertical-align:0em;"></span><span class="mord mathnormal">m</span><span class="mspace" style="margin-right:0.2777777777777778em;"></span><span class="mrel">=</span><span class="mspace" style="margin-right:0.2777777777777778em;"></span></span><span class="base"><span class="strut" style="height:2.00744em;vertical-align:-0.686em;"></span><span class="mopen">⌊</span><span class="mord"><span class="mopen nulldelimiter"></span><span class="mfrac"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:1.32144em;"><span style="top:-2.314em;"><span class="pstrut" style="height:3em;"></span><span class="mord"><span class="mord mathnormal">s</span></span></span><span style="top:-3.23em;"><span class="pstrut" style="height:3em;"></span><span class="frac-line" style="border-bottom-width:0.04em;"></span></span><span style="top:-3.677em;"><span class="pstrut" style="height:3em;"></span><span class="mord"><span class="mord mathnormal">n</span><span class="mspace" style="margin-right:0.2222222222222222em;"></span><span class="mbin">−</span><span class="mspace" style="margin-right:0.2222222222222222em;"></span><span class="mord">1</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.686em;"><span></span></span></span></span></span><span class="mclose nulldelimiter"></span></span><span class="mspace" style="margin-right:0.2222222222222222em;"></span><span class="mbin">+</span><span class="mspace" style="margin-right:0.2222222222222222em;"></span></span><span class="base"><span class="strut" style="height:1em;vertical-align:-0.25em;"></span><span class="mord">1</span><span class="mclose">⌋</span></span></span></span></span><br>
当s=1时，大小不变；当s=2时，若输入n为偶数则m为其一半，为奇数则m相当于对n/2向上取整。</p>
<p>当padding只补一边时很有意思，caffe补左上，Tensorflow补右下，Pytorch补一圈（仍要保证大小不变时有待探究）。</p>
<hr>
<h3 class="mume-header" id="%E6%84%9F%E5%8F%97%E9%87%8E">感受野</h3>
<p><span class="katex-display"><span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML" display="block"><semantics><mrow><mi>R</mi><msub><mi>F</mi><mi>n</mi></msub><mo>=</mo><mi>R</mi><msub><mi>F</mi><mrow><mi>n</mi><mo>−</mo><mn>1</mn></mrow></msub><mo>+</mo><mo stretchy="false">(</mo><mi>k</mi><mi>e</mi><mi>r</mi><mi>n</mi><mi>e</mi><mi>l</mi><mi mathvariant="normal">_</mi><mi>s</mi><mi>i</mi><mi>z</mi><mi>e</mi><mo>−</mo><mn>1</mn><mo stretchy="false">)</mo><mo>∗</mo><mi>s</mi><mi>t</mi><mi>r</mi><mi>i</mi><mi>d</mi><mi>e</mi></mrow><annotation encoding="application/x-tex">RF_n = RF_{n-1} + (kernel\_size - 1) * stride</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:0.83333em;vertical-align:-0.15em;"></span><span class="mord mathnormal" style="margin-right:0.00773em;">R</span><span class="mord"><span class="mord mathnormal" style="margin-right:0.13889em;">F</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.151392em;"><span style="top:-2.5500000000000003em;margin-left:-0.13889em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight">n</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.15em;"><span></span></span></span></span></span></span><span class="mspace" style="margin-right:0.2777777777777778em;"></span><span class="mrel">=</span><span class="mspace" style="margin-right:0.2777777777777778em;"></span></span><span class="base"><span class="strut" style="height:0.891661em;vertical-align:-0.208331em;"></span><span class="mord mathnormal" style="margin-right:0.00773em;">R</span><span class="mord"><span class="mord mathnormal" style="margin-right:0.13889em;">F</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.301108em;"><span style="top:-2.5500000000000003em;margin-left:-0.13889em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight">n</span><span class="mbin mtight">−</span><span class="mord mtight">1</span></span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.208331em;"><span></span></span></span></span></span></span><span class="mspace" style="margin-right:0.2222222222222222em;"></span><span class="mbin">+</span><span class="mspace" style="margin-right:0.2222222222222222em;"></span></span><span class="base"><span class="strut" style="height:1.06em;vertical-align:-0.31em;"></span><span class="mopen">(</span><span class="mord mathnormal" style="margin-right:0.03148em;">k</span><span class="mord mathnormal" style="margin-right:0.02778em;">er</span><span class="mord mathnormal">n</span><span class="mord mathnormal">e</span><span class="mord mathnormal" style="margin-right:0.01968em;">l</span><span class="mord" style="margin-right:0.02778em;">_</span><span class="mord mathnormal">s</span><span class="mord mathnormal">i</span><span class="mord mathnormal">ze</span><span class="mspace" style="margin-right:0.2222222222222222em;"></span><span class="mbin">−</span><span class="mspace" style="margin-right:0.2222222222222222em;"></span></span><span class="base"><span class="strut" style="height:1em;vertical-align:-0.25em;"></span><span class="mord">1</span><span class="mclose">)</span><span class="mspace" style="margin-right:0.2222222222222222em;"></span><span class="mbin">∗</span><span class="mspace" style="margin-right:0.2222222222222222em;"></span></span><span class="base"><span class="strut" style="height:0.69444em;vertical-align:0em;"></span><span class="mord mathnormal">s</span><span class="mord mathnormal">t</span><span class="mord mathnormal" style="margin-right:0.02778em;">r</span><span class="mord mathnormal">i</span><span class="mord mathnormal">d</span><span class="mord mathnormal">e</span></span></span></span></span></p>
<p>RF(n): 当前层感受野<br>
RF(n-1): 上一层感受野<br>
kernel_size: 当前层卷积核大小<br>
stride: 之前所有层stride的乘积</p>
<p>特殊: 二维stride=1时，共<span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>l</mi></mrow><annotation encoding="application/x-tex">l</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:0.69444em;vertical-align:0em;"></span><span class="mord mathnormal" style="margin-right:0.01968em;">l</span></span></span></span>层，每层kernel大小均为<span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>k</mi></mrow><annotation encoding="application/x-tex">k</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:0.69444em;vertical-align:0em;"></span><span class="mord mathnormal" style="margin-right:0.03148em;">k</span></span></span></span>（即<span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>k</mi><mo>×</mo><mi>k</mi></mrow><annotation encoding="application/x-tex">k \times k</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:0.77777em;vertical-align:-0.08333em;"></span><span class="mord mathnormal" style="margin-right:0.03148em;">k</span><span class="mspace" style="margin-right:0.2222222222222222em;"></span><span class="mbin">×</span><span class="mspace" style="margin-right:0.2222222222222222em;"></span></span><span class="base"><span class="strut" style="height:0.69444em;vertical-align:0em;"></span><span class="mord mathnormal" style="margin-right:0.03148em;">k</span></span></span></span>），则最后一层像素点感受野为<br>
<span class="katex-display"><span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML" display="block"><semantics><mrow><mo stretchy="false">(</mo><mi>k</mi><mo>−</mo><mn>1</mn><mo stretchy="false">)</mo><mo>∗</mo><mi>l</mi><mo>+</mo><mn>1</mn></mrow><annotation encoding="application/x-tex">(k - 1) * l + 1</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:1em;vertical-align:-0.25em;"></span><span class="mopen">(</span><span class="mord mathnormal" style="margin-right:0.03148em;">k</span><span class="mspace" style="margin-right:0.2222222222222222em;"></span><span class="mbin">−</span><span class="mspace" style="margin-right:0.2222222222222222em;"></span></span><span class="base"><span class="strut" style="height:1em;vertical-align:-0.25em;"></span><span class="mord">1</span><span class="mclose">)</span><span class="mspace" style="margin-right:0.2222222222222222em;"></span><span class="mbin">∗</span><span class="mspace" style="margin-right:0.2222222222222222em;"></span></span><span class="base"><span class="strut" style="height:0.77777em;vertical-align:-0.08333em;"></span><span class="mord mathnormal" style="margin-right:0.01968em;">l</span><span class="mspace" style="margin-right:0.2222222222222222em;"></span><span class="mbin">+</span><span class="mspace" style="margin-right:0.2222222222222222em;"></span></span><span class="base"><span class="strut" style="height:0.64444em;vertical-align:0em;"></span><span class="mord">1</span></span></span></span></span></p>
<hr>
<h3 class="mume-header" id="resnet">ResNet</h3>
<p>Bottleneck每个block出去channel 为 planes * expansion, 如 512 * 4 。</p>
<hr>
<h3 class="mume-header" id="ncwh">(N,C,W,H)</h3>
<p>tensorflow默认为NHWC，其访存局部性更好；而torch的NCHW为GPU推荐方式。</p>
<h3 class="mume-header" id="%E4%BC%98%E5%8C%96%E5%99%A8">优化器</h3>
<p>据说SGD比ADAM稳定。</p>
<h3 class="mume-header" id="training-accuracy">training accuracy</h3>
<p>某些情况下全体数据集上的training accuracy显示为100%时不一定为100%，如在采用BatchNormalization模块时，<span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>μ</mi></mrow><annotation encoding="application/x-tex">\mu</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:0.625em;vertical-align:-0.19444em;"></span><span class="mord mathnormal">μ</span></span></span></span>和<span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>σ</mi></mrow><annotation encoding="application/x-tex">\sigma</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:0.43056em;vertical-align:0em;"></span><span class="mord mathnormal" style="margin-right:0.03588em;">σ</span></span></span></span>随着batch变化而变化，而训练集上的准确度是以batch为单位来测量的。</p>
<hr>
<h2 class="mume-header" id="python-1">Python</h2>
<hr>
<h3 class="mume-header" id="print">print</h3>
<p>不换行输出，<code>\r</code>表示会回到行头：<br>
<code>print(*, end='\r')</code></p>
<hr>
<h3 class="mume-header" id="npclip">np.clip()</h3>
<p>上下界截取。</p>
<hr>
<h3 class="mume-header" id="nprandomchoice">np.random.choice()</h3>
<p><code>random.choice()</code>函数每次只能选择一个，而<code>np.random.choice()</code>则可选择多个，但需注意一个默认的参数<code>replace=True</code>，表示选取的元素可能重复。</p>
<p>如下代码表示从<code>a</code>中不重复地选取三个元素，<code>a</code>可以是<code>list</code>、<code>np.array</code>等类型，其中每一个元素被选到的比例均记录在参数<code>p</code>中。<br>
<code>np.random.choice(a, 3, replace=False, p=[*])</code></p>
<hr>
<h3 class="mume-header" id="%E6%8E%92%E5%BA%8F">排序</h3>
<pre class="language-text">a.sort() a也变 返回 None
sorted(a) a不变 返回 变后结果
</pre>
<hr>
<h3 class="mume-header" id="zip">zip</h3>
<p>python3中 <code>zip()</code> 返回iterator，没有<code>.sort()</code>属性, 可<code>list(zip())</code>再调用<code>sort()</code> 或者 直接<code>sorted(zip())</code>。</p>
<p><code>zip(*a)</code> 解压</p>
<hr>
<h3 class="mume-header" id="eval">eval()</h3>
<p><code>eval()</code>函数十分强大，官方demo解释为：将字符串str当成有效的表达式来求值并返回计算结果。<code>exec()</code>功能类似，执行python语句。</p>
<hr>
<h3 class="mume-header" id="f-string">f-string</h3>
<pre class="language-text">f"string"
</pre>
<p>类似str.format{}接受的格式字符串。<br>
注：Python3.6才开始有。</p>
<p>参考：<a href="https://blog.csdn.net/s740556472/article/details/81111493">python3.6 新特性：f-string PEP 498: Formatted string literals</a></p>
<hr>
<h3 class="mume-header" id="glob">glob</h3>
<p><code>glob.glob()</code>函数，里面的通配符匹配，在Windows下是不区分大小写的，而在Linux下是区分大小写的。</p>
<p>故比如在Windows中读取图片时，*.jpg和*.JPG若都放在形参里，则会读取两次，注意。</p>
<p>另需注意<code>glob.glob()</code>搜索时需有分隔符，即 \ 或 / ，不然搜索结果为空。</p>
<p>推荐写法：</p>
<pre class="language-text">impot glob
import os
imgs = glob.glob(os.path.join(img_path, '*.jpg'))
for img in imgs:
print(os.path.split(img)[-1])
</pre>
<p>总结：<code>glob.glob</code>的参数是一个只含有方括号、问号、正斜线的正则表达式，同时也是shell命令。</p>
<hr>
<h3 class="mume-header" id="ospathdirname__file__">os.path.dirname(__file__)</h3>
<p>在脚本test.py里写入<code>print(os.path.dirname(__file__))</code>。</p>
<ul>
<li>当脚本是以完整路径被运行的， 那么将输出该脚本所在的完整路径，比如：<pre data-role="codeBlock" data-info class="language-"><code>python d:/pythonSrc/test/test.py
输出: d:/pythonSrc/test
</code></pre></li>
<li>当脚本是以相对路径被运行的， 那么将输出空目录，比如：<pre data-role="codeBlock" data-info class="language-"><code>python test.py
输出: 空字符串
</code></pre></li>
</ul>
<h3 class="mume-header" id="argparse">argparse</h3>
<pre class="language-text">import argparse
parser = argparse.ArgumentParser()
parser.add_argument('-n', '--name')
args = parser.parse_args()
</pre>
<p>当 <code>-n</code> 和 <code>--name</code> 同时存在时，默认采用后者。命令行中输入二者均可（如 <code>python test.py -n ht</code>），代码中调用时用全称即 <code>args.name</code>。</p>
<h4 class="mume-header" id="bool%E5%9E%8Bargparse-%E5%9D%91">bool型argparse 坑</h4>
<p>在使用argparse时发现无法传递bool型变量，无论命令行输入True还是False，解析出来之后都是True。因为输入首先均作为str类型处理。</p>
<p>出错版本：代码<code>test.py</code>中</p>
<pre class="language-text">parser.add_argument('--trained', type=bool, default=False)
</pre>
<p>解决办法1：<br>
注册自定义回调函数：</p>
<pre class="language-text">def str2bool(v):
if v.lower() in ('yes', 'true', 't', 'y', '1'):
return True
elif v.lower() in ('no', 'false', 'f', 'n', '0'):
return False
else:
raise argparse.ArgumentTypeError('Unsupported value encountered.')
</pre>
<p>从而将type由bool改为str2bool即可：</p>
<pre class="language-text">parser.add_argument('--trained', type=str2bool, default=False)
</pre>
<p>解决办法2：<br>
将bool型变为str型：</p>
<pre class="language-text">parser.add_argument('--trained', type=str, default='False')
</pre>
<p>在主函数中对应判断稍加修改：</p>
<pre class="language-text">if args.trained == 'False':
code1
elif args.trained == 'True'
code2
</pre>
<p>解决办法3：(推荐!!!)<br>
argparse有参数action, 取值有两种：</p>
<pre class="language-text">action='store_true'
action='store_false'
</pre>
<p>此时运行代码的命令行只需要参数名 <code>--trained</code> 即可，其后不需输入具体参数。</p>
<p><code>action='store_true'</code>意味着<code>trained</code>取值默认为<code>False</code>。即<code>python test.py</code>时<code>trained</code>取值为<code>False</code>；而<code>python test.py --trained</code>时<code>trained</code>取值变为<code>True</code>。<br>
若<code>argparse()</code>中添加<code>default</code>, 则<code>trained</code>默认取值与<code>default</code>值相同。<br>
<code>action='store_true'</code>分析同理。</p>
<p>参考：<a href="https://stackoverflow.com/questions/15008758/parsing-boolean-values-with-argparse">https://stackoverflow.com/questions/15008758/parsing-boolean-values-with-argparse</a></p>
<hr>
<h3 class="mume-header" id="config">config</h3>
<p>很多代码库会通过<code>yacs</code>库来控制实验参数<code>config</code>。<br>
After <code>cfg.freeze()</code>, if you want to modfiy cfg, you have to <code>cfg.defrost()</code>。</p>
<hr>
<h3 class="mume-header" id="class">class</h3>
<p>class里面有多个类的属性时，如多个全连接层fc1, fc2, fc3：</p>
<ol>
<li>
<p>采用setattr 和getattr</p>
<pre class="language-text"> setattr(‘fc%i’, i)
</pre>
</li>
<li>
<p>直接self里构造一个list</p>
</li>
</ol>
<hr>
<h3 class="mume-header" id="__call__">__call__()</h3>
<blockquote>
<p>所有的函数都是可调用对象。</p>
</blockquote>
<blockquote>
<p>一个类实例也可以变成一个可调用对象，只需要实现一个特殊方法<code>__call__()</code></p>
</blockquote>
<hr>
<h3 class="mume-header" id="__dir__">__dir__()</h3>
<p>取类的属性，如<code>a.__dir__()</code>，<code>a</code>表示一个类的对象。</p>
<hr>
<h3 class="mume-header" id="python%E5%87%BD%E6%95%B0%E4%BC%A0%E5%AF%B9%E8%B1%A1call-by-object">Python函数——传对象(call by object)</h3>
<blockquote>
<p>结论：Python不允许程序员选择采用传值还是传引用。Python参数传递采用的肯定是“传对象引用”的方式。这种方式相当于传值和传引用的一种综合。如果函数收到的是一个<strong>可变对象</strong>（比如字典或者列表）的引用，就能修改对象的原始值－－相当于通过“<strong>传引用</strong>”来传递对象。如果函数收到的是一个<strong>不可变对象</strong>（比如数字、字符或者元组）的引用，就不能直接修改原始对象－－相当于通过“<strong>传值</strong>'来传递对象。</p>
</blockquote>
<p>测试：numpy数组也是“传引用”，不想传引用时可采用：</p>
<pre class="language-text">import copy
def temp(A):
x = copy.copy(A)
………
</pre>
<p>注意对于list等涉及两个维度时可能需采用<code>copy.deepcopy()</code>.</p>
<p>举例：</p>
<pre class="language-text">a = [[1, 2], [3, 4], 5]
print(a)
b = a.copy()
print(b)
b[0][0] = 9999
print(b)
print(a)
c = a.copy()
print(c)
c[2] = 8888
print(c)
print(a)
</pre>
<p>输出为：</p>
<pre class="language-text">[[1, 2], [3, 4], 5]
[[1, 2], [3, 4], 5]
[[9999, 2], [3, 4], 5]
[[9999, 2], [3, 4], 5]
[[9999, 2], [3, 4], 5]
[[9999, 2], [3, 4], 8888]
[[9999, 2], [3, 4], 5]
</pre>
<hr>
<h3 class="mume-header" id="globals">globals()</h3>
<p>该函数会以字典类型返回当前位置的全部全局变量。</p>
<hr>
<h3 class="mume-header" id="zfill">zfill</h3>
<pre class="language-text">str.zfill(n)
</pre>
<p>字符串前面补0 至n位(str指代字符串)。</p>
<hr>
<h3 class="mume-header" id="ravel-flatten">ravel() & flatten()</h3>
<p><code>a.ravel()</code>和<code>a.flatten()</code>效果一样。<br>
但前者是产生视图，令<code>b=a.ravel()</code> ，b变a也变，<code>flatten</code>则不变。<br>
但</p>
<pre class="language-text">b = a.ravel()
b is a 输出：False
c = a.flatten()
c is a 输出：False
</pre>
<hr>
<h3 class="mume-header" id="%E4%B8%8D%E8%BF%9E%E7%BB%AD%E5%88%87%E7%89%87-take">不连续切片 take()</h3>
<pre data-role="codeBlock" data-info class="language-"><code>A = np.random.rand(5, 2, 2)
B = A.take([0, 3, 4], 0) # B为矩阵A在第一个维度上只保留0、3和4索引的切片，shape为(3, 2, 2)
</code></pre><hr>
<h3 class="mume-header" id="nprollaxis">np.rollaxis（）</h3>
<p>改变维度顺序。</p>
<pre class="language-text">np.rollaxis(a, n1, n2)
</pre>
<p>a是一个数组，将第n2个维度移到n1维度前。<br>
常见于图片张量中，比如C x H x W 通过np.rollaxis(a, 2, 1)变为C x W x H 。</p>
<p>测试样例：</p>
<pre class="language-text">import numpy as np
a = np.arange(24).reshape(2, 3, 4)
print(a, '\n')
print(np.rollaxis(a, 2), '\n')
print(np.rollaxis(a, 2, 1), '\n')
print(np.rollaxis(a, 1, 0), '\n')
print(np.rollaxis(a, 1, 2), '\n')
</pre>
<p>输出：</p>
<pre class="language-text">[[[ 0 1 2 3]
[ 4 5 6 7]
[ 8 9 10 11]]
[[12 13 14 15]
[16 17 18 19]
[20 21 22 23]]]
[[[ 0 4 8]
[12 16 20]]
[[ 1 5 9]
[13 17 21]]
[[ 2 6 10]
[14 18 22]]
[[ 3 7 11]
[15 19 23]]]
[[[ 0 4 8]
[ 1 5 9]
[ 2 6 10]
[ 3 7 11]]
[[12 16 20]
[13 17 21]
[14 18 22]
[15 19 23]]]
[[[ 0 1 2 3]
[12 13 14 15]]
[[ 4 5 6 7]
[16 17 18 19]]