Skip to content

Commit 43add18

Browse files
authored
Update README.md
1 parent f386c91 commit 43add18

File tree

1 file changed

+5
-45
lines changed

1 file changed

+5
-45
lines changed

README.md

Lines changed: 5 additions & 45 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# PyTorch implementation of Deformable Convolution
22
* By Wei OUYANG @ Institut Pasteur
3-
* The original implementation in Keras/TensorFlow: https://github.com/felixlaumon/deform-conv
3+
* Thanks to Felix Lau's Keras/TensorFlow implementation: https://github.com/felixlaumon/deform-conv
44

55
### TODO List
66
- [x] implement offsets mapping in pytorch
@@ -13,53 +13,13 @@
1313
- [ ] support input image with different width and height
1414
- [ ] benchmark with tensorflow implementation
1515

16-
## Understanding Deformable Convolution
16+
## Deformable Convolutional Networks
1717
> Dai, Jifeng, Haozhi Qi, Yuwen Xiong, Yi Li, Guodong Zhang, Han Hu, and Yichen
1818
Wei. 2017. “Deformable Convolutional Networks.” arXiv [cs.CV]. arXiv.
1919
http://arxiv.org/abs/1703.06211
2020

21-
![](deformable-learned-offset-filtered.gif)
22-
23-
Check out
24-
https://medium.com/@phelixlau/notes-on-deformable-convolutional-networks-baaabbc11cf3
25-
for my summary of the paper.
26-
27-
## Experiment on MNIST and Scaled Data Augmentation
28-
29-
To demonstrate the effectiveness of deformable convolution with scaled images,
30-
we show that by simply replacing regular convolution with deformable convolution
31-
and fine-tuning just the offsets with a scale-augmented datasets, deformable CNN
32-
performs significantly better than regular CNN on the scaled MNIST dataset. This
33-
indicates that deformable convolution is able to more effectively utilize
34-
already learned feature map to represent geometric distortion.
35-
36-
First, we train a 4-layer CNN with regular convolution on MNIST without any data
37-
augmentation. Then we replace all regular convolution layers with deformable
38-
convolution layers and freeze the weights of all layers except the newly added
39-
convolution layers responsible for learning the offsets. This model is then
40-
fine-tuned on the scale-augmented MNIST dataset.
21+
The following animation is generated by Felix Lau (with his tensorflow implementation):
4122

42-
In this set up, the deformable CNN is forced to make better use of the learned
43-
feature map by only changing its receptive field.
44-
45-
Note that the deformable CNN did not receive additional supervision other than
46-
the labels and is trained with cross-entropy just like the regular CNN.
47-
48-
| Test Accuracy | Regular CNN | Deformable CNN |
49-
| --- | --- | --- |
50-
| Regular MNIST | 98.74% | 97.27% |
51-
| Scaled MNIST | 57.01% | 92.55% |
52-
53-
Please refer to `scripts/scaled_mnist.py` for reproducing this result.
54-
55-
## Notes on Implementation
23+
![](deformable-learned-offset-filtered.gif)
5624

57-
- This implementation is not efficient. In fact a forward pass with deformed
58-
convolution takes 260 ms, while regular convolution takes only 10 ms. Also,
59-
GPU average utilization is only around 10%.
60-
- This implementation also does not take advantage of the fact that offsets and
61-
the input have similar shape (in `tf_batch_map_offsets`). (So STN-style
62-
bilinear sampling will help)
63-
- The TensorFlow Keras backend must be used (channel-last)
64-
- You can check ensure the TensorFlow implementation is equivalent to its scipy
65-
counterpart by running unit tests (e.g. `py.test -x -v --pdb`)
25+
Also Check out Felix Lau's summary of the paper: https://medium.com/@phelixlau/notes-on-deformable-convolutional-networks-baaabbc11cf3

0 commit comments

Comments
 (0)