Skip to content

Commit fc072cc

Browse files
Song-JingyuSong-Jingyu
committed
copy from crkd template
1 parent 1a15969 commit fc072cc

File tree

279 files changed

+3470
-0
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

279 files changed

+3470
-0
lines changed

‎docs/index.html‎

Lines changed: 349 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,349 @@
1+
<!DOCTYPE html>
2+
<html>
3+
<head>
4+
<meta charset="utf-8">
5+
<meta name="description"
6+
content="Enhanced Camera-Radar Object Detection with Cross-modality Knowledge Distillation.">
7+
<meta name="keywords" content="CRKD">
8+
<meta name="viewport" content="width=device-width, initial-scale=1">
9+
<title>CRKD: Enhanced Camera-Radar Object Detection with Cross-modality Knowledge Distillation</title>
10+
11+
<!-- Global site tag (gtag.js) - Google Analytics -->
12+
<script async src="https://www.googletagmanager.com/gtag/js?id=G-PYVRSFMDRL"></script>
13+
<script>
14+
window.dataLayer = window.dataLayer || [];
15+
16+
function gtag() {
17+
dataLayer.push(arguments);
18+
}
19+
20+
gtag('js', new Date());
21+
22+
gtag('config', 'G-PYVRSFMDRL');
23+
</script>
24+
25+
<link href="https://fonts.googleapis.com/css?family=Google+Sans|Noto+Sans|Castoro"
26+
rel="stylesheet">
27+
28+
<link rel="stylesheet" href="./static/css/bulma.min.css">
29+
<link rel="stylesheet" href="./static/css/bulma-carousel.min.css">
30+
<link rel="stylesheet" href="./static/css/bulma-slider.min.css">
31+
<link rel="stylesheet" href="./static/css/fontawesome.all.min.css">
32+
<link rel="stylesheet"
33+
href="https://cdn.jsdelivr.net/gh/jpswalsh/academicons@1/css/academicons.min.css">
34+
<link rel="stylesheet" href="./static/css/index.css">
35+
<link rel="icon" href="./static/images/favicon.svg">
36+
37+
<script src="https://ajax.googleapis.com/ajax/libs/jquery/3.5.1/jquery.min.js"></script>
38+
<script defer src="./static/js/fontawesome.all.min.js"></script>
39+
<script src="./static/js/bulma-carousel.min.js"></script>
40+
<script src="./static/js/bulma-slider.min.js"></script>
41+
<script src="./static/js/index.js"></script>
42+
</head>
43+
<body>
44+
45+
<nav class="navbar" role="navigation" aria-label="main navigation">
46+
<div class="navbar-brand">
47+
<a role="button" class="navbar-burger" aria-label="menu" aria-expanded="false">
48+
<span aria-hidden="true"></span>
49+
<span aria-hidden="true"></span>
50+
<span aria-hidden="true"></span>
51+
</a>
52+
</div>
53+
<div class="navbar-menu">
54+
<div class="navbar-start" style="flex-grow: 1; justify-content: center;">
55+
<!-- <a class="navbar-item" href="https://keunhong.com">
56+
<span class="icon">
57+
<i class="fas fa-home"></i>
58+
</span>
59+
</a> -->
60+
61+
<div class="navbar-item has-dropdown is-hoverable">
62+
<a class="navbar-link">
63+
More Research
64+
</a>
65+
<div class="navbar-dropdown">
66+
<a class="navbar-item" href="https://github.com/Song-Jingyu/LiRaFusion">
67+
LiRaFusion
68+
</a>
69+
<!-- <a class="navbar-item" href="https://nerfies.github.io">
70+
Nerfies
71+
</a>
72+
<a class="navbar-item" href="https://latentfusion.github.io">
73+
LatentFusion
74+
</a>
75+
<a class="navbar-item" href="https://photoshape.github.io">
76+
PhotoShape
77+
</a> -->
78+
</div>
79+
</div>
80+
</div>
81+
82+
</div>
83+
</nav>
84+
85+
86+
<section class="hero">
87+
<div class="hero-body">
88+
<div class="container is-max-desktop">
89+
<div class="columns is-centered">
90+
<div class="column has-text-centered">
91+
<h1 class="title is-1 publication-title">MemFusionMap: Working Memory Fusion for Online Vectorized HD Map Construction</h1>
92+
<div class="is-size-5 publication-authors">
93+
<span class="author-block">
94+
<a href="https://junshao0104.github.io/">Lingjun Zhao</a><sup>1*</sup>,</span>
95+
<span class="author-block">
96+
<a href="https://www.linkedin.com/in/jingyu-song-93763a132/">Jingyu Song</a><sup>1*✉</sup>,</span>
97+
<span class="author-block">
98+
<a href="https://fieldrobotics.engin.umich.edu/">Katherine A. Skinner</a><sup>1</sup>
99+
</span>
100+
<!-- <span class="author-block">
101+
<a href="http://sofienbouaziz.com">Sofien Bouaziz</a><sup>2</sup>,
102+
</span>
103+
<span class="author-block">
104+
<a href="https://www.danbgoldman.com">Dan B Goldman</a><sup>2</sup>,
105+
</span>
106+
<span class="author-block">
107+
<a href="https://homes.cs.washington.edu/~seitz/">Steven M. Seitz</a><sup>1,2</sup>,
108+
</span>
109+
<span class="author-block">
110+
<a href="http://www.ricardomartinbrualla.com">Ricardo Martin-Brualla</a><sup>2</sup>
111+
</span> -->
112+
</div>
113+
114+
<div class="is-size-5 publication-authors">
115+
<span class="author-block"><sup>*</sup>Equal Contribution</span>
116+
<!-- <span class="author-block"><sup>†</sup>Corresponding Author</span> -->
117+
<span class="author-block"><sup>1</sup>University of Michigan</span>
118+
</div>
119+
120+
<div class="column has-text-centered">
121+
<div class="publication-links">
122+
<!-- PDF Link. -->
123+
<span class="link-block">
124+
<a href="https://arxiv.org/pdf/2403.19104.pdf"
125+
class="external-link button is-normal is-rounded is-dark">
126+
<span class="icon">
127+
<i class="fas fa-file-pdf"></i>
128+
</span>
129+
<span>Paper</span>
130+
</a>
131+
</span>
132+
<span class="link-block">
133+
<a href="https://arxiv.org/abs/2403.19104"
134+
class="external-link button is-normal is-rounded is-dark">
135+
<span class="icon">
136+
<i class="ai ai-arxiv"></i>
137+
</span>
138+
<span>arXiv</span>
139+
</a>
140+
</span>
141+
<!-- Video Link. -->
142+
<span class="link-block">
143+
<a href="https://youtu.be/A63ho5ESz8w?si=i_xoSdJHlDclfyUF"
144+
class="external-link button is-normal is-rounded is-dark">
145+
<span class="icon">
146+
<i class="fab fa-youtube"></i>
147+
</span>
148+
<span>Video</span>
149+
</a>
150+
</span>
151+
<!-- Code Link. -->
152+
<span class="link-block">
153+
<a href="https://github.com/Song-Jingyu/CRKD"
154+
class="external-link button is-normal is-rounded is-dark">
155+
<span class="icon">
156+
<i class="fab fa-github"></i>
157+
</span>
158+
<span>Code</span>
159+
</a>
160+
</span>
161+
<!-- Dataset Link. -->
162+
<!-- <span class="link-block">
163+
<a href="https://github.com/google/nerfies/releases/tag/0.1"
164+
class="external-link button is-normal is-rounded is-dark">
165+
<span class="icon">
166+
<i class="far fa-images"></i>
167+
</span>
168+
<span>Data</span>
169+
</a>
170+
</div> -->
171+
172+
</div>
173+
</div>
174+
</div>
175+
</div>
176+
</div>
177+
</section>
178+
179+
<section class="hero teaser">
180+
<div class="container is-max-desktop">
181+
<div class="hero-body">
182+
<!-- <video id="teaser" autoplay muted loop playsinline height="100%">
183+
<source src="./static/videos/teaser.mp4"
184+
type="video/mp4">
185+
</video> -->
186+
<!-- add image docs/static/images/pitch.svg-->
187+
<img src="./static/images/pitch.svg" alt="CRKD" style="width: 100%; height: auto;">
188+
<h2 class="subtitle has-text-centered">
189+
We propose CRKD to conduct a novel cross-modality knowledge distillation path from LiDAR-camera teacher to camera-radar student.
190+
</h2>
191+
</div>
192+
</div>
193+
</section>
194+
195+
196+
<!-- <section class="hero is-light is-small">
197+
<div class="hero-body">
198+
<div class="container">
199+
<div id="results-carousel" class="carousel results-carousel">
200+
<div class="item item-steve">
201+
<video poster="" id="steve" autoplay controls muted loop playsinline height="100%">
202+
<source src="./static/videos/steve.mp4"
203+
type="video/mp4">
204+
</video>
205+
</div>
206+
<div class="item item-chair-tp">
207+
<video poster="" id="chair-tp" autoplay controls muted loop playsinline height="100%">
208+
<source src="./static/videos/chair-tp.mp4"
209+
type="video/mp4">
210+
</video>
211+
</div>
212+
<div class="item item-shiba">
213+
<video poster="" id="shiba" autoplay controls muted loop playsinline height="100%">
214+
<source src="./static/videos/shiba.mp4"
215+
type="video/mp4">
216+
</video>
217+
</div>
218+
<div class="item item-fullbody">
219+
<video poster="" id="fullbody" autoplay controls muted loop playsinline height="100%">
220+
<source src="./static/videos/fullbody.mp4"
221+
type="video/mp4">
222+
</video>
223+
</div>
224+
<div class="item item-blueshirt">
225+
<video poster="" id="blueshirt" autoplay controls muted loop playsinline height="100%">
226+
<source src="./static/videos/blueshirt.mp4"
227+
type="video/mp4">
228+
</video>
229+
</div>
230+
<div class="item item-mask">
231+
<video poster="" id="mask" autoplay controls muted loop playsinline height="100%">
232+
<source src="./static/videos/mask.mp4"
233+
type="video/mp4">
234+
</video>
235+
</div>
236+
<div class="item item-coffee">
237+
<video poster="" id="coffee" autoplay controls muted loop playsinline height="100%">
238+
<source src="./static/videos/coffee.mp4"
239+
type="video/mp4">
240+
</video>
241+
</div>
242+
<div class="item item-toby">
243+
<video poster="" id="toby" autoplay controls muted loop playsinline height="100%">
244+
<source src="./static/videos/toby2.mp4"
245+
type="video/mp4">
246+
</video>
247+
</div>
248+
</div>
249+
</div>
250+
</div>
251+
</section> -->
252+
253+
254+
<section class="section">
255+
<div class="container is-max-desktop">
256+
<!-- Abstract. -->
257+
<div class="columns is-centered has-text-centered">
258+
<div class="column is-four-fifths">
259+
<h2 class="title is-3">Abstract</h2>
260+
<div class="content has-text-justified">
261+
In the field of 3D object detection for autonomous driving, LiDAR-Camera (LC) fusion is the top-performing sensor configuration.
262+
Still, LiDAR is relatively high cost, which hinders adoption of this technology for consumer automobiles.
263+
Alternatively, camera and radar are commonly deployed on vehicles already on the road today, but performance of Camera-Radar (CR) fusion falls behind LC fusion.
264+
In this work, we propose CRKD to bridge the performance gap between LC and CR detectors with a novel cross-modality knowledge distillation (KD) framework.
265+
We use the Bird's-Eye-View (BEV) representation as the shared feature space to enable effective knowledge distillation.
266+
To accommodate the unique cross-modality KD path, we propose four distillation losses to help the student learn crucial features from the teacher model.
267+
We present extensive evaluations on the nuScenes dataset to demonstrate the effectiveness of the proposed CRKD framework.
268+
</div>
269+
</div>
270+
</div>
271+
<!--/ Abstract. -->
272+
273+
<!-- Paper video. -->
274+
<!-- <div class="columns is-centered has-text-centered">
275+
<div class="column is-four-fifths">
276+
<h2 class="title is-3">Video</h2>
277+
<div class="publication-video">
278+
<iframe src="https://www.youtube.com/embed/MrKrnHhk8IA?rel=0&amp;showinfo=0"
279+
frameborder="0" allow="autoplay; encrypted-media" allowfullscreen></iframe>
280+
</div>
281+
</div>
282+
</div> -->
283+
<!--/ Paper video. -->
284+
</div>
285+
</section>
286+
287+
<!-- Overview section -->
288+
<section class="section">
289+
<div class="container is-max-desktop">
290+
<div class="columns is-centered">
291+
<div class="column is-full-width">
292+
<h2 class="title is-3">Overview</h2>
293+
<div class="content has-text-justified">
294+
<p>
295+
We propose a novel cross-modality KD framework to enable LC-to-CR distillation in the BEV feature space. With the transferred knowledge
296+
from an LC teacher detector, the CR student detector can outperform existing baselines without additional cost during inference.
297+
</p>
298+
<p>
299+
We design four KD modules to address the notable discrepancies between different sensors to realize realize effective cross-modality KD.
300+
As we operate KD in the BEV space, the proposed loss designs can be applied to other KD configurations.
301+
Our improvement also includes adding a gated network to the baseline model for adaptive fusion.
302+
</p>
303+
<p>
304+
We conduct extensive evaluation on nuScenes to demonstrate the effectiveness of CRKD.
305+
CRKD can improve the mAP and NDS of student detectors by 3.5% and 3.2% respectively.
306+
Since our method focuses on a novel KD path with distinctively large modality gap, we provide thorough study and analysis to support our design choices.
307+
</div>
308+
</div>
309+
</div>
310+
<!-- add an image docs/static/images/Overall_Diagram.svg -->
311+
<img src="./static/images/Overall_Diagram.jpg" alt="CRKD Overview" style="width: 100%; height: auto;">
312+
</div>
313+
</section>
314+
315+
<!-- Results section -->
316+
317+
318+
319+
<section class="section" id="BibTeX">
320+
<div class="container is-max-desktop content">
321+
<h2 class="title">BibTeX</h2>
322+
<pre><code>@inproceedings{zhao2024crkd,
323+
author = {Zhao, Lingjun and Song, Jingyu and Skinner, Katherine A},
324+
title = {CRKD: Enhanced Camera-Radar Object Detection with Cross-modality Knowledge Distillation},
325+
journal = {2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
326+
year = {2024},
327+
}</code></pre>
328+
</div>
329+
</section>
330+
331+
<!-- cite nerfies project page https://github.com/nerfies/nerfies.github.io -->
332+
<footer class="footer">
333+
<div class="container">
334+
<div class="content has-text-centered">
335+
</div>
336+
<div class="columns is-centered">
337+
<div class="column is-8">
338+
<div class="content">
339+
<p>
340+
This page is modified based on <a href="https://github.com/nerfies/nerfies.github.io">Nerfies</a>.
341+
</p>
342+
</div>
343+
</div>
344+
</div>
345+
</div>
346+
</footer>
347+
348+
</body>
349+
</html>

‎docs/static/css/bulma-carousel.min.css‎

Lines changed: 1 addition & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

0 commit comments

Comments
 (0)