-
Notifications
You must be signed in to change notification settings - Fork 0
Expand file tree
/
Copy pathindex.html
More file actions
209 lines (177 loc) · 10.9 KB
/
Copy pathindex.html
File metadata and controls
209 lines (177 loc) · 10.9 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
---
redirect_from: lightfielddepth/
---
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Edge-aware Bidirectional Diffusion (BMVC 2021) and View-consistent 4D Light Field Depth Estimation (BMVC 2020)</title>
<!-- Brown brand fonts (Minion 3 + Freight Sans), the lab's Adobe Fonts kit. -->
<link rel="preconnect" href="https://use.typekit.net" crossorigin>
<link rel="stylesheet" href="https://use.typekit.net/diw5pgv.css">
<!-- Shared BVC project-page styles + JS (bar, logo registry, theme). -->
<link rel="stylesheet" href="https://visual.cs.brown.edu/projects/shared/project.css">
<script defer src="https://visual.cs.brown.edu/projects/shared/project.js"></script>
<!-- Pre-paint theme: apply the stored/OS choice before first render (no
flash). Same localStorage 'theme' key + attribute as visual.cs.brown.edu. -->
<script>
(function () {
var t; try { t = localStorage.getItem('theme'); } catch (e) {}
var dark = t === 'dark' || (!t && matchMedia('(prefers-color-scheme: dark)').matches);
document.documentElement.setAttribute('data-bs-theme', dark ? 'dark' : 'light');
})();
</script>
</head>
<body>
<!-- The BVC bar (crest, section links, social icons, theme toggle) is
injected by the shared project.js. -->
<main class="project">
<!-- ============ Paper 1: Edge-aware Bidirectional Diffusion ============ -->
<h1 class="project-title">
Edge-aware Bidirectional Diffusion
<span class="subtitle">Dense Depth Estimation from Light Fields</span>
</h1>
<p class="venue">British Machine Vision Conference (BMVC) 2021</p>
<div class="authors">
<span class="author"><a href="https://cs.brown.edu/~nkhan6" target="_blank" rel="noopener">Numair Khan</a><sup>1</sup>,</span>
<span class="author"><a href="http://vclab.kaist.ac.kr/minhkim/" target="_blank" rel="noopener">Min H. Kim</a><sup>2</sup>,</span>
<span class="author"><a href="https://www.jamestompkin.com/" target="_blank" rel="noopener">James Tompkin</a><sup>1</sup></span>
</div>
<div class="affil-logos">
<span class="logo-item"><sup>1</sup><span data-logo="brown-cs"></span></span>
<span class="logo-item"><sup>2</sup><span data-logo="kaist"></span></span>
</div>
<nav class="resource-row">
<a href="https://arxiv.org/abs/2107.02967">arXiv</a>
<a href="https://github.com/brownvc/lightfielddepth">Code</a>
</nav>
<figure class="teaser">
<video src="./video/bunny_editing.mp4" autoplay muted loop playsinline></video>
<figcaption>Using reconstructed 4D depth for light field editing.</figcaption>
</figure>
<section id="abstract-section">
<h2 id="abstract">Abstract</h2>
<p>
We present an algorithm to estimate fast and accurate depth maps from light fields via a sparse set of depth edges and gradients. Our proposed approach is based around the idea that true depth edges are more sensitive than texture edges to local constraints, and so they can be reliably disambiguated through a bidirectional diffusion process. First, we use epipolar-plane images to estimate sub-pixel disparity at a sparse set of pixels. To find sparse points efficiently, we propose an entropy-based refinement approach to a line estimate from a limited set of oriented filter banks. Next, to estimate the diffusion direction away from sparse points, we optimize constraints at these points via our bidirectional diffusion method. This resolves the ambiguity of which surface the edge belongs to and reliably separates depth from texture edges, allowing us to diffuse the sparse set in a depth-edge and occlusion-aware manner to obtain accurate dense depth maps.
</p>
</section>
<section id="method-section">
<h2 id="method">Method</h2>
<figure>
<img src="./img/bididepth.png" alt="Light field editing comparison: incorrect depth edges cause insertion artifacts, while accurate depth edges allow clean occlusion handling">
<figcaption>
Light field editing requires high-accuracy depth and occlusion edges. Middle: incorrect depth or occlusion edges make inserted content appear with artifacts and with overhanging regions. Right: the accuracy of our depth edges allows easier occlusion handling when editing light fields. The inset shows our disparity map.
</figcaption>
</figure>
</section>
<section id="citation-section">
<h2 id="citation">Citation</h2>
<pre class="citation"><code>@inproceedings{Khan_2021,
series = {BMVC 2021},
title = {Edge-aware Bidirectional Diffusion for Dense Depth Estimation from Light Fields},
url = {http://dx.doi.org/10.5244/C.35.182},
DOI = {10.5244/c.35.182},
booktitle = {Proceedings of the British Machine Vision Conference 2021},
publisher = {British Machine Vision Association},
author = {Khan, Numair and Kim, Min H. and Tompkin, James},
year = {2021},
collection= {BMVC 2021}
}</code></pre>
<p class="muted">This work initially appeared as a Brown University technical report, <a href="http://cs.brown.edu/research/pubs/techreports/reports/CS-20-01.html">Fast and Accurate 4D Light Field Depth Estimation (CS-20-01)</a>.</p>
</section>
<hr>
<!-- ========= Paper 2: View-consistent 4D Light Field Depth Estimation ========= -->
<h1 class="project-title">
View-consistent 4D Light Field Depth Estimation
</h1>
<p class="venue">British Machine Vision Conference (BMVC) 2020</p>
<div class="authors">
<span class="author"><a href="https://cs.brown.edu/~nkhan6" target="_blank" rel="noopener">Numair Khan</a><sup>1</sup>,</span>
<span class="author"><a href="http://vclab.kaist.ac.kr/minhkim/" target="_blank" rel="noopener">Min H. Kim</a><sup>2</sup>,</span>
<span class="author"><a href="https://www.jamestompkin.com/" target="_blank" rel="noopener">James Tompkin</a><sup>1</sup></span>
</div>
<div class="affil-logos">
<span class="logo-item"><sup>1</sup><span data-logo="brown-cs"></span></span>
<span class="logo-item"><sup>2</sup><span data-logo="kaist"></span></span>
</div>
<nav class="resource-row">
<a href="https://www.bmvc2020-conference.com/assets/papers/0395.pdf">Paper</a>
<a href="https://arxiv.org/abs/2009.04065">arXiv</a>
<a href="https://github.com/brownvc/lightfielddepth">Code</a>
<a href="#video-2">Video</a>
</nav>
<section id="abstract-2-section">
<h2 id="abstract-2">Abstract</h2>
<p>
We propose a method to compute depth maps for every sub-aperture image in a lightfield in a view consistent way. Previous light field depth estimation methods typically estimate a depth map only for the central sub-aperture view, and struggle with view consistent estimation. Our method precisely defines depth edges via EPIs, then we diffuse these edges spatially within the central view. These depth estimates are then propagated to all other views in an occlusion-aware way. Finally, disoccluded regions are completed by diffusion in EPI space. Our method runs efficiently with respect to both other classical and deep learning-based approaches, and achieves competitive quantitative metrics and qualitative performance on both synthetic and real-world light fields.
</p>
</section>
<section id="results-2-section">
<h2 id="results-2">Results</h2>
<p>Depth consistency comparisons across novel views:</p>
<div class="snap-row">
<img src="./img/cotton_novel.gif" alt="View-consistent depth on the Cotton scene">
<img src="./img/boxes_novel.gif" alt="View-consistent depth on the Boxes scene">
<img src="./img/dino_novel.gif" alt="View-consistent depth on the Dino scene">
<img src="./img/sideboard_novel.gif" alt="View-consistent depth on the Sideboard scene">
</div>
<p>Our method works with both real and synthetic datasets, and a range of baselines:</p>
<div class="snap-row">
<img src="./img/result_stillLife.gif" alt="Depth result: Still Life">
<img src="./img/result_rustyFence.gif" alt="Depth result: Rusty Fence">
<img src="./img/result_bulldozer.gif" alt="Depth result: Bulldozer">
<img src="./img/result_bikes.gif" alt="Depth result: Bikes">
<img src="./img/result_buddha.gif" alt="Depth result: Buddha">
<img src="./img/result_eucalyptus.gif" alt="Depth result: Eucalyptus">
<img src="./img/result_jellybeans.gif" alt="Depth result: Jellybeans">
<img src="./img/result_horses.gif" alt="Depth result: Horses">
<img src="./img/result_jewels.gif" alt="Depth result: Jewels">
<img src="./img/result_papillon.gif" alt="Depth result: Papillon">
<img src="./img/result_silos.gif" alt="Depth result: Silos">
<img src="./img/result_truck.gif" alt="Depth result: Truck">
</div>
</section>
<section id="video-2-section">
<h2 id="video-2">Video</h2>
<video class="center" style="display:block; width:100%;" controls preload="metadata">
<source src="./video/bmvc2020_supplementary_vid.mp4" type="video/mp4">
</video>
<p class="centerContent"><a href="./video/bmvc2020_supplementary_vid.mp4" download>Download supplemental video (MP4)</a></p>
<!-- VERIFY: presentation.mp4 is placed here under the BMVC 2020 oral paper
(matches the legacy page layout and this paper's oral status), but
pubs.json links video_presentation to the 2021 Edge-aware paper. -->
<h3>Presentation</h3>
<video class="center" style="display:block; width:100%;" controls preload="metadata">
<source src="./video/presentation.mp4" type="video/mp4">
</video>
<p class="centerContent"><a href="./video/presentation.mp4" download>Download presentation video (MP4)</a></p>
</section>
<section id="citation-2-section">
<h2 id="citation-2">Citation</h2>
<pre class="citation"><code>@inproceedings{Khan_2020,
series = {BMVC 2020},
title = {View-consistent 4D Light Field Depth Estimation},
url = {http://dx.doi.org/10.5244/C.34.103},
DOI = {10.5244/c.34.103},
booktitle = {Proceedings of the British Machine Vision Conference 2020},
publisher = {British Machine Vision Association},
author = {Khan, Numair and Kim, Min H. and Tompkin, James},
year = {2020},
collection= {BMVC 2020}
}</code></pre>
</section>
<!-- ==================== Shared closing sections ==================== -->
<section id="acknowledgements-section">
<h2 id="acknowledgements">Acknowledgements</h2>
<p>
Numair Khan thanks an Andy van Dam PhD Fellowship, James Tompkin thanks a gift from Cognex, and Min H. Kim thanks a Korea NRF grant (2019R1A2C3007229).
</p>
</section>
<footer class="logos">
<span data-logo="brown-cs"></span>
<span data-logo="kaist"></span>
</footer>
</main>
</body>
</html>