蹄牛操作系统TINIUX
ROS中文社区
查看: 4761|回复: 21

[电子书]Computer Vision:Algorithms and Applications

  [复制链接]
发表于 2016-2-19 15:46:50 | 显示全部楼层 |阅读模式
Turtlebot2
本帖最后由 newera 于 2016-2-19 22:23 编辑

慕尼黑技术大学 课程

SzeliskiBook_20100903_draft.pdf

22.14 MB, 下载次数: 193

慕尼黑技术大学 课程

回复

使用道具 举报

蹄牛操作系统TINIUX技术论坛
发表于 2016-2-19 22:20:53 | 显示全部楼层
MakeBlock
感谢楼主为论坛朋友们提供的内部资料, Computer Vision:Algorithms and Applications, Richard Szeliski


The seeds for this book were first planted in 2001 when Steve Seitz at the University of Washington
invited me to co-teach a course called “Computer Vision for Computer Graphics”. At
that time, computer vision techniques were increasingly being used in computer graphics to
create image-based models of real-world objects, to create visual effects, and to merge realworld
imagery using computational photography techniques. Our decision to focus on the
applications of computer vision to fun problems such as image stitching and photo-based 3D
modeling from personal photos seemed to resonate well with our students.
Since that time, a similar syllabus and project-oriented course structure has been used to
teach general computer vision courses both at the University of Washington and at Stanford.
(The latter was a course I co-taught with David Fleet in 2003.) Similar curricula have been
adopted at a number of other universities and also incorporated into more specialized courses
on computational photography. (For ideas on how to use this book in your own course, please
see Table 1.1 in Section 1.4.)
This book also reflects my 20 years’ experience doing computer vision research in corporate
research labs, mostly at Digital Equipment Corporation’s Cambridge Research Lab and
at Microsoft Research. In pursuing my work, I have mostly focused on problems and solution
techniques (algorithms) that have practical real-world applications and that work well in
practice. Thus, this book has more emphasis on basic techniques that work under real-world
conditions and less on more esoteric mathematics that has intrinsic elegance but less practical
applicability.
This book is suitable for teaching a senior-level undergraduate course in computer vision
to students in both computer science and electrical engineering. I prefer students to have
either an image processing or a computer graphics course as a prerequisite so that they can
spend less time learning general background mathematics and more time studying computer
vision techniques. The book is also suitable for teaching graduate-level courses in computer
vision (by delving into the more demanding application and algorithmic areas) and as a general
reference to fundamental techniques and the recent research literature. To this end, I have
attempted wherever possible to at least cite the newest research in each sub-field, even if the
technical details are too complex to cover in the book itself.
In teaching our courses, we have found it useful for the students to attempt a number of
small implementation projects, which often build on one another, in order to get them used to
working with real-world images and the challenges that these present. The students are then
asked to choose an individual topic for each of their small-group, final projects. (Sometimes
these projects even turn into conference papers!) The exercises at the end of each chapter
contain numerous suggestions for smaller mid-term projects, as well as more open-ended
problems whose solutions are still active research topics. Wherever possible, I encourage
students to try their algorithms on their own personal photographs, since this better motivates
them, often leads to creative variants on the problems, and better acquaints them with the
variety and complexity of real-world imagery.
In formulating and solving computer vision problems, I have often found it useful to draw
inspiration from three high-level approaches:
• Scientific: build detailed models of the image formation process and develop mathematical
techniques to invert these in order to recover the quantities of interest (where
necessary, making simplifying assumption to make the mathematics more tractable).
• Statistical: use probabilistic models to quantify the prior likelihood of your unknowns
and the noisy measurement processes that produce the input images, then infer the best
possible estimates of your desired quantities and analyze their resulting uncertainties.
The inference algorithms used are often closely related to the optimization techniques
used to invert the (scientific) image formation processes.
• Engineering: develop techniques that are simple to describe and implement but that
are also known to work well in practice. Test these techniques to understand their
limitation and failure modes, as well as their expected computational costs (run-time
performance).
These three approaches build on each other and are used throughout the book.
My personal research and development philosophy (and hence the exercises in the book)
have a strong emphasis on testing algorithms. It’s too easy in computer vision to develop an
algorithm that does something plausible on a few images rather than something correct. The
best way to validate your algorithms is to use a three-part strategy.
First, test your algorithm on clean synthetic data, for which the exact results are known.
Second, add noise to the data and evaluate how the performance degrades as a function of
noise level. Finally, test the algorithm on real-world data, preferably drawn from a wide
variety of sources, such as photos found on the Web. Only then can you truly know if your
algorithm can deal with real-world complexity, i.e., images that do not fit some simplified
model or assumptions.
In order to help students in this process, this books comes with a large amount of supplementary
material, which can be found on the book’s Web site http://szeliski.org/Book. This
material, which is described in Appendix C, includes:
• pointers to commonly used data sets for the problems, which can be found on the Web
• pointers to software libraries, which can help students get started with basic tasks such
as reading/writing images or creating and manipulating images
• slide sets corresponding to the material covered in this book
• a BibTeX bibliography of the papers cited in this book.
The latter two resources may be of more interest to instructors and researchers publishing
new papers in this field, but they will probably come in handy even with regular students.
Some of the software libraries contain implementations of a wide variety of computer vision
algorithms, which can enable you to tackle more ambitious projects (with your instructor’s
consent).
Acknowledgements
I would like to gratefully acknowledge all of the people whose passion for research and
inquiry as well as encouragement have helped me write this book.
Steve Zucker at McGill University first introduced me to computer vision, taught all of
his students to question and debate research results and techniques, and encouraged me to
pursue a graduate career in this area.
Takeo Kanade and Geoff Hinton, my Ph. D. thesis advisors at Carnegie Mellon University,
taught me the fundamentals of good research, writing, and presentation. They fired up my
interest in visual processing, 3D modeling, and statistical methods, while Larry Matthies
introduced me to Kalman filtering and stereo matching.
Demetri Terzopoulos was my mentor at my first industrial research job and taught me the
ropes of successful publishing. Yvan Leclerc and Pascal Fua, colleagues from my brief interlude
at SRI International, gave me new perspectives on alternative approaches to computer
vision.
During my six years of research at Digital Equipment Corporation’s Cambridge Research
Lab, I was fortunate to work with a great set of colleagues, including Ingrid Carlbom, Gudrun
Klinker, Keith Waters, Richard Weiss, Stephane Lavall ´ ee, and Sing Bing Kang, as well as to ´
supervise the first of a long string of outstanding summer interns, including David Tonnesen,
Sing Bing Kang, James Coughlan, and Harry Shum. This is also where I began my long-term
collaboration with Daniel Scharstein, now at Middlebury College.
At Microsoft Research, I’ve had the outstanding fortune to work with some of the world’s
best researchers in computer vision and computer graphics, including Michael Cohen, Hugues
Hoppe, Stephen Gortler, Steve Shafer, Matthew Turk, Harry Shum, Anandan, Phil Torr, Antonio
Criminisi, Georg Petschnigg, Kentaro Toyama, Ramin Zabih, Shai Avidan, Sing Bing
Kang, Matt Uyttendaele, Patrice Simard, Larry Zitnick, Richard Hartley, Simon Winder,
Drew Steedly, Chris Pal, Nebojsa Jojic, Patrick Baudisch, Dani Lischinski, Matthew Brown,
Simon Baker, Michael Goesele, Eric Stollnitz, David Nister, Blaise Aguera y Arcas, Sudipta ´
Sinha, Johannes Kopf, Neel Joshi, and Krishnan Ramnath. I was also lucky to have as interns
such great students as Polina Golland, Simon Baker, Mei Han, Arno Schodl, Ron Dror, ¨
Ashley Eden, Jinxiang Chai, Rahul Swaminathan, Yanghai Tsin, Sam Hasinoff, Anat Levin,
Matthew Brown, Eric Bennett, Vaibhav Vaish, Jan-Michael Frahm, James Diebel, Ce Liu,
Josef Sivic, Grant Schindler, Colin Zheng, Neel Joshi, Sudipta Sinha, Zeev Farbman, Rahul
Garg, Tim Cho, Yekeun Jeong, Richard Roberts, Varsha Hedau, and Dilip Krishnan.
While working at Microsoft, I’ve also had the opportunity to collaborate with wonderful
colleagues at the University of Washington, where I hold an Affiliate Professor appointment.
I’m indebted to Tony DeRose and David Salesin, who first encouraged me to get involved
with the research going on at UW, my long-time collaborators Brian Curless, Steve Seitz,
Maneesh Agrawala, Sameer Agarwal, and Yasu Furukawa, as well as the students I have
had the privilege to supervise and interact with, including Frederic Pighin, Yung-Yu Chuang, ´
Doug Zongker, Colin Zheng, Aseem Agarwala, Dan Goldman, Noah Snavely, Rahul Garg,
and Ryan Kaminsky. As I mentioned at the beginning of this preface, this book owes its
inception to the vision course that Steve Seitz invited me to co-teach, as well as to Steve’s
encouragement, course notes, and editorial input.
I’m also grateful to the many other computer vision researchers who have given me so
many constructive suggestions about the book, including Sing Bing Kang, who was my informal
book editor, Vladimir Kolmogorov, who contributed Appendix B.5.5 on linear programming
techniques for MRF inference, Daniel Scharstein, Richard Hartley, Simon Baker, Noah
Snavely, Bill Freeman, Svetlana Lazebnik, Matthew Turk, Jitendra Malik, Alyosha Efros,
Michael Black, Brian Curless, Sameer Agarwal, Li Zhang, Deva Ramanan, Olga Veksler,
Yuri Boykov, Carsten Rother, Phil Torr, Bill Triggs, Bruce Maxwell, Jana Koseck ˇ a, Eero Si- ´
moncelli, Aaron Hertzmann, Antonio Torralba, Tomaso Poggio, Theo Pavlidis, Baba Vemuri,
Nando de Freitas, Chuck Dyer, Song Yi, Falk Schubert, Roman Pflugfelder, Marshall Tappen,
James Coughlan, Sammy Rogmans, Klaus Strobel, Shanmuganathan, Andreas Siebert,
Yongjun Wu, Fred Pighin, Juan Cockburn, Ronald Mallet, Tim Soper, Georgios Evangelidis,
Dwight Fowler, Itzik Bayaz, Daniel O’Connor, and Srikrishna Bhat. Shena Deuchers did a
fantastic job copy-editing the book and suggesting many useful improvements and Wayne
Wheeler and Simon Rees at Springer were most helpful throughout the whole book publishing
process. Keith Price’s Annotated Computer Vision Bibliography was invaluable in
tracking down references and finding related work.
If you have any suggestions for improving the book, please send me an e-mail, as I would
like to keep the book as accurate, informative, and timely as possible.
Lastly, this book would not have been possible or worthwhile without the incredible support
and encouragement of my family. I dedicate this book to my parents, Zdzisław and
Jadwiga, whose love, generosity, and accomplishments have always inspired me; to my sister
Basia for her lifelong friendship; and especially to Lyn, Anne, and Stephen, whose daily
encouragement in all matters (including this book project) makes it all worthwhile.

Lake Wenatchee
August, 2010

回复 支持 反对

使用道具 举报

发表于 2016-2-19 22:22:06 | 显示全部楼层
SeeedStudio
Contents
Preface vii
Contents xiii
1 Introduction 1
1.1 What is computer vision? . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.2 A brief history . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
1.3 Book overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
1.4 Sample syllabus . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
1.5 A note on notation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
1.6 Additional reading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
2 Image formation 29
2.1 Geometric primitives and transformations . . . . . . . . . . . . . . . . . . . 31
2.1.1 Geometric primitives . . . . . . . . . . . . . . . . . . . . . . . . . . 32
2.1.2 2D transformations . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
2.1.3 3D transformations . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
2.1.4 3D rotations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
2.1.5 3D to 2D projections . . . . . . . . . . . . . . . . . . . . . . . . . . 46
2.1.6 Lens distortions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
2.2 Photometric image formation . . . . . . . . . . . . . . . . . . . . . . . . . . 60
2.2.1 Lighting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
2.2.2 Reflectance and shading . . . . . . . . . . . . . . . . . . . . . . . . 62
2.2.3 Optics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
2.3 The digital camera . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
2.3.1 Sampling and aliasing . . . . . . . . . . . . . . . . . . . . . . . . . 77
2.3.2 Color . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80
2.3.3 Compression . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90

2.4 Additional reading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93
2.5 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93
3 Image processing 99
3.1 Point operators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101
3.1.1 Pixel transforms . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103
3.1.2 Color transforms . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104
3.1.3 Compositing and matting . . . . . . . . . . . . . . . . . . . . . . . . 105
3.1.4 Histogram equalization . . . . . . . . . . . . . . . . . . . . . . . . . 107
3.1.5 Application: Tonal adjustment . . . . . . . . . . . . . . . . . . . . . 111
3.2 Linear filtering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111
3.2.1 Separable filtering . . . . . . . . . . . . . . . . . . . . . . . . . . . 115
3.2.2 Examples of linear filtering . . . . . . . . . . . . . . . . . . . . . . . 117
3.2.3 Band-pass and steerable filters . . . . . . . . . . . . . . . . . . . . . 118
3.3 More neighborhood operators . . . . . . . . . . . . . . . . . . . . . . . . . . 122
3.3.1 Non-linear filtering . . . . . . . . . . . . . . . . . . . . . . . . . . . 122
3.3.2 Morphology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127
3.3.3 Distance transforms . . . . . . . . . . . . . . . . . . . . . . . . . . 129
3.3.4 Connected components . . . . . . . . . . . . . . . . . . . . . . . . . 131
3.4 Fourier transforms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132
3.4.1 Fourier transform pairs . . . . . . . . . . . . . . . . . . . . . . . . . 136
3.4.2 Two-dimensional Fourier transforms . . . . . . . . . . . . . . . . . . 140
3.4.3 Wiener filtering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140
3.4.4 Application: Sharpening, blur, and noise removal . . . . . . . . . . . 144
3.5 Pyramids and wavelets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144
3.5.1 Interpolation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145
3.5.2 Decimation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 148
3.5.3 Multi-resolution representations . . . . . . . . . . . . . . . . . . . . 150
3.5.4 Wavelets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154
3.5.5 Application: Image blending . . . . . . . . . . . . . . . . . . . . . . 160
3.6 Geometric transformations . . . . . . . . . . . . . . . . . . . . . . . . . . . 162
3.6.1 Parametric transformations . . . . . . . . . . . . . . . . . . . . . . . 163
3.6.2 Mesh-based warping . . . . . . . . . . . . . . . . . . . . . . . . . . 170
3.6.3 Application: Feature-based morphing . . . . . . . . . . . . . . . . . 173
3.7 Global optimization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 174
3.7.1 Regularization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 174
3.7.2 Markov random fields . . . . . . . . . . . . . . . . . . . . . . . . . 180
3.7.3 Application: Image restoration . . . . . . . . . . . . . . . . . . . . . 192

3.8 Additional reading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192
3.9 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194
4 Feature detection and matching 205
4.1 Points and patches . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 207
4.1.1 Feature detectors . . . . . . . . . . . . . . . . . . . . . . . . . . . . 209
4.1.2 Feature descriptors . . . . . . . . . . . . . . . . . . . . . . . . . . . 222
4.1.3 Feature matching . . . . . . . . . . . . . . . . . . . . . . . . . . . . 225
4.1.4 Feature tracking . . . . . . . . . . . . . . . . . . . . . . . . . . . . 235
4.1.5 Application: Performance-driven animation . . . . . . . . . . . . . . 237
4.2 Edges . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 238
4.2.1 Edge detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 238
4.2.2 Edge linking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 244
4.2.3 Application: Edge editing and enhancement . . . . . . . . . . . . . . 249
4.3 Lines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 250
4.3.1 Successive approximation . . . . . . . . . . . . . . . . . . . . . . . 250
4.3.2 Hough transforms . . . . . . . . . . . . . . . . . . . . . . . . . . . . 251
4.3.3 Vanishing points . . . . . . . . . . . . . . . . . . . . . . . . . . . . 254
4.3.4 Application: Rectangle detection . . . . . . . . . . . . . . . . . . . . 257
4.4 Additional reading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 257
4.5 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 259
5 Segmentation 267
5.1 Active contours . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 270
5.1.1 Snakes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 270
5.1.2 Dynamic snakes and CONDENSATION . . . . . . . . . . . . . . . . 276
5.1.3 Scissors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 280
5.1.4 Level Sets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 281
5.1.5 Application: Contour tracking and rotoscoping . . . . . . . . . . . . 282
5.2 Split and merge . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 284
5.2.1 Watershed . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 284
5.2.2 Region splitting (divisive clustering) . . . . . . . . . . . . . . . . . . 286
5.2.3 Region merging (agglomerative clustering) . . . . . . . . . . . . . . 286
5.2.4 Graph-based segmentation . . . . . . . . . . . . . . . . . . . . . . . 286
5.2.5 Probabilistic aggregation . . . . . . . . . . . . . . . . . . . . . . . . 288
5.3 Mean shift and mode finding . . . . . . . . . . . . . . . . . . . . . . . . . . 289
5.3.1 K-means and mixtures of Gaussians . . . . . . . . . . . . . . . . . . 289
5.3.2 Mean shift . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 292

5.4 Normalized cuts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 296
5.5 Graph cuts and energy-based methods . . . . . . . . . . . . . . . . . . . . . 300
5.5.1 Application: Medical image segmentation . . . . . . . . . . . . . . . 304
5.6 Additional reading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 305
5.7 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 306
6 Feature-based alignment 309
6.1 2D and 3D feature-based alignment . . . . . . . . . . . . . . . . . . . . . . 311
6.1.1 2D alignment using least squares . . . . . . . . . . . . . . . . . . . . 312
6.1.2 Application: Panography . . . . . . . . . . . . . . . . . . . . . . . . 314
6.1.3 Iterative algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . 315
6.1.4 Robust least squares and RANSAC . . . . . . . . . . . . . . . . . . 318
6.1.5 3D alignment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 320
6.2 Pose estimation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 321
6.2.1 Linear algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . . 322
6.2.2 Iterative algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . 324
6.2.3 Application: Augmented reality . . . . . . . . . . . . . . . . . . . . 326
6.3 Geometric intrinsic calibration . . . . . . . . . . . . . . . . . . . . . . . . . 327
6.3.1 Calibration patterns . . . . . . . . . . . . . . . . . . . . . . . . . . . 327
6.3.2 Vanishing points . . . . . . . . . . . . . . . . . . . . . . . . . . . . 329
6.3.3 Application: Single view metrology . . . . . . . . . . . . . . . . . . 331
6.3.4 Rotational motion . . . . . . . . . . . . . . . . . . . . . . . . . . . 332
6.3.5 Radial distortion . . . . . . . . . . . . . . . . . . . . . . . . . . . . 334
6.4 Additional reading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 335
6.5 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 336
7 Structure from motion 343
7.1 Triangulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 345
7.2 Two-frame structure from motion . . . . . . . . . . . . . . . . . . . . . . . . 347
7.2.1 Projective (uncalibrated) reconstruction . . . . . . . . . . . . . . . . 353
7.2.2 Self-calibration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 355
7.2.3 Application: View morphing . . . . . . . . . . . . . . . . . . . . . . 357
7.3 Factorization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 357
7.3.1 Perspective and projective factorization . . . . . . . . . . . . . . . . 360
7.3.2 Application: Sparse 3D model extraction . . . . . . . . . . . . . . . 362
7.4 Bundle adjustment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 363
7.4.1 Exploiting sparsity . . . . . . . . . . . . . . . . . . . . . . . . . . . 364
7.4.2 Application: Match move and augmented reality . . . . . . . . . . . 368
7.4.3 Uncertainty and ambiguities . . . . . . . . . . . . . . . . . . . . . . 370
7.4.4 Application: Reconstruction from Internet photos . . . . . . . . . . . 371
7.5 Constrained structure and motion . . . . . . . . . . . . . . . . . . . . . . . . 374
7.5.1 Line-based techniques . . . . . . . . . . . . . . . . . . . . . . . . . 374
7.5.2 Plane-based techniques . . . . . . . . . . . . . . . . . . . . . . . . . 376
7.6 Additional reading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 377
7.7 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 377
8 Dense motion estimation 381
8.1 Translational alignment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 384
8.1.1 Hierarchical motion estimation . . . . . . . . . . . . . . . . . . . . . 387
8.1.2 Fourier-based alignment . . . . . . . . . . . . . . . . . . . . . . . . 388
8.1.3 Incremental refinement . . . . . . . . . . . . . . . . . . . . . . . . . 392
8.2 Parametric motion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 398
8.2.1 Application: Video stabilization . . . . . . . . . . . . . . . . . . . . 401
8.2.2 Learned motion models . . . . . . . . . . . . . . . . . . . . . . . . . 403
8.3 Spline-based motion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 404
8.3.1 Application: Medical image registration . . . . . . . . . . . . . . . . 408
8.4 Optical flow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 409
8.4.1 Multi-frame motion estimation . . . . . . . . . . . . . . . . . . . . . 413
8.4.2 Application: Video denoising . . . . . . . . . . . . . . . . . . . . . 414
8.4.3 Application: De-interlacing . . . . . . . . . . . . . . . . . . . . . . 415
8.5 Layered motion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 415
8.5.1 Application: Frame interpolation . . . . . . . . . . . . . . . . . . . . 418
8.5.2 Transparent layers and reflections . . . . . . . . . . . . . . . . . . . 419
8.6 Additional reading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 421
8.7 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 422
9 Image stitching 427
9.1 Motion models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 430
9.1.1 Planar perspective motion . . . . . . . . . . . . . . . . . . . . . . . 431
9.1.2 Application: Whiteboard and document scanning . . . . . . . . . . . 432
9.1.3 Rotational panoramas . . . . . . . . . . . . . . . . . . . . . . . . . . 433
9.1.4 Gap closing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 435
9.1.5 Application: Video summarization and compression . . . . . . . . . 436
9.1.6 Cylindrical and spherical coordinates . . . . . . . . . . . . . . . . . 438
9.2 Global alignment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 441
9.2.1 Bundle adjustment . . . . . . . . . . . . . . . . . . . . . . . . . . . 441
9.2.2 Parallax removal . . . . . . . . . . . . . . . . . . . . . . . . . . . . 445
9.2.3 Recognizing panoramas . . . . . . . . . . . . . . . . . . . . . . . . 446
9.2.4 Direct vs. feature-based alignment . . . . . . . . . . . . . . . . . . . 450
9.3 Compositing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 450
9.3.1 Choosing a compositing surface . . . . . . . . . . . . . . . . . . . . 451
9.3.2 Pixel selection and weighting (de-ghosting) . . . . . . . . . . . . . . 453
9.3.3 Application: Photomontage . . . . . . . . . . . . . . . . . . . . . . 459
9.3.4 Blending . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 459
9.4 Additional reading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 462
9.5 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 463
10 Computational photography 467
10.1 Photometric calibration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 470
10.1.1 Radiometric response function . . . . . . . . . . . . . . . . . . . . . 470
10.1.2 Noise level estimation . . . . . . . . . . . . . . . . . . . . . . . . . 473
10.1.3 Vignetting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 474
10.1.4 Optical blur (spatial response) estimation . . . . . . . . . . . . . . . 476
10.2 High dynamic range imaging . . . . . . . . . . . . . . . . . . . . . . . . . . 479
10.2.1 Tone mapping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 487
10.2.2 Application: Flash photography . . . . . . . . . . . . . . . . . . . . 494
10.3 Super-resolution and blur removal . . . . . . . . . . . . . . . . . . . . . . . 497
10.3.1 Color image demosaicing . . . . . . . . . . . . . . . . . . . . . . . 502
10.3.2 Application: Colorization . . . . . . . . . . . . . . . . . . . . . . . 504
10.4 Image matting and compositing . . . . . . . . . . . . . . . . . . . . . . . . . 505
10.4.1 Blue screen matting . . . . . . . . . . . . . . . . . . . . . . . . . . . 507
10.4.2 Natural image matting . . . . . . . . . . . . . . . . . . . . . . . . . 509
10.4.3 Optimization-based matting . . . . . . . . . . . . . . . . . . . . . . 513
10.4.4 Smoke, shadow, and flash matting . . . . . . . . . . . . . . . . . . . 516
10.4.5 Video matting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 518
10.5 Texture analysis and synthesis . . . . . . . . . . . . . . . . . . . . . . . . . 518
10.5.1 Application: Hole filling and inpainting . . . . . . . . . . . . . . . . 521
10.5.2 Application: Non-photorealistic rendering . . . . . . . . . . . . . . . 522
10.6 Additional reading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 524
10.7 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 526
11 Stereo correspondence 533
11.1 Epipolar geometry . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 537
11.1.1 Rectification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 538
11.1.2 Plane sweep . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 540
11.2 Sparse correspondence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 543
11.2.1 3D curves and profiles . . . . . . . . . . . . . . . . . . . . . . . . . 543
11.3 Dense correspondence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 545
11.3.1 Similarity measures . . . . . . . . . . . . . . . . . . . . . . . . . . . 546
11.4 Local methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 548
11.4.1 Sub-pixel estimation and uncertainty . . . . . . . . . . . . . . . . . . 550
11.4.2 Application: Stereo-based head tracking . . . . . . . . . . . . . . . . 551
11.5 Global optimization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 552
11.5.1 Dynamic programming . . . . . . . . . . . . . . . . . . . . . . . . . 554
11.5.2 Segmentation-based techniques . . . . . . . . . . . . . . . . . . . . 556
11.5.3 Application: Z-keying and background replacement . . . . . . . . . . 558
11.6 Multi-view stereo . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 558
11.6.1 Volumetric and 3D surface reconstruction . . . . . . . . . . . . . . . 562
11.6.2 Shape from silhouettes . . . . . . . . . . . . . . . . . . . . . . . . . 567
11.7 Additional reading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 570
11.8 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 571
12 3D reconstruction 577
12.1 Shape from X . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 580
12.1.1 Shape from shading and photometric stereo . . . . . . . . . . . . . . 580
12.1.2 Shape from texture . . . . . . . . . . . . . . . . . . . . . . . . . . . 583
12.1.3 Shape from focus . . . . . . . . . . . . . . . . . . . . . . . . . . . . 584
12.2 Active rangefinding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 585
12.2.1 Range data merging . . . . . . . . . . . . . . . . . . . . . . . . . . 588
12.2.2 Application: Digital heritage . . . . . . . . . . . . . . . . . . . . . . 590
12.3 Surface representations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 591
12.3.1 Surface interpolation . . . . . . . . . . . . . . . . . . . . . . . . . . 592
12.3.2 Surface simplification . . . . . . . . . . . . . . . . . . . . . . . . . 594
12.3.3 Geometry images . . . . . . . . . . . . . . . . . . . . . . . . . . . . 594
12.4 Point-based representations . . . . . . . . . . . . . . . . . . . . . . . . . . . 595
12.5 Volumetric representations . . . . . . . . . . . . . . . . . . . . . . . . . . . 596
12.5.1 Implicit surfaces and level sets . . . . . . . . . . . . . . . . . . . . . 596
12.6 Model-based reconstruction . . . . . . . . . . . . . . . . . . . . . . . . . . . 598
12.6.1 Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 598
12.6.2 Heads and faces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 601
12.6.3 Application: Facial animation . . . . . . . . . . . . . . . . . . . . . 603
12.6.4 Whole body modeling and tracking . . . . . . . . . . . . . . . . . . 605
12.7 Recovering texture maps and albedos . . . . . . . . . . . . . . . . . . . . . 610
12.7.1 Estimating BRDFs . . . . . . . . . . . . . . . . . . . . . . . . . . . 612
12.7.2 Application: 3D photography . . . . . . . . . . . . . . . . . . . . . 613
12.8 Additional reading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 614
12.9 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 616
13 Image-based rendering 619
13.1 View interpolation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 621
13.1.1 View-dependent texture maps . . . . . . . . . . . . . . . . . . . . . 623
13.1.2 Application: Photo Tourism . . . . . . . . . . . . . . . . . . . . . . 624
13.2 Layered depth images . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 626
13.2.1 Impostors, sprites, and layers . . . . . . . . . . . . . . . . . . . . . . 626
13.3 Light fields and Lumigraphs . . . . . . . . . . . . . . . . . . . . . . . . . . 628
13.3.1 Unstructured Lumigraph . . . . . . . . . . . . . . . . . . . . . . . . 632
13.3.2 Surface light fields . . . . . . . . . . . . . . . . . . . . . . . . . . . 632
13.3.3 Application: Concentric mosaics . . . . . . . . . . . . . . . . . . . . 634
13.4 Environment mattes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 634
13.4.1 Higher-dimensional light fields . . . . . . . . . . . . . . . . . . . . . 636
13.4.2 The modeling to rendering continuum . . . . . . . . . . . . . . . . . 637
13.5 Video-based rendering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 638
13.5.1 Video-based animation . . . . . . . . . . . . . . . . . . . . . . . . . 639
13.5.2 Video textures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 640
13.5.3 Application: Animating pictures . . . . . . . . . . . . . . . . . . . . 643
13.5.4 3D Video . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 643
13.5.5 Application: Video-based walkthroughs . . . . . . . . . . . . . . . . 645
13.6 Additional reading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 648
13.7 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 650
14 Recognition 655
14.1 Object detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 658
14.1.1 Face detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 658
14.1.2 Pedestrian detection . . . . . . . . . . . . . . . . . . . . . . . . . . 666
14.2 Face recognition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 668
14.2.1 Eigenfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 671
14.2.2 Active appearance and 3D shape models . . . . . . . . . . . . . . . . 679
14.2.3 Application: Personal photo collections . . . . . . . . . . . . . . . . 684
14.3 Instance recognition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 685
14.3.1 Geometric alignment . . . . . . . . . . . . . . . . . . . . . . . . . . 686
14.3.2 Large databases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 687
14.3.3 Application: Location recognition . . . . . . . . . . . . . . . . . . . 693
14.4 Category recognition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 696
14.4.1 Bag of words . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 697
14.4.2 Part-based models . . . . . . . . . . . . . . . . . . . . . . . . . . . 701
14.4.3 Recognition with segmentation . . . . . . . . . . . . . . . . . . . . . 704
14.4.4 Application: Intelligent photo editing . . . . . . . . . . . . . . . . . 709
14.5 Context and scene understanding . . . . . . . . . . . . . . . . . . . . . . . . 712
14.5.1 Learning and large image collections . . . . . . . . . . . . . . . . . 714
14.5.2 Application: Image search . . . . . . . . . . . . . . . . . . . . . . . 717
14.6 Recognition databases and test sets . . . . . . . . . . . . . . . . . . . . . . . 718
14.7 Additional reading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 722
14.8 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 725
15 Conclusion 731
A Linear algebra and numerical techniques 735
A.1 Matrix decompositions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 736
A.1.1 Singular value decomposition . . . . . . . . . . . . . . . . . . . . . 736
A.1.2 Eigenvalue decomposition . . . . . . . . . . . . . . . . . . . . . . . 737
A.1.3 QR factorization . . . . . . . . . . . . . . . . . . . . . . . . . . . . 740
A.1.4 Cholesky factorization . . . . . . . . . . . . . . . . . . . . . . . . . 741
A.2 Linear least squares . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 742
A.2.1 Total least squares . . . . . . . . . . . . . . . . . . . . . . . . . . . 744
A.3 Non-linear least squares . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 746
A.4 Direct sparse matrix techniques . . . . . . . . . . . . . . . . . . . . . . . . . 747
A.4.1 Variable reordering . . . . . . . . . . . . . . . . . . . . . . . . . . . 748
A.5 Iterative techniques . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 748
A.5.1 Conjugate gradient . . . . . . . . . . . . . . . . . . . . . . . . . . . 749
A.5.2 Preconditioning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 751
A.5.3 Multigrid . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 753
B Bayesian modeling and inference 755
B.1 Estimation theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 757
B.1.1 Likelihood for multivariate Gaussian noise . . . . . . . . . . . . . . 757
B.2 Maximum likelihood estimation and least squares . . . . . . . . . . . . . . . 759
B.3 Robust statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 760
B.4 Prior models and Bayesian inference . . . . . . . . . . . . . . . . . . . . . . 762
B.5 Markov random fields . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 763
B.5.1 Gradient descent and simulated annealing . . . . . . . . . . . . . . . 765
B.5.2 Dynamic programming . . . . . . . . . . . . . . . . . . . . . . . . . 766
B.5.3 Belief propagation . . . . . . . . . . . . . . . . . . . . . . . . . . . 768
B.5.4 Graph cuts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 770
B.5.5 Linear programming . . . . . . . . . . . . . . . . . . . . . . . . . . 773
B.6 Uncertainty estimation (error analysis) . . . . . . . . . . . . . . . . . . . . . 775
C Supplementary material 777
C.1 Data sets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 778
C.2 Software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 780
C.3 Slides and lectures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 789
C.4 Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 790
References 791
Index 933
回复 支持 反对

使用道具 举报

发表于 2016-2-22 10:47:31 | 显示全部楼层
谢谢楼主分享,只可惜没金币
回复 支持 反对

使用道具 举报

发表于 2016-2-22 23:47:11 | 显示全部楼层
很好的资料
回复 支持 反对

使用道具 举报

发表于 2016-3-7 02:12:56 | 显示全部楼层
thanks dude
回复 支持 反对

使用道具 举报

发表于 2016-3-7 02:13:13 | 显示全部楼层
thanks dude
回复 支持 反对

使用道具 举报

发表于 2016-3-11 23:40:09 | 显示全部楼层
不错的文章
回复 支持 反对

使用道具 举报

您需要登录后才可以回帖 登录 | 立即注册

本版积分规则

Archiver|手机版|小黑屋|智能硬件开发社区 ( 京ICP备14010787号 )

GMT+8, 2019-2-23 14:30 , Processed in 0.082147 second(s), 18 queries .

Powered by Discuz! X3.3

© 2001-2013 Comsenz Inc.

快速回复 返回顶部 返回列表