Date/Place:

September 27th, 2009, 2pm - 5:30pm
Kyoto University, Engineering Building #3, room 1F

Lecturers:

Siwei Lyu, Computer Science Department, University at Albany, State University of New York

Stefan Roth, Department of Computer Science, TU Darmstadt, Germany

Slides:

• Part I: Fundamentals and Local Models (PDF, 12.9MB)
• Part II: MRF Models in the Pixel Domain (PDF, 20.1MB)
• Part III: MRF Models in the Wavelet Domain (PDF, 2.3MB)
  

Course Description:

As David Marr put it, “vision is a process that produces from images of the external world a description that is useful to the viewer”. But, as the basic input to any computer vision or biological vision system, the total number of all possible images is enormous. As a simple example, there are about 10^1,000 different 8-bit gray-scale images of a size as small as 20 by 20 pixels, while the current estimated total number of atoms in the universe is only about 10¹⁰⁰. However, most of these images, such as those that can be obtained by picking values for each pixel randomly, look like noise and lack any “interesting” structure. Moreover, they are very unlikely to be encountered by an imaging device (an eye or a camera) in the real world. Those that do, on the other hand, are loosely tagged as natural images.

Though occupying only a tiny fraction of the image space, natural images stand out with particular statistical properties, which play an essential role in low-level computer vision tasks, where corruptions that can affect higher-level vision tasks, such as noise, blur, damage, and low resolution, are reduced and removed. Similar challenges exist for a variety of other dense scene representations and their applications, including scene depth and image motion. Recently, we have witnessed a surge of interest in modeling statistics of natural images in the computer vision community with applications to problems ranging from low-level (e.g., denoising, super-resolution, inpainting, de-blurring), over mid-level (e.g., segmentation, color constancy, scene categorization) to high-level vision (e.g., object recognition). This short course will give an introduction to the basic aspects of natural image statistics, focusing on basic representations and statistical regularities. It will also describe recent developments in modeling natural image statistics and their applications to computer vision tasks.

Syllabus:

Part I: Fundamentals and Local Models (slides)

1. Introduction and motivation
   - Computer vision applications of natural image statistics
2. Themes and principles in natural image statistics research
   - Representation / observation / modeling / application cycle
   - Efficient coding / dependency reduction
   - Bayesian framework
3. Image representations from efficient coding transformations
   - Fourier
   - Multi-scale and multi-resolution representations
4. Statistical regularities of natural images
   - Frequency domain
   - Multi-scale domains
5. Local statistical models

Part II: MRF Models in the Pixel Domain (slides)

1. Overview of Markov random fields (MRFs)
   - Definition and representation
   - Basic properties
   - Pairwise MRFs (Geman & Geman, ...)
2. Overview of learning and inference
   - Maximum likelihood
   - Contrastive divergence
   - Score matching
   - Maximum pseudo-likelihood
   - Inference overview
3. Overviews of conditional random fields (CRFs)
   - loss-function based learning
4. Global MRF-based statistical models for natural images
   - MRFs and maximum entropy
   - Early models (Gaussian MRFs, Geman & Reynolds model, FRAME)
   - Fields of Experts
   - Gaussian CRFs for denoising
   - Steerable random fields
5. Applications and related challenges
   - denoising / inpainting / super-resolution / image-based rendering
   - modeling depth and image motion
   - applications to segmentation / recognition
   

Part III: MRF Models in the Wavelet Domain (slides)

1. Field of Gaussian Scale Mixtures
2. Implicit MRF model for natural images
3. Wrap-up and references
   
   

Biographies:

Siwei Lyu

Computer Science Department, University at Albany, State University of New York
lsw@cs.albany.edu

Siwei Lyu received his B.S. degree in Information Science in 1997 and his MS degree in Computer Science in 2000, both from Peking University (Beijing, China), and his Ph.D. degree in Computer Science from Dartmouth College in 2005. From 2000 to 2001, he worked at Microsoft Research Asia (then Microsoft Research China) as an assistant researcher. From 2005 to 2008, he was a post-doctoral research associate at Center for Neurosciences, New York University. He is currently an assistant professor at the Computer Science Department at SUNY Albany. His research interests are natural image statistics, digital image forensics, machine learning and computer vision.

Stefan Roth

Department of Computer Science, TU Darmstadt, Germany
sroth@cs.tu-darmstadt.de

Stefan Roth received a Diplom degree in Computer Science and Engineering in 2001 from Universität Mannheim, Germany, and his Sc.M. and Ph.D. degrees in Computer Science from Brown University in 2003 and 2007 respectively. Since September 2007 he has been an assistant professor of Computer Science at Technische Universität Darmstadt, Germany. His research interests focus on computer vision and machine learning, and include probabilistic approaches to image modeling and optical flow estimation, human detection and tracking, and object recognition. His dissertation received the Joukowsky Family Foundation Outstanding Dissertation Award from Brown University and was nominated for the ACM Doctoral Dissertation Award 2007. He received an honorable mention for the David Marr Prize at ICCV 2005 (with M. Black).