CMSC733 Computer Processing of Pictorial Information

General Information



Class Time  

Tue., Thurs. 11:00-12:15


CSI 3118

Course Info

See below








David Jacobs

Hao Zhou


djacobs at cs

zhhoper at gmail


AVW 4421

AVW 4420

Office hours

Tuesdays 2-3

Mondays 2-4



This class will provide a general, graduate level introduction to computer vision. We will attempt to give students a broad understanding of all important topics in computer vision.


Knowledge of and comfort with mathematics will be very helpful. At a minimum, students should know multivariable calculus, probability and linear algebra. No knowledge of computer vision will be assumed, but we will go quickly, so some background in computer vision will certainly be helpful.


There is no single required text. Required readings for the class will be listed below, and will be available online. Three books that will be useful are:


A draft of Richard Szeliski's computer vision book is available online. There will be some required readings from this book.


Introductory Techniques for 3-D Computer Vision by Trucco and Verri. This is more of an undergraduate text, and a bit old, so many topics are not covered. However, the fundamentals are explained very clearly.


Computer Vision: A Modern Approach by Forsyth and Ponce (2nd edition). This is well worth buying if you are a graduate student in computer vision.


Assigned work for the class will consist of problem sets (40% of grade), a take-home midterm (20%), and a final exam (40%).

Homework assignments are to be written up neatly and clearly, and programming assignments must be clear and well-documented. Programs should be written inMatlab. A full paper copy of all of the homework must be turned in. In addition, we will ask you to email a copy of all code to the TA, Hao Zhou, at: zhhoper at gmail.


Problem Sets

(These problem sets are listed to give you an idea of what will be required. They may change, so do not start on them until the date on which they are assigned).





Problem Set 1



Edge Detection. Implement 2D edge detection, paper and pencil problems about convolution.  Problem Set  You will also use the following Matlab files: test_smooth_image.m, test_image_gradient.m, test_gradient_magnitude_direction.m, interpolate_gradients.m, and the images: swanbw.jpg, swanedges.jpg, swanedges_h.jpg

Problem Set 2



Normalized Cut and Texture Synthesis. Problem Set.

To test your normalized cut code, you will use the routine: test_normalized_cut_points.m. My results with this routine are here and here.

For texture synthesis, look at the Efros and Leung paper. You can use the brick image to test your code.

Problem Set 3



E-M and Mosaicing. Problem Set.

For the mosaicing problem, you will use these images: Image 1 Image 2 and this matlab code.




Problem Set 4



Stereo matching with graph cuts. Problem Set.

First test image pair: I1L.jpg I1R.jpg

Second test image pair: T3bw.jpg T4bw.jpg Tresult.jpg

Problem Set 5



Bag of words classification. Problem Set. ps5.m

Review for Final





Class Schedule

This schedule should be considered more of a guideline than a rigid plan.




Required Reading

Background Reading

Problem Sets

1. 9/2





2. 9/4

Fourier 1

Fourier Transforms

This material is covered in many standard techniques.  You might look at:A Wavelet Tour of Signal Processing , by Mallat for this and material on wavelets. Chapters 2 and 3 are on the Fourier Transform.


I also like the discussion in Elementary Functional Analysis by Shilov (This is part of the Dover Classics series, so there is a cheap paperback edition).


Some of this material is discussed in Forsyth and Ponce, Chapter 7.


3. 9/9

Fourier 2



Szeliski, 3.2 and 3.4


4. 9/11

Diffusion and smoothing



Diffusion Phenomena, by Ghez, Sections 1.1-1.4 (available from instructor)



5. 9/16

Edge Detection

Edge Detection


Canny edge detector

6. 9/18

Non-linear Diffusion

Non-linear Diffusion


"A review of nonlinear diffusion filtering," by Joachim Weickert.  In Scale-Space Theory in Computer Vision, Lecture Notes in Computer Science, Vol. 1252, Springer, Berlin, pp. 3-28, 1997.

See also Weickert's book: Anisotropic Diffusion in Image Processing


7. 9/23

Bilateral Filtering and Normalized Cut

Carlo Tomasi and Roberto ManduchiBilateral Filtering for Gray and Color Images. ICCV 1998.

Normalized Cut.

Jianbo Shi and Jitendra Malik. Normalized cuts and image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence , 22(8):888-905, August 2000.

Michael EladOn the Origin of the Bilateral Filter and ways to improve it.  IEEE Trans. on Image Processing, 2002.



8. 9/25

Attend Talk:

Visual Recognition for the People and of the People: Tamara Berg

AV Williams 2460




9. 9/30



Texture Synthesis by Non-parametric Sampling by Efros and Leung. Web page contains links to the paper and pseudocode.


Texture synthesis


K-means, E-M, Color Quantization and Background Subtraction

E-M, Mean shift, Mixtures of Gaussians


Adaptive background mixture models for real-time tracking, by Stauffer and Grimson

Forsyth and Ponce, Computer Vision A Modern Approach, Chapter 16

E-M tutorial by Yair Weiss

Szeliski, Section 5.3


13. 10/14

Matching: SIFT and RANSAC

David G. Lowe, "Distinctive image features from scale-invariant keypoints," International Journal of Computer Vision, 60, 2 (2004), pp. 91-110.

RANSAC slides

The RANSAC song



14. 10/16

Geometric transformations and mosaicing

Geometric Transformation Slides


15. 10/21

Biological Vision



16. 10/23

Cameras, perspective projection



Projective Geomery for Image Analysis by Mohr and Triggs



17. 10/28

Projective geometry



18. 10/30

Stereo geometry

Slides, Szeliski, 11-11.1.1


19. 11/4

Markov Processes and Markov Random Fields




20. 11/6

Graph cuts for segmentation, stereo and MRFs

Interactive Graph Cuts for Optimal Boundary & Region Segmentation of Objects in N-D images.
Yuri Boykov and Marie-Pierre Jolly. In International Conference on Computer Vision, (ICCV), vol. I, pp. 105-112, 2001.



21. 11/11

Sliding window detection

Rapid object detection using a boosted cascade of simple features by Viola and Jones



22. 11/13




Graph cut stereo

23. 11/18

Stereo matching (continued): dynamic programming and graph cuts

Szeliski, 11.3-11.5.1

Fast approximate energy minimization via graph cuts, by Boykov, Veksler, and Zabih



23. 11/18



24. 11/20

Structure-from-motion with perspective, the Essential matrix

Szeliski, 7.2 through 7.2.1



25. 11/25

Classification: Bag of Words

Slides (from Andrew Zisserman)

Text (from Kristin Grauman)


Bag of words classifier

26. 12/2

Optical Flow, corner detection, Motion Flow



Handout available from instructor


27. 12/4

Deep Learning and Classification



28. 12/9

Fine-grained classification




29. 12/11




12/15 8:00-10:00