CMSC828J
Advanced Topics in Information Processing: Image Segmentation
General Information 


Announcements:
Papers are now complete for discussion on March 7.
Rubric for presentations posted
The midterm was posted 3/26 (see below).
On 3/27 I updated the midterm with two minor clarifications.
On Monday, April 2, office hours will be 9:3010:30, and after class.
Description
What differentiates image segmentation from other clustering problems is that images have a natural 2d neighborhood structure. As a consequence, many segmentation algorithms can be thought of as diffusing information about image similarity among nearby pixels. We will begin by discussing diffusion processes, including anisotropic diffusion processes, which do exactly that. At the same time, we will discuss other local operations, such as edge detectors, that make judgements about image boundaries based on this information. We will then discuss approaches that diffuse probabilistic information by assuming Markov models of image probabilities. These methods include Markov random fields, belief propagation, and linear relaxation labeling. Other segmentation methods that rely on the natural graph structure of the image include normalized cut approaches to image segmentation, algebraic multigrid methods, and other graph algorithms such as shortest path methods for finding image segmentations. Finally, we will discuss methods for applying more generic clustering techniques, such as EM, to capitalize on the neighborhood structure of images. Along the way, we will consider segmentation methods that rely on texture, color, and motion cues. The goal of the class will be to familiarize students with current research approaches to image segmentation, while at the same time teaching the theoretical foundations underlying this work. A secondary goal will be to introduce many concepts that are fundamental in lowlevel vision (eg., filtering, edge detection, color and texture analysis).
The class will consist of lectures on basic material, and discussion and reading of work that is more current and/or speculative. Students will be required to prepare for and help lead some of the discussions. Students who are not leading discussions will still be expected to read papers to prepare for them. They will also do problem sets or projects in which they implement and test an approach to segmentation, or propose novel work on segmentation. There will also be a midterm and a final exam covering the basic computational techniques we’ve learned. Students will find it important to have some prior knowledge of vision, mathematical sophistication, and familiarity with topics such as calculus, linear algebra, probability and statistics.
Requirements
Here is my current plan for the workload of the class. This may change during the first two weeks, as the number of students settles down.
1) Reports. There will be (probably) 8 classes in which we discuss research papers. Prior to each of these classes, students must turn in a one page report on the papers to be discussed. Prior to class, I will pose one or more questions, one of which you should answer in your report. I prefer if you turn in a hardcopy to me at the start of class. Late papers will not be accepted, since the goal of these reports is to get you to think about papers before we discuss them. However, each student need not turn in a report on days when they are presenting a paper, and may also skip one additional report. 10% of grade
2) Presentations. Each student will be in charge of presenting an inclass summary of one paper, and leading a discussion about that paper. On discussion days, three papers will be presented. The three students presenting are encouraged to work together and coordinate their presentations. Rubric 15 % of grade.
3) Midterm, Final. These will be based on material from the lectures, and background reading for the lectures. 50% of grade
4) Problem Set/Project. Student will choose one: 25% of grade
a) Three problem sets will be assigned, requiring implementations of three of the algorithms discussed in class.
b) Programming/research project: This is meant to be a more openended project for students interested in research in image segmentation. It should involve implementation of existing or novel algorithms for segmentation, and experiments on a realworld data set.
Problem Sets
Please hand in your solution to the problem sets, including: 1) A document, with pictures when appropriate, describing your results; 2) Your code. I would prefer to receive your code by email in a zip file, and a hardcopy of the document, but I'll accept everything by email.
If problem sets are late, I will deduct 10% for each day they are late. Problem sets will not be accepted after the class following the due date.
Problem Set 
Supplementary Material 
Assigned 
Due 
2/27/12 
3/12/12 



4/4/12  
Problem Set 2  4/4/12  4/18/12  
4/26/12  5/9/12 
Class Schedule
This schedule should be considered more of a guideline than a rigid plan.
Lectures
Class 
Presenters 
Topic 
Background 
1. 1/25 
Jacobs 


2. 1/30 
Jacobs 
Vision Science, by Stephen Palmer, Chapter 6.
You're responsible for this material. See also:
Subjective Contours in Early Vision and Beyond, by Bela Julesz. Kanizsa, G., "Subjective Contours" Sci. Am. 234 (1976) 4852. 

3. 2/1

Jacobs 
This material is covered in many standard techniques. You might look at:A Wavelet Tour of Signal Processing , by Mallat for this and material on wavelets. Chapters 2 and 3 are on the Fourier Transform.
I also like the discussion in Elementary Functional Analysis by Shilov (This is part of the Dover Classics series, so there is a cheap paperback edition).
Some of this material is discussed in Forsyth and Ponce, Chapter 7. 

4. 2/6 
Jacobs 

5. 2/8 
Jacobs 
R. Ghez, Diffusion Phenomena . John Wiley and Sons, 2001, chapter 1. You're responsible for this material. 

6. 2/13 
Jacobs 
Forsyth and 

7. 2/15 
Jacobs 
"A review of nonlinear diffusion filtering," by Joachim Weickert. In ScaleSpace Theory in Computer Vision, Lecture Notes in Computer Science, Vol. 1252, Springer, Berlin, pp. 328, 1997. See also Weickert's book: Anisotropic Diffusion in Image Processing 

8. 2/20 
Alex, Abhishek and Kaustav 
Bilateral Filtering and Nonlocal Means.
Write a one page (or less) paper answering one of the following questions: 1) Based on all methods reviewed in both papers, does bilateral filtering still seem like a good filtering method? Why or why not? 2) If you had to pick a single reason why NLmeans works well, what is it? Please explain. What is the biggest disadvantage of NLmeans relative to other methods? 
Michael Elad. On the Origin of the Bilateral Filter and ways to improve it. IEEE Trans. on Image Processing, 2002.
A. Buades, B. Coll and J.M. Morel. A Review of Image Denoising Algorithms, with a New One. Siam Journal on Multiscale Modeling and Simulation, 2005.
See also:
Carlo Tomasi and Roberto Manduchi. Bilateral Filtering for Gray and Color Images. ICCV 1998.
Sylvain Paris and Fredo Durand. A fast approximation of the bilateral filter using a signal processing approach. ECCV 2006 
9. 2/22 
Jacobs 
D. Geiger, A. Gupta, L.A. Costa, and J. Vlontzos, "Dynamic programming for detecting, tracking, and matching deformable contours", IEEE Trans. PAMI, vol. PAMI17, no. 3, pp. 294302, Mar. 1995. Intelligent Scissors for Image Composition, by Eric Mortensen and William Barrett, SIGGRAPH '95. Williams, L.R. and K.K. Thornber, A Comparison of Measures for Detecting Natural Shapes in Cluttered Backgrounds, Intl. Journal of Computer Vision 34 (2/3), pp. 8196, 1999. A. Shashua and S. Ullman. Structural saliency: The detection of globallly salient structures using a locally connected network. In International Conference on Computer Vision, pages 321327, 1988. 

10. 2/27 
Jacobs 
Markov Random Fields Notes in text Also see Boykov, Veksler and Zabih. 
Markov Random Field Modeling in Image Analysis (Computer Science Workbench) by Stan Z. Li.
Fast Approximate Energy Minimization via Graph Cuts, by Boykov, Veksler, and Zabih. S.Geman and D.Geman. "Stochastic relaxation, gibbs distributions, and the bayesian restoration of images", IEEE Transactions on Pattern Analysis and Machine Intelligence, 6:721741, 1984. 
11. 2/29 
Jacobs 
Conditional Random Fields 
C. Sutton
and A. McCallum. An
introduction to conditional random fields for relational learning. In An
Introduction to Statistical Relational Learning, edited by Getoor and Taskar. 
12. 3/5 
Jacobs 
Graph Cuts See Boykov and Jolly 
Interactive Graph Cuts for Optimal Boundary & Region Segmentation of Objects in
ND images.
GrabCut
 Interactive Foreground Extraction using Iterated Graph Cuts 
13. 3/7 
Yue Angjoo Austin 
CRFs and MRFs and Graphbased methods 
Kumar, S., and Hebert, M. (2006). Discriminative random fields. International Journal of Computer Vision, 68(2). Felzenszwalb and Huttenlocher. Efficient Graphbased image segmentation. IJCV 2004. Felzenszwalb and Veksler. Tiered Scene Labeling with Dynamic Programming. CVPR 2010. 
14. 3/12 
Jacobs 
Forsyth and Jianbo Shi and Jitendra Malik. Normalized cuts and image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence , 22(8):888905, August 2000. Luxburg. A Tutorial on Spectral Clustering. http://www.kyb.mpg.de/publications/attachments/Luxburg06_TR_[0].pdf Agarwal, et al. Beyond Pairwise Clustering. 

15. 3/14 
Jacobs 
Distribution modeling: EM, Mean shift, Mixtures of Gaussians 
Forsyth and EM tutorial by Yair Weiss 
16. 3/26 
Jacobs 
J. A. Sethian, Level Set Methods and Fast
Marching Methods. Level
Set Methods in Image Science Richard Tsai and 

17. 3/28 
Jacobs 
Level Sets (continued) 
Chan, T.F.; Vese, 
18. 4/2 
Jacobs 
Wavelets and wavelet shrinkage 
There are many texts on wavelets available. I have made use of the following: A Wavelet Tour of Signal Processing, by Stephane Mallat, Academic Press, 1998. Ten Lectures on Wavelets, Ingrid Daubechies,
D. Donoho, I.Johnstone. Ideal spatial adaptation
by wavelet shrinkage. Biometrika, 1994. 
19. 4/4 
No class today 

An Introduction to Algebraic Multigrid,
by Klaus Stuben. Appendix A in Multigrid,
by U. Trottenberg, C. Oosterlee
and A. Schuller,, Academic Press, 2001. 
20. 4/9 
Jacobs 
Forsyth and 

21. 4/11 
Discussion: Fan, Victoria, Guangxiao 
Discussion 
Discussion: three of these papers M. Galun, E. Sharon, R. Basri, A. Brandt, Texture Segmentation by Multiscale Aggregation of Filter Responses and Shape
Elements,
Proceedings IEEE International Conference on Computer Vision, 716723, Nice, France, 2003.
Jitendra Malik, Serge Belongie,
Thomas Leung, and Jianbo Shi. Contour and
texture analysis for image segmentation. International Journal of
Computer Vision, 2000.
Shotton, Winn, Rother and Criminisi Yang, Wright, Ma and Sastry. Unsupervised segmentation of natural images via lossy data compression. CVIU 2008. 
22. 4/16 
Jacobs 
Motion Segmentation (we didn't get to optical flow) 
A. Jepson and M. Black, Mixture
models for optical flow, Tech. Report, Res. in Biol. and
Comp. Vision, Dept. of Comp. Sci.,
C.W. Gear, Multibody grouping from motion images. IJCV,
1998. (Available from me) 
23. 4/18 
Jacobs 
Riemannian manifolds  We didn't make it to this topic this year. 
Pennec, Fillard and Ayache. A Riemannian Framework for Tensor Computing. IJCV 2005.
Sochen, Kimmel, and Malladi. A general framework for lowlevel vision. IEEE Trans. on Image Processing, 1998. 
24. 4/23 
Snigdha Varun Brianna 
Discussion: 
Martin, Fowlkes, and Malik Learning to Detect Natural Image Boundaries Using Local Brightness, Color and Texture Cues
Dollar, P., Tu, Z., Belongie, S.: Supervised learning of edges and object boundaries In: CVPR. (2006) 19641971,
Sharon Alpert, Meirav Galun, Boaz Nadler, and Ronen Basri, “Detecting faint curved edges in noisy images,” European Conf. on Computer Vision (ECCV10), Crete, Greece, 2010 
25. 4/25 
Ejaz Sravanthi Sumit 
Discussion 
Y. Chai, V. Lempitsky, A. Zisserman, BiCoS: A Bilevel CoSegmentation Method for Image Classification, ICCV 2011.
Vicente, Kolmogorov, and Rother Cosegmentation Revisited: Models and Optimization ECCV 2010 
26. 4/30 
Shawn Steven (Xi) Chengxi 
Discussion 
Object detection with discriminatively trained partbased models. Felzenszwalb, Girshick, McAllester and Ramanan. PAMI 2010 Bagon and Galun, A unified multiscale framework for discrete energy minimization (available from me) Fast Motion Deblurring, by Cho and Lee

27. 5/2 
Konstantinos Jacobs 
Discussion 
Clustering by passing messages between daa points, by Frey and Dueck, Science. 
28. 5/7 
Jianyu (Leo) Priyanka An 
Discussion 
A. Levin and Y. Weiss,
Learning to Combine BottomUp and TopDown Segmentation, IJCV 2009. Decomposing a Scene Into Geometric and Sematically Consistent Regions Stephen Gould, Richard Fulton, Daphne Kohler Segmentation of Brain MR Images Through a Hidden Markov Random Field Model and the ExpectationMaximization Algorithm Yongyue Zhang, Michael Brady, and Stephen Smith IEEE Trans. on Medical Images, 2001 


29. 5/9 
Lee Jingjing Mohammed



FINAL 5/15 
1:30 PM, in Classroom 


Student Honor Code
The University of Maryland, College Park has a nationally recognized Code of Academic Integrity, administered by the Student Honor Council. This Code sets standards for academic integrity at Maryland for all undergraduate and graduate students. As a student you are responsible for upholding these standards for this course. It is very important for you to be aware of the consequences of cheating, fabrication, facilitation, and plagiarism. For more information on the Code of Academic Integrity or the Student Honor Council, please visit http://www.shc.umd.edu. To further exhibit your commitment to academic integrity, remember to sign the Honor Pledge on all examinations and assignments: "I pledge on my honor that I have not given or received any unauthorized assistance on this examination (assignment)."