Posted on Mon, Apr 28. The project must be submitted to the Teaching Assistant by Friday May 18. The project must be submitted by electronic mail to guerra@cs.umd.edu with subject "CMSC 733: P3". You must submit only a readme file (readme.txt), your source files (Matlab .m). You are required to have two files named "A.m" and "B.m" among your source files. These files implement each part of the project, respectively. They must read the given input and display the expected output. You must tar your multiple files into a single file named p3.tar. You must post a web page where the visualization of the processed video is available. Your readme file must contain additionally the web page address, your name, loginID and SSN.

Overview.

The purpose of this project is to perform some video manipulation. You need to alter video's content by inserting or deleting particular objects. In Part A, you must remove the content of a board (Bailey's banner) in the background of a  video  sequence and replace it with any other texture. The texture may be a planar picture (e.g. advertisement) of your choice. This picture must be properly projected onto the hockey rink wall. You may assume that the wall of the rink is a plane, but not a fronto-parallel plane. The replacement involves the automatic identification of the board corners according to the previous image and the detection of objects moving in front of the board sight of view in order to avoid wrong pixel replacements. In Part B (Final Exam Waiver), you must remove a moving person from the scene. Your input is a video took with a camera that seems to be rotating through the y-axis. The exepected output would be a corresponding  video  where the person (independent movement) disappears.

Part A: Replacing Banners.

Initially, identify manually the board corners in the first image and get the corresponding coordinates. The automatic detection of the board corners in the next image is implemented by searching the neighborhood of the previous coordinates in the next image Candidates for matches are computed according to a similarity criteria function (e.g. correlation).  You may also convert the images to gray scale or binary (black and white) images in order to ease this task. At this point, you may enforce the rectangular shape of the board by constraining the corners to form a parallelogram.  Given the four corner coordinates in two images, you may compute the homography mapping the points on the board between the two images. Basically, this is the same task as texture mapping in Computer Graphics. Note that each replaced pixel must be inside this parallelogram.

You also need to check if each pixel belongs either to the board or to any object in front of the board. In order to perform this task, you should refer to any motion segmentation (independent motion detection) technique. One approach is suggested as follows. Initially, you may generate a model background for the board. This model may be computed by superimposing (mosaicing) a number of images such that any occluding object is removed (review project 1).  In order to detect if any object (skaters) is occluding the banner you must compare this model against the banner image. Using the corners coordinates, apply a homography to convert the banner image into a fronto-planar (canonical) image. Then, partition the image into small blocks and linearly search for that block in the generated model according to the difference between this image and the generated model. The difference is computed using any metric-similarity function (e.g. Euclidean distance). You may use a threshold in order to identify whether there is an object occluding the banner at that block location and, hence, that part of the image is not replaced. Otherwise, you must replace that block with the corresponding part of the new texture.