Posted on Sat, May 04. The project must be submitted to the Teaching Assistant by Monday May 20. The project must be submitted by electronic mail from your class account ya426XX to the class account ya42601@umd5.umd.edu. You must post a web page describing your implementation in details. This web page must be reachable from anywhere and by anyone who has access to the internet. The web page must allow the downloading of your source files, and the visualization of the processed video. Your submission e-mail must have a subject "p4 web" and contain the web page address, your name, loginID and SSN.
Overview.
The purpose of this project is to remove the content of a board (Bailey's banner) in the background of a video sequence and replace it with any other texture. The texture may be a planar picture (e.g. advertisement) of your choice. This picture must be properly projected onto the hockey rink wall. You may assume that the wall of the rink is a plane, but not a fronto-parallel plane. The replacement involves the automatic identification of the board corners according to the previous image and the detection of objects moving in front of the board sight of view in order to avoid wrong pixel replacements.
Independent Motion Detection.
Initially, identify manually the board corners in the first image and get the corresponding coordinates. The automatic detection of the board corners in the next image is implemented by searching the neighborhood of the previous coordinates in the next image Candidates for matches are computed according to a similarity criteria function (e.g. correlation). You may also convert the images to gray scale or binary (black and white) images in order to ease this task. At this point, you may enforce the rectangular shape of the board by constraining the corners to form a parallelogram. Given the four corner coordinates in two images, you may compute the homography mapping the points on the board between the two images. Basically, this is the same task as texture mapping in Computer Graphics. Note that each replaced pixel must be inside this parallelogram.
You also need to check if each pixel belongs either to the board or to any object in front of the board. In order to perform this task, you should refer to any motion segmentation (independent motion detection) technique. One approach is suggested as follows. Initially, you may generate a model background for the board. This model may be computed by superimposing (mosaicing) a number of images such that any occluding object is removed (review project 3). In order to detect if any object (skaters) is occluding the banner you must compare this model against the banner image. Using the corners coordinates, apply a homography to convert the banner image into a fronto-planar (canonical) image. Then, partition the image into small blocks and linearly search for that block in the generated model according to the difference between this image and the generated model. The difference is computed using any metric-similarity function (e.g. Euclidean distance). You may use a threshold in order to identify whether there is an object occluding the banner at that block location and, hence, that part of the image is not replaced. Otherwise, you must replace that block with the corresponding part of the new texture.
Reference Hints