PhD Proposal: Learning Robust Feature Representation for Unconstrained Video-Based Face Recognition

Face identification and verification is an important problem in computer vision and has been actively researched for over two decades. Many algorithms have shown to work well on images collected in controlled settings. However, the performance of these algorithms often degrades significantly on images that have large variations such as pose, illumination, expression, aging, cosmetics, and occlusion.
How to extract robust and discriminative feature representations from face images/videos is an important problem to achieve good performance in these tasks. In this proposal, we present several approaches to extract robust feature representation from a set of images/video frames for face identification and verification problems.
First, we present a dictionary approach with dense facial landmark features. Each face video is segmented into K partitions first, and the multi-scale features are extracted from the patches centered at detected facial landmarks. Then, compact and representative dictionaries are learned from dense features for each partition of a video and then concatenated together into a video dictionary representation for the video. The results show that the representation is effective for the unconstrained video-based face identification task. Secondly, we present a landmark-based Fisher vector approach for video-based face verification problems. The approach encodes over-complete local features into a high-dimensional feature representation followed by a learned joint Bayesian metric to project the feature vector into a low-dimensional space and to compute the similarity score. Finally, we present an approach using the features from a deep convolutional neural network (DCNN) trained using a large-scale face dataset. Our experimental results show that the DCNN model is able to characterize the face variations from the large-scale source face dataset and generalizes well to another smaller one.
Examining Committee:
Committee Chair: - Dr. Rama Chellappa
Dept's Representative - Dr. James A. Reggia
Committee Member(s): - Dr. David W. Jacobs