Analyzing Faces with Attributes

Attributes are semantic features of objects, people, and activities. They allow computers to describe people and things in the way humans would, which makes them very popular for recognition. Facial attributes - gender, hair color, makeup, eye color, etc. - are useful for a variety of different tasks, including face verification and recognition, user interface applications, and surveillance, to name a few.

We know that facial attributes are related, e.g. heavy makeup and wearing lipstick or male and goatee. Using attribute relationships, we show that we are able to improve the accuracy of their predictions. We incorporate these relationships into a deep CNN in three different ways allowing for learning of implicit and explicit attribute relationships.

A facial attribute classifier must be able to accurately predict attributes in low quality imagery, whether that is from a laptop, cellphone camera, or surveillance video. In order to ensure proper attribute prediction, we must have some way of adapting the model we have learned to new data. We develop several unsupervised domain adaptation approaches for facial attribute recognition. Using our domain adaptation methods, we are able to adapt an attribute model to any new set of face data.

Robust facial attribute recognition algorithms are necessary for improving the applications which use these attributes. With video data, attributes can be localized through motion, and their stability over time can be studied, leading to more robust attribute classifiers. The activations of deep networks can be used for attribute-based face parsing (or semantic segmentation), which will be useful for face detection, pose estimation, and facial landmark extraction. Facial attributes lend themselves nicely to the problem of subject clustering, and yet they have never been applied to this problem. Attribute-based clustering can highlight the discriminative power of different attributes, and the relationships amongst attributes, which can be used to improve face recognition by ignoring attributes that do not provide identity-related information. Framing the problem of attribute prediction as one of constructing hash codes for image retrieval, more robust attribute classifiers can be developed by focusing on the capability of the attributes to reconstruct the original image.

The problem of predicting facial attributes is still relatively new in computer vision. There are many different directions for future research, with the use of attributes for clustering, image parsing, and hashing largely unexplored. I plan to explore methods for building more robust attribute classifiers for these applications.

Examining Committee:

Chair: Dr. Rama Chellappa

Dept rep: Dr. Don Perlis

Member: Dr. David Jacobs