PhD Proposal: Hyperdimensional Binary Vector Models for Representation, Integration and Learning of Arbitrary Data

Peter Sutor
12.14.2021 10:00 to 12:00

IRB 4105

When it comes to various areas of AI and learning, or modalities of data, each enjoys their own unique discipline of machine learning with their own techniques for representing and processing data. As a result, fields such as computer vision have difficulty integrating with fields such as natural language processing. When they do, we find that training generally occurs for each in isolation, or must be tediously made end-to-end by integrating vastly different models before training. This is mostly due to the fact that data has no innate universal representation or even "currency" across the disciplines. The closest we have are real numbered vectors of weights of varying lengths.In this proposal, we look towards Hyperdimensional Binary Vectors (HBVs) and Hyperdimensional Computing for inspiration to generate a universal currency between modalities and learning paradigms: very long binary vectors of equal lengths. The proposed paradigm represents any unit of data/information as vectors of constant length. Whether something is a pixel, a character, a pattern of pixels/characters, full-blown images and text, or combinations of these - all exist as equally long binary vector embeddings in the same space, that are meaningfully constructed. Our current results show that such representations offer flexibility of representation of any data or feature, beyond even what is typical in each field (for example, irregularly shaped images). The mechanism behind learning semantically meaningful HBVs also allows for rapid learning and inference. We propose to create a general framework to achieve these results for arbitrary information with HBVs, primarily for vision and language, and show that classical machine learning techniques can benefit from such representations. We also propose to develop new methods of inference and learning that rely solely on HBV representations, to investigate the capabilities of HBVs to enable cross-modality fusion of data at an elemental level. As an end product, we propose to deliver an implementation of this framework that can be used with modern machine learning techniques, or as a standalone system for learning via Hyperdimensional Computing.Examining Committee:

Chair:Department Representative:Members:

Dr. Yiannis Aloimonos Dr. Ramani Duraiswami Dr. Cornelia Fermuller