Matthew Walmer

I'm a Computer Vision Research Scientist and Engineer. I recently finished obtaining my PhD at the University of Maryland, College Park, working under the advisement of Professor Abhinav Shrivastava in the Department of Computer Science.

I previously completed a Bachelor’s and Master’s degree in Biomedical Engineering at Johns Hopkins University with a focus on computational biology. My previous studies in neuroscience and human learning led me to take a deep interest in Machine Learning and Computer Vision and the amazing advances these fields have seen in the past decade and a half.

Email  /  Resume  /  CV  /  Google Scholar  /  LinkedIn  /  Twitter

My research focuses on understanding how Deep Neural Networks learn (or sometimes fail to learn). This includes research in Adversarial/Backdoor Attacks and Representation Learning. I have published works that examine the properties of the features of Vision Transformers (ViTs) trained with varying methods, and I have also proposed methods to improve the usefulness of ViT features for dense tasks and video tasks.

We present a method to efficiently upsample coarse pre-trained visual representations into pixel-dense features. Our approach, UPLiFT, has strong performance in predictive and generative downstream tasks and has better scaling properties than recent cross-attention-based methods. To achieve this, we propose a new Local Attender module based on a neighborhood of fixed spatial offsets.

We propose a method to leverage point-tracking trajectories to extract semantic-aware and motion-aligned visual representations for action recognition. Using these trajectory-aligned tokens (Trokens), we model both intra- and inter-trajectory dynamics, and we achieve state-of-the-art results in few-shot action recognition.

We revisit the design of transformer-based architectures for fine-grained video representation, and we present MV-Former, a Multi-entity Video Transformer that learns to parse video scenes as collections of salient entities.

We present a simple, self-supervised method to boost the density of pretrained ViT features, which enhances their performance in a range of spatially dense downstream tasks. LiFT is trained quickly with a multi-scale feature reconstruction objective and can be applied iteratively to make pixel-dense features.

A comparative study of Vision Transformers (ViTs) trained through different methods of supervision, including fully supervised and self-supervised methods. This analysis focuses on the networks’ attention, features, and downstream task performance.

A defense strategy for Dual-Key Backdoor Attacks through Trigger Inversion using Joint Optimization (TIJO). We show that joint optimization of both triggers in both domains is essential to overcome multimodal backdoor attacks.

Investigating ways to extend backdoor/trojan attacks into the multimodal domain, specifically Visual Question Answering (VQA). The proposed Dual-Key Backdoor Attack utilized multiple triggers in different modalities.

An investigation of the “in-the-wild” effectiveness of Adversarial Patch Attacks on Object Detection models. This includes the creation of the APRICOT dataset, as well as a study of patch effectiveness and defenses.

We model the activity of single neurons recorded in human subjects participating in a multi-source interference task. We show that neural activation correlates with an estimated latent cognitive state variable for focus.

Website template by Jon Barron (Source Code)