Updates/News

  • 9/20/2019
    ACM SAP Best Poster Award 2019
    Identifying Emotions from Walking using Affective and Deep Features. (Tanmay Randhavane, Uttaran Bhattacharya, Aniket Bera, Kyra Kapsaskis, Kurt Gray & Dinesh Manocha)
  • 9/17/2019
    Co-chair IROS 2019 Session: Human Detection and Tracking
  • 9/17/2019
    Presented 1 paper and 2 workshop talks at SAP 2019
  • 9/1/2019
    Started position at the University of Maryland College Park
  • 6/14/2019
    Presented 1 paper and 4 workshop talks at CVPR 2019
  • 5/31/2019
    Presented 1 paper and 3 workshop papers and talks at ICRA 2019
  • GAMMA
    GAMMA has published 5 papers in AI and Vision Conferences in 2019 (AAAI, CVPR, ICCV, NIPS)
  • GAMMA
    GAMMA has published 5 papers in IEEE VR in 2019 (AAAI, CVPR, ICCV, NIPS)
  • GAMMA
    GAMMA has published 15 papers in Robotics Conferences in 2019 (IROS, ICRA)
  • GAMMA
    GAMMA has published 4 papers in ACM SIGGRAPH 2018

Latest Publications

  • EVA: Modeling Perceived Emotions of Virtual Agents using Expressive Features of Gait and Gaze
    ACM SAP 2019
    Conference Papers Best Poster Award
    Tanmay Randhavane, Aniket Bera, Kyra Kapsaskis, Kurt Gray, Dinesh Manocha

  • DensePeds: Pedestrian Tracking in Dense Crowds Using FRVO and Sparse Features
    IROS 2019
    Conference Papers
    Rohan Chandra, Uttaran Bhattacharya, Aniket Bera, Dinesh Manocha

  • FVA: Modeling Perceived Friendliness of Virtual Agents Using Movement Characteristics
    TVCG / ISMAR 2019
    Journal Papers Conference Papers
    Tanmay Randhavane, Aniket Bera, Kyra Kapsaskis, Kurt Gray, Dinesh Manocha

  • TraPHic: Trajectory Prediction in Dense and Heterogeneous Traffic Using Weighted Interactions
    CVPR 2019
    Conference Papers
    Rohan Chandra, Uttaran Bhattacharya, Aniket Bera, Dinesh Manocha

  • LCrowdV: Generating labeled videos for pedestrian detectors training and crowd be-havior learning
    Neurocomputing 2019
    Journal Papers
    Ernest Cheung, Tsan Kwong Wong,Aniket Bera, Dinesh Manocha

  • Pedestrian Dominance Modeling for Socially-Aware Robot Navigation
    ICRA 2019
    Conference Papers
    Tanmay Randhavane,Aniket Bera, Emily Kubin, Austin Wang, Kurt Gray, Dinesh Manocha

  • TrackNPred: A Software Framework for End-to-End Trajectory Prediction
    ACM CSCS 2019
    Conference Papers
    Rohan Chandra, Uttaran Bhattacharya, Aniket Bera, Dinesh Manocha

Experience

  • Present 2019

    Assistant Research Professor

    University of Maryland College Park, UMIACS

  • 2019 2018

    Research Assistant Professor

    University of North Carolina Chapel Hill, Computer Science

  • 2018 2017

    Postdoctoral Research Associate

    University of North Carolina Chapel Hill, Computer Science

  • 2017 2012

    Graduate Student Researcher

    University of North Carolina Chapel Hill, Computer Science

  • Summer 2014

    Research Intern

    Disney Research LA

  • Summer 2013

    Research Intern

    Intel Corporation / Intel Labs

  • 2012 2010

    Research Intern

    C-DAC (India)

Education

  • Ph.D. 2017

    Ph.D. in Computer Science

    University of North Carolina at Chapel Hill

  • M.S. 2015

    M.S. in Computer Science

    University of North Carolina at Chapel Hill

  • M.B.A. 2012

    M.B.A. in Information Systems

    Jaypee Business School

  • B.Tech. 2011

    B.Tech. in Computer Science and Engineering

    Jaypee Institute of Information Technology

Student Supervision

  • Tanmay Randhavane (University Of North Carolina at Chapel Hill)
  • Rohan Chandra (University of Maryland at College Park)
  • Uttaran Bhattacharya (University of Maryland at College Park)
  • Trisha Mittal (University of Maryland at College Park)
  • Ernest Cheung (University Of North Carolina at Chapel Hill)
  • Vasavi Gajarla (Facebook)
  • Rohan Chabra (University Of North Carolina at Chapel Hill)
  • Rohan Prinja (Google)
  • Austin Wang (University Of North Carolina at Chapel Hill)
  • Anson Wong (The Chinese University of Hong Kong)
  • Husam Shaik (University Of North Carolina at Chapel Hill)

Public Datasets

  • Simulated pedestrian dataset with varying crowd density, population, lighting conditions, background scene, camera angles, agent personality and noise level to help in training models for crowd understanding, including pedestrian detection, crowd classification, etc.
    Link: http://gamma.cs.unc.edu/LCrowdV/#dataset
  • A public domain dataset, EWalk, with videos of walking individuals. We also provide their gaits in the form of 3D positions of 16 joints and labeled emotions obtained using a perception study
    Link: http://gamma.cs.unc.edu/GAIT/##EWalk
  • High resolution crowd dataset captured in dense pedestrian crossings in India. Dataset consists of videos of indoor and outdoor scenes recorded at different locations, each with 30-80 pedestrians.
    Link: http://gamma.cs.unc.edu/RCrowdT/
  • A traffic dataset (TRAF) comprising of dense and heterogeneous traffic. The dataset consists of the following road-agents: cars, buses, trucks, rickshaws, pedestrians, scooters, motorcycles, carts, and animals and is collected in dense Asian cities.
    Link: https://go.umd.edu/TRAF-Dataset

Our collaborators and funding agencies







"I think everyone should learn how to program a computer, because it teaches you how to think. I view computer science as a liberal art, something everyone should learn to do." - Steve Jobs

Artwork by Ashley Trinh, Inspiration by Paul Kube

Filter by type:

Sort by year:

RobustTP: End-to-End Trajectory Prediction for Heterogeneous Road-Agents in Dense Traffic with Noisy Sensor Inputs

Rohan Chandra, Uttaran Bhattacharya, Christian Roncal, Aniket Bera, Dinesh Manocha
Conference Papers ACM Computer Science for Cars Symposium 2019

Abstract

We present RobustTP, an end-to-end algorithm for predicting future trajectories of road-agents in dense traffic with noisy sensorinput trajectories obtained from RGB cameras (either static or moving) through a tracking algorithm. In this case, we consider noise as the deviation from the ground truth trajectory. The amount of noise depends on the accuracy of the tracking algorithm. Our approach is designed for dense heterogeneous traffic, where the road agents corresponding to a mixture of buses, cars, scooters, bicycles, or pedestrians. RobustTP is an approach that first computes trajectories using a combination of a non-linear motion model and a deep learning-based instance segmentation algorithm. Next, these noisy trajectories are trained using an LSTM-CNN neural network architecture that models the interactions between roadagents in dense and heterogeneous traffic. Our trajectory prediction algorithm outperforms state-of-the-art methods for end-to-end trajectory prediction using sensor inputs. We achieve an improvement of upto 18% in average displacement error and an improvement of up to 35.5% in final displacement error at the end of the prediction window (5 seconds) over the next best method. All experiments were set up on an Nvidia TiTan Xp GPU. Additionally, we release a software framework, TrackNPred. The framework consists of implementations of state-of-the-art tracking and trajectory prediction methods and tools to benchmark and evaluate them on real-world dense traffic datasets.

EVA: Generating Emotional Behavior of Virtual Agents using Expressive Features of Gait and Gaze

Tanmay Randhavane, Aniket Bera, Kyra Kapsaskis, Kurt Gray, Dinesh Manocha
Conference Papers ACM Symposium on Applied Perception 2019

Abstract

We present a novel, real-time algorithm, EVA, for generating virtual agents with various emotions. Our approach is based on using non-verbal movement cues such as gaze and gait to convey emotions corresponding to happy, sad, angry, or neutral. Our studies suggest that the use of EVA and gazing behavior can considerably increase the sense of presence in scenarios with multiple virtual agents. Our results also indicate that both gait and gazing features contribute to the perceptions of emotions in virtual agents.

FVA: Modeling Perceived Friendliness of Virtual Agents Using Movement Characteristics

Tanmay Randhavane, Aniket Bera, Kyra Kapsaskis, Kurt Gray, Dinesh Manocha
Conference Papers Journal Paper IEEE TVCG Special Issue for International Symposium on Mixed and Augmented Reality (ISMAR) 2019

Abstract

We present a new approach to improve the friendliness and warmth of a virtual agent in an AR environment by generating appropriate movement characteristics. Our algorithm is based on a novel data-driven friendliness model that is computed using a user-study and psychological characteristics. We investigated the perception of a user in an AR setting and observed that an FVA has a statistically significant improvement in the perceived friendliness and social presence of a user compared to an agent without the friendliness modeling.

DensePeds: Pedestrian Tracking in Dense Crowds Using Front-RVO and Sparse Features

Rohan Chandra, Uttaran Bhattacharya, Aniket Bera, Dinesh Manocha
Conference Papers International Conference on Intelligent Robots and Systems (IROS'19)

Abstract

We present a pedestrian tracking algorithm, DensePeds, that tracks individuals in highly dense crowds (greater than 2 pedestrians per square meter). Our approach is designed for videos captured from front-facing or elevated cameras. We present a new motion model called Front-RVO (FRVO) for predicting pedestrian movements in dense situations using collision avoidance constraints and combine it with state-of-the-art Mask R-CNN to compute sparse feature vectors that reduce the loss of pedestrian tracks (false negatives). We evaluate DensePeds on the standard MOT benchmarks as well as a new dense crowd dataset. In practice, our approach is 4.5 times faster than prior tracking algorithms on the MOT benchmark and we are state-of-the-art in dense crowd videos by over 2.6% on the absolute scale on average.

LCrowdV: Generating labeled videos for pedestrian detectors training and crowd behavior learning

Ernest Cheung, Tsan Kwong Wong, Aniket Bera, Xiaogang Wang, Dinesh Manocha
Journal Papers NeuroComputing 2019

Abstract

We present a novel procedural framework to generate an arbitrary number of labeled crowd videos (LCrowdV). The resulting crowd video datasets are used to design accurate algorithms or training models for crowded scene understanding. Our overall approach is composed of two components: a procedural simulation framework to generate crowd movements and behaviors, and a procedural rendering framework to generate different videos or images. Each video or image is automatically labeled based on the environment, number of pedestrians, density, behavior (personality), flow, lighting conditions, viewpoint, type of noise, etc. Furthermore, we can increase the realism by combining synthetically-generated behaviors with real-world background videos. We demonstrate the benefits of LCrowdV over prior lableled crowd datasets by augmenting a real dataset with it and improving the accuracy in pedestrian detection and crowd classification. Furthermore, we evaluate the impact of removing the variety in different LCrowdV parameters to show the importance of the diversity of data generated from our framework. LCrowdV has been made available as an online resource.

Pedestrian Dominance Modeling for Socially-Aware Robot Navigation

Randhavane, Tanmay ; Bera, Aniket; Wang, Austin; Kubin, Emily; Gray, Kurt; Manocha, Dinesh
Conference Papers IEEE International Conference on Robotics and Automation (ICRA'19)

Abstract

We present a Pedestrian Dominance Model (PDM) to identify the dominance characteristics of pedestrians for robot navigation. Through a perception study on a simulated dataset of pedestrians, PDM models the perceived dominance levels of pedestrians with varying motion behaviors corresponding to trajectory, speed, and personal space. At runtime, we use PDM to identify the dominance levels of pedestrians to facilitate socially-aware navigation for the robots. PDM can predict dominance levels from trajectories with ~85% accuracy. Prior studies in psychology literature indicate that when interacting with humans, people are more comfortable around people that exhibit complementary movement behaviors. Our algorithm leverages this by enabling the robots to exhibit complementing responses to pedestrian dominance. We also present an application of PDM for generating dominance-based collision-avoidance behaviors in the navigation of autonomous vehicles among pedestrians. We demonstrate the benefits of our algorithm for robots navigating among tens of pedestrians in simulated environments

TraPHic: Trajectory Prediction in Dense and Heterogeneous Traffic Using Weighted Interactions

Chandra, Rohan; Bhattacharya, Uttaran; Bera, Aniket; Manocha, Dinesh
Conference Papers IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2019)

Abstract

We present a new algorithm for predicting the near-term trajectories of road-agents in dense traffic videos. Our approach is designed for heterogeneous traffic, where the roadagents may correspond to buses, cars, scooters, bi-cycles, or pedestrians. We model the interactions between different roadagents using a novel LSTM-CNN hybrid network for trajectory prediction. In particular, we take into account heterogeneous interactions that implicitly accounts for the varying shapes, dynamics, and behaviors of different road agents. In addition, we model horizon-based interactions which are used to implicitly model the driving behavior of each road-agent. We evaluate the performance of our prediction algorithm, TraPHic, on the standard datasets and also introduce a new dense, heterogeneous traffic dataset corresponding to urban Asian videos and agent trajectories. We outperform state-of-the-art methods on dense traffic datasets by 30%.

Interactive Surveillance Technologies for Dense Crowds

Aniket Bera, Tanmay Randhavane, Dinesh Manocha
Workshop Papers Multi-modal Learning from Videos, IEEE Conference on Computer Vision and Pattern Recognition Workshops 2019

Behavior Modeling for Autonomous Driving

Aniket Bera, Tanmay Randhavane, Dinesh Manocha
Workshop Papers Human-Robot Teaming Beyond Human Operational Speeds ICRA 2019 Workshop

The Emotionally Intelligent Robot: Socially-aware Human Prediction

Aniket Bera, Tanmay Randhavane, Dinesh Manocha
Workshop Papers Long-term Human Motion Prediction Workshop 2019 & Human Movement Science for Physical Human-Robot Collaboration ICRA 2019 Workshop

The Socially Invisible Robot: Navigation in the Social World using Robot Entitativity

Bera, Aniket; Randhavane, Tanmay ; Kubin, Emily; Wang, Austin; Gray, Kurt; Manocha, Dinesh
Conference Papers IEEE/RSJ International Conference on Intelligent Robots and Systems 2018

Abstract

We present a real-time, data-driven algorithm to enhance the social-invisibility of robots within crowds. Our approach is based on prior psychological research, which reveals that people notice and–importantly–react negatively to groups of social actors when they have high entitativity, moving in a tight group with similar appearances and trajectories. In order to evaluate that behavior, we performed a user study to develop navigational algorithms that minimize entitativity. This study establishes mapping between emotional reactions and multi-robot trajectories and appearances, and further generalizes the finding across various environmental conditions. We demonstrate the applicability of our entitativity modeling for trajectory computation for active surveillance and dynamic intervention in simulated robot-human interaction scenarios. Our approach empirically shows that various levels of entitative robots can be used to both avoid and influence pedestrians while not eliciting strong emotional reactions, giving multi-robot systems socially-invisibility.

Data-Driven Modeling of Group Entitativity in Virtual Environments

Bera, Aniket; Randhavane, Tanmay ; Kubin, Emily; Shaik, Husam; Gray, Kurt; Manocha, Dinesh
Conference Papers ACM Symposium on Virtual Reality Software and Technology (VRST) 2018

Abstract

We present a data-driven algorithm to model and predict the socio-emotional impact of groups on observers. Psychological research finds that highly entitative i.e. cohesive and uniform groups induce threat and unease in observers. Our algorithm models realistic trajectory-level behaviors to classify and map the motion-based entitativity of crowds. This mapping is based on a statistical scheme that dynamically learns pedestrian behavior and computes the resultant entitativity induced emotion through group motion characteristics. We also present a novel interactive multi-agent simulation algorithm to model entitative groups and conduct a VR user study to validate the socio-emotional predictive power of our algorithm. We further show that model-generated high-entitativity groups do induce more negative emotions than low-entitative groups.

Classifying Driver Behaviors for Autonomous Vehicle Navigation

Cheung, Ernest; Bera, Aniket; Kubin, Emily; Gray, Kurt; Manocha, Dinesh
Workshop Papers Workshop on Planning, Perception and Navigation for Intelligent Vehicles 2018

Abstract

We present a novel approach to automatically identify driver behaviors from vehicle trajectories and use them for safe navigation of autonomous vehicles. We propose a novel set of features that can be easily extracted from car trajectories. We derive a data-driven mapping between these features and six driver behaviors using an elaborate web-based user study. We also compute a summarized score indicating a level of awareness that is needed while driving next to other vehicles. We also incorporate our algorithm into a vehicle navigation simulation system and demonstrate its benefits in terms of safer realtime navigation, while driving next to aggressive or dangerous drivers.

Socially Invisible Navigation for Intelligent Vehicles

Bera, Aniket; Randhavane, Tanmay ; Kubin, Emily; Wang, Austin; Gray, Kurt; Manocha, Dinesh
Workshop Papers Workshop on Planning, Perception and Navigation for Intelligent Vehicles 2018

Abstract

We present a real-time, data-driven algorithm to enhance the social-invisibility of autonomous vehicles within crowds. Our approach is based on prior psychological research, which reveals that people notice and–importantly–react negatively to groups of social actors when they have high entitativity, moving in a tight group with similar appearances and trajectories. In order to evaluate that behavior, we performed a user study to develop navigational algorithms that minimize entitativity. This study establishes mapping between emotional reactions and multi-robot trajectories and appearances, and further generalizes the finding across various environmental conditions. We demonstrate the applicability of our entitativity modeling for trajectory computation for active surveillance and dynamic intervention in simulated robot-human interaction scenarios. Our approach empirically shows that various levels of entitative robots can be used to both avoid and influence pedestrians while not eliciting strong emotional reactions, giving multi-robot systems socially-invisibility.

Identifying Driver Behaviors using Trajectory Features for Vehicle Navigation

Cheung, Ernest; Bera, Aniket; Kubin, Emily; Gray, Kurt; Manocha, Dinesh
Conference PapersIROS 2018

Abstract

We present a novel approach to automatically identify driver behaviors from vehicle trajectories and use them for safe navigation of autonomous vehicles. We propose a novel set of features that can be easily extracted from car trajectories. We derive a data-driven mapping between these features and six driver behaviors using an elaborate web-based user study. We also compute a summarized score indicating a level of awareness that is needed while driving next to other vehicles. We also incorporate our algorithm into a vehicle navigation simulation system and demonstrate its benefits in terms of safer real-time navigation, while driving next to aggressive or dangerous drivers.

Efficient and safe vehicle navigation based on driver behavior classification

Ernest Cheung, Aniket Bera, Emily Kubin, Kurt Gray, Dinesh Manocha
Workshop Papers Conference on Computer Vision and Pattern Recognition Workshops 2018

PORCA: Modeling and Planning for Autonomous Driving among Many Pedestrians

Luo, Yuanfu; Cai, Panpan; Bera, Aniket; Hsu, David; Lee, Wee-Sun; Manocha, Dinesh
Journal Papers Conference PapersIROS 2018

Abstract

This paper presents a planning system for autonomous driving among many pedestrians. A key ingredient of our approach is PORCA, a pedestrian motion prediction model that accounts for both a pedestrian’s global navigation intention and local interactions with the vehicle and other pedestrians. Unfortunately, the autonomous vehicle does not know the pedestrians’ intentions a priori and requires a planning algorithm that hedges against the uncertainty in pedestrian intentions. Our planning system combines a POMDP algorithm with the pedestrian motion model and runs in real time. Experiments show that it enables a robot scooter to drive safely, efficiently, and smoothly in a crowd with a density of nearly one person per square meter

Interactive Surveillance Technologies for Dense Crowds

Aniket Bera, Dinesh Manocha
Workshop Papers Association for the Advancement of Artificial Intelligence 2018 Fall Symposium Series 2018 (Artificial Intelligence in Government and Public Sector)

Behavior Modeling for Autonomous Driving

Aniket Bera, Dinesh Manocha
Workshop Papers Association for the Advancement of Artificial Intelligence 2018 Fall Symposium Series 2018 (Reasoning and Learning in Real-World Systems for Long-Term Autonomy)

Classifying Group Emotions for Socially-Aware Autonomous Vehicle Navigation

Bera, Aniket; Randhavane, Tanmay ; Kubin, Emily; Wang, Austin; Gray, Kurt; Manocha, Dinesh
Workshop Papers IEEE Conference on Computer Vision and Pattern Recognition Workshops 2018

Abstract

We present a real-time, data-driven algorithm to enhance the social-invisibility of autonomous vehicles within crowds. Our approach is based on prior psychological research, which reveals that people notice and–importantly–react negatively to groups of social actors when they have high entitativity, moving in a tight group with similar appearances and trajectories. In order to evaluate that behavior, we performed a user study to develop navigational algorithms that minimize entitativity. This study establishes mapping between emotional reactions and multi-robot trajectories and appearances, and further generalizes the finding across various environmental conditions. We demonstrate the applicability of our entitativity modeling for trajectory computation for active surveillance and dynamic intervention in simulated robot-human interaction scenarios. Our approach empirically shows that various levels of entitative robots can be used to both avoid and influence pedestrians while not eliciting strong emotional reactions, giving multi-robot systems socially-invisibility.

PORCA: Modeling and Planning for Autonomous Driving among Many Pedestrians

Bera, Aniket; Kim, Sujeong; Manocha, Dinesh
Journal Papers Collective Dynamics 2018

Abstract

We present a novel interactive multi-agent simulation algorithm to model pedestrian movement dynamics. We use statistical techniques to compute the movement patterns and motion dynamics from 2D trajectories extracted from crowd videos. Our formulation extracts the dynamic behavior features of real-world agents and uses them to learn movement characteristics on the fly. The learned behaviors are used to generate plausible trajectories of virtual agents as well as for long-term pedestrian trajectory prediction. Our approach can be integrated with any trajectory extraction method, including manual tracking, sensors, and online tracking methods. We highlight the benefits of our approach on many indoor and outdoor scenarios with noisy, sparsely sampled trajectory in terms of trajectory prediction and data-driven pedestrian simulation

Automatically Learning Driver Behaviors for Safe Autonomous Vehicle Navigation

Ernest Cheung, Aniket Bera, Emily Kubin, Kurt Gray, Dinesh Manocha
Workshop Papers Workshop on Planning, Perception and Navigation for Intelligent Vehicles 2018

MixedPeds: Pedestrian Detection in Unannotated Videos using Synthetically Generated Human-agents for Training

Cheung, Ernest; Wong, Anson; Bera, Aniket; Manocha, Dinesh
Conference Papers AAAI 2018

Abstract

We present a new method for training pedestrian detectors on an unannotated set of images. We produce a mixed reality dataset that is composed of real-world background images and synthetically generated static human-agents. Our approach is general, robust, and makes few assumptions about the unannotated dataset. We automatically extract from the dataset: i) the vanishing point to calibrate the virtual camera, and ii) the pedestrians' scales to generate a Spawn Probability Map, which is a novel concept that guides our algorithm to place the pedestrians at appropriate locations. After putting synthetic human-agents in the unannotated images, we use these augmented images to train a Pedestrian Detector, with the annotations generated along with the synthetic agents. We conducted our experiments using Faster R-CNN by comparing the detection results on the unannotated dataset performed by the detector trained using our approach and detectors trained with other manually labeled datasets. We showed that our approach improves the average precision by 5-13% over these detectors.

Realtime Pedestrian Tracking and Prediction in Dense Crowds

Aniket Bera, David Wolinski, Julien Pettré, Dinesh Manocha
Book Chapter Copyright © 2017 Elsevier Inc. | 2017 | ISBN-10: 978-0-12-809276-7

The problems of video-based pedestrian detection and path prediction have received considerable attention in robotics, traffic management, intelligent vehicles, video surveillance, and multimedia applications. In particular, some key challenges include the development of realtime or online methods as well as handling crowd videos with medium or high densities of pedestrians. We give an overview of realtime algorithms for extracting the trajectory of each pedestrian in a crowd video using a combination of nonlinear motion models and learning methods. These motion models are based on new collision-avoidance and local navigation algorithms that provide improved accuracy in dense settings. The resulting tracking algorithm can handle dense crowds with tens of pedestrians at realtime rates (25–30 fps). We also give an overview of techniques that combine these motion models with global movement patterns and Bayesian inference to predict the future position of each pedestrian over a long horizon. The combination of local and global features enables us to accurately predict the trajectory of each pedestrian in a dense crowd at realtime rates. We highlight the performance in real-world crowd videos with medium crowd density.

Interactive Tracking, Prediction and Behavior Learning of Pedestrians in Dense Crowds

Bera, Aniket
Thesis Report

Abstract

The ability to automatically recognize human motions and behaviors is a key skill for autonomous machines to exhibit to interact intelligently with a human-inhabited environment. The capabilities autonomous machines should have include computing the motion trajectory of each pedestrian in a crowd, predicting his or her position in the near future, and analyzing the personality characteristics of the pedestrian. Such techniques are frequently used for collision-free robot navigation, data-driven crowd simulation, and crowd surveillance applications. However, prior methods for these problems have been restricted to low-density or sparse crowds where the pedestrian movement is modeled using simple motion models. In this thesis, we present several interactive algorithms to extract pedestrian trajectories from videos in dense crowds. Our approach combines different pedestrian motion models with particle tracking and mixture models and can obtain an average of 20% improvement in accuracy in medium-density crowds over prior work. We compute the pedestrian dynamics from these trajectories using Bayesian learning techniques and combine them with global methods for long-term pedestrian prediction in densely crowded settings. Finally, we combine these techniques with Personality Trait Theory to automatically classify the dynamic behavior or the personality of a pedestrian based on his or her movements in a crowded scene. The resulting algorithms are robust and can handle sparse and noisy motion trajectories. We demonstrate the benefits of our long-term prediction and behavior classification methods in dense crowds and highlight the benefits over prior techniques. We highlight the performance of our novel algorithms on three different applications. The first application is interactive data-driven crowd simulation, which includes crowd replication as well as the combination of pedestrian behaviors from different videos. Secondly, we combine the prediction scheme with proxemic characteristics from psychology and use them to perform socially-aware navigation. Finally, we present novel techniques for anomaly detection in low-to medium-density crowd videos using trajectory-level behavior learning

F2FCrowds: Planning Agent Movements to Enable Face-to-Face Interactions

Randhavane, Tanmay ; Bera, Aniket; Manocha, Dinesh
Journal Papers Presence: Teleoperators and Virtual Environments 2017

Abstract

We present an approach for multi-agent navigation that facilitates face-to-face interaction in virtual crowds, based on a novel interaction velocity prediction (IVP) algorithm. Our user evaluation indicates that such techniques enabling face-to-face interactions can improve the sense of presence felt by the user. The virtual agents using these algorithms also appear more responsive and are able to elicit more reaction from the users.n

SocioSense: Robot Navigation Amongst Pedestrians with Social and Psychological Constraints

Bera, Aniket; Randhavane, Tanmay ; Manocha, Dinesh.
Conference PapersIROS 2017

Abstract

We present a real-time algorithm, SocioSense, for socially-aware navigation of a robot amongst pedestrians. Our approach computes time-varying behaviors of each pedestrian using Bayesian learning and Personality Trait theory. These psychological characteristics are used for long-term path prediction and generating proxemic characteristics for each pedestrian. We combine these psychological constraints with social constraints to perform human-aware robot navigation in low- to medium-density crowds. The estimation of timevarying behaviors and pedestrian personalities can improve the performance of long-term path prediction by 21%, as compared to prior interactive path prediction algorithms. We also demonstrate the benefits of our socially-aware navigation in simulated environments with tens of pedestrians.

Aggressive, Tense, or Shy? Identifying Personality Traits from Crowd Videos

Bera, Aniket; Randhavane, Tanmay ; Manocha, Dinesh
Conference PapersIJCAI 2017

Abstract

We present a real-time algorithm to automatically classify the dynamic behavior or personality of a pedestrian based on his or her movements in a crowd video. We present a statistical scheme that dynamically learns the behavior of every pedestrian in a scene and computes that pedestrian's motion model. This model is combined with global crowd characteristics to compute the movement patterns and motion dynamics, which can also be used to predict the crowd movement and behavior.

LCrowdV: Generating Labeled Videos for Simulation-based Crowd Behavior Learning

Cheung, Ernest; Wong, Tsan Kwong; Bera, Aniket; Wang, Xiaogang; Manocha, Dinesh
Workshop Papers European Conference on Computer Vision Workshop 2016

Abstract

We present a novel procedural framework to generate an arbitrary number of labeled crowd videos (LCrowdV). The resulting crowd video datasets are used to design accurate algorithms or training models for crowded scene understanding. Our overall approach is composed of two components: a procedural simulation framework for generating crowd movements and behaviors, and a procedural rendering framework to generate different videos or images. Each video or image is automatically labeled based on the environment, number of pedestrians, density, behavior (agent personality), flow, lighting conditions, viewpoint, noise, etc. Furthermore, we can increase the realism by combining syntheticallygenerated behaviors with real-world background videos. We demonstrate the benefits of LCrowdV over prior lableled crowd datasets, by augmenting real dataset with it and improving the accuracy in pedestrian detection. LCrowdV has been made available as an online resource.

Interactive Crowd-Behavior Learning for Surveillance and Training

Bera, Aniket; Kim, Sujeong; Manocha, Dinesh
Journal Papers IEEE Computer Graphics and Applications 2016 (Special Edition)

Abstract

The proposed interactive crowd-behavior learning algorithms can be used to analyze crowd videos for surveillance and training applications. The authors' formulation combines online tracking algorithms from computer vision, nonlinear pedestrian motion models from computer graphics, and machine learning techniques to automatically compute trajectory-level pedestrian behaviors for each agent in the video. These learned behaviors are used to automatically detect anomalous behaviors, perform motion segmentation, and generate realistic behaviors for virtual reality training applications.

Online parameter learning for data-driven crowd simulation and content generation

Bera, Aniket; Kim, Sujeong; Manocha, Dinesh
Journal Papers Computers & Graphics 2016

Abstract

We present an online parameter learning algorithm for data-driven crowd simulation and crowd content generation. Our formulation is based on incrementally learning pedestrian motion models and behaviors from crowd videos. We combine the learned crowd-simulation model with an online tracker to compute accurate, smooth pedestrian trajectories. We refine the motion model using an optimization technique to estimate the agents׳ simulation parameters. We also use an adaptive-particle filtering scheme for improved computational efficiency. We highlight the benefits of our approach for improved data-driven crowd simulation, including crowd replication, augmented crowds and merging the behavior of pedestrians from multiple videos. We highlight our algorithm׳s performance in various test scenarios containing tens of human-like agents and evaluate it using standard metrics.

Realtime Anomaly Detection Using Trajectory-Level Crowd Behavior Learning

Bera, Aniket; Kim, Sujeong ; Manocha, Dinesh
Conference PapersPETS / IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)

Abstract

We present an algorithm for realtime anomaly detection in low to medium density crowd videos using trajectorylevel behavior learning. Our formulation combines online tracking algorithms from computer vision, non-linear pedestrian motion models from crowd simulation, and Bayesian learning techniques to automatically compute the trajectory-level pedestrian behaviors for each agent in the video. These learned behaviors are used to segment the trajectories and motions of different pedestrians or agents and detect anomalies. We demonstrate the interactive performance on the PETS 2016 ARENA dataset as well as indoor and outdoor crowd video benchmarks consisting of tens of human agents.

Realtime Anomaly Detection Using Trajectory-Level Crowd Behavior Learning

Bera, Aniket; Kim, Sujeong; Randhavane, Tanmay; Pratapa, Srihari; Manocha, Dinesh
Conference PapersICRA 2016

Abstract

We present a novel real-time algorithm to predict the path of pedestrians in cluttered environments. Our approach makes no assumption about pedestrian motion or crowd density, and is useful for short-term as well as long-term prediction. We interactively learn the characteristics of pedestrian motion and movement patterns from 2D trajectories using Bayesian inference. These include local movement patterns corresponding to the current and preferred velocities and global characteristics such as entry points and movement features. Our approach involves no precomputation and we demonstrate the real-time performance of our prediction algorithm on sparse and noisy trajectory data extracted from dense indoor and outdoor crowd videos. The combination of local and global movement patterns can improve the accuracy of long-term prediction by 12-18% over prior methods in high-density videos.

Interactive and adaptive data-driven crowd simulation

Sujeong Kim, Aniket Bera, Dinesh Manocha
Conference PapersIEEE VR 2016

Abstract

We present an adaptive data-driven algorithm for interactive crowd simulation. Our approach combines realistic trajectory behaviors extracted from videos with synthetic multi-agent algorithms to generate plausible simulations. We use statistical techniques to compute the movement patterns and motion dynamics from noisy 2D trajectories extracted from crowd videos. These learned pedestrian dynamic characteristics are used to generate collision-free trajectories of virtual pedestrians in slightly different environments or situations. The overall approach is robust and can generate perceptually realistic crowd movements at interactive rates in dynamic environments. We also present results from preliminary user studies that evaluate the trajectory behaviors generated by our algorithm.

Interactive crowd content generation and analysis using trajectory-level behavior learn-ing

Sujeong Kim, Aniket Bera, Dinesh Manocha
Conference PapersIEEE International Symposium on Multimedia 2015

Abstract

We present an interactive approach for analyzing crowd videos and generating content for multimedia applications. Our formulation combines online tracking algorithms from computer vision, non-linear pedestrian motion models from computer graphics, and machine learning techniques to automatically compute the trajectory-level pedestrian behaviors for each agent in the video. These learned behaviors are used to detect anomalous behaviors, perform crowd replication, augment crowd videos with virtual agents, and segment the motion of pedestrians. We demonstrate the performance of these tasks using indoor and outdoor crowd video benchmarks consisting of tens of human agents, moreover, our algorithm takes less than a tenth of a second per frame on a multi-core PC. The overall approach can handle dense and heterogeneous crowd behaviors and is useful for realtime crowd scene analysis applications.

Efficient Trajectory Extraction and Parameter Learning for Data-Driven Crowd Simulation

Sujeong Kim, Aniket Bera, Dinesh Manocha
Conference PapersGraphics Interface 2015

Abstract

We present a trajectory extraction and behavior-learning algorithm for data-driven crowd simulation. Our formulation is based on incrementally learning pedestrian motion models and behaviors from crowd videos. We combine this learned crowd-simulation model with an online tracker based on particle filtering to compute accurate, smooth pedestrian trajectories. We refine this motion model using an optimization technique to estimate the agents’ simulation parameters. We highlight the benefits of our approach for improved data-driven crowd simulation, including crowd replication from videos and merging the behavior of pedestrians from multiple videos. We highlight our algorithm’s performance in various test scenarios containing tens of human-like agents

Efficient Trajectory Extraction and Parameter Learning for Data-Driven Crowd Simulation

Aniket Bera, Dinesh Manocha
Conference PapersICRA 2015

Abstract

We present a novel, real-time algorithm to extract the trajectory of each pedestrian in moderately dense crowd videos. In order to improve the tracking accuracy, we use a hybrid motion model that combines discrete and continuous flow models. The discrete model is based on microscopic agent formulation and is used for local navigation, interaction, and collision avoidance. The continuum model accounts for macroscopic behaviors, including crowd orientation and flow. We use our hybrid model with particle filters to compute the trajectories at interactive rates. We demonstrate its performance in moderately-dense crowd videos with tens of pedestrians and highlight the improved accuracy on different datasets.

Realtime Multilevel Crowd Tracking using Reciprocal Velocity Obstacles

Aniket Bera, Dinesh Manocha
Conference PapersIEEE International Conference on Pattern Recognition 2014

Abstract

We present a novel, realtime algorithm to compute the trajectory of each pedestrian in moderately dense crowd scenes. Our formulation is based on an adaptive particle filtering scheme that uses a multi-agent motion model based on velocity-obstacles, and takes into account local interactions as well as physical and personal constraints of each pedestrian. Our method dynamically changes the number of particles allocated to each pedestrian based on different confidence metrics. Additionally, we use a new high-definition crowd video dataset, which is used to evaluate the performance of different pedestrian tracking algorithms. This dataset consists of videos of indoor and outdoor scenes, recorded at different locations with 30-80 pedestrians. We highlight the performance benefits of our algorithm over prior techniques using this dataset. In practice, our algorithm can compute trajectories of tens of pedestrians on a multi-core desktop CPU at interactive rates (27-30 frames per second). To the best of our knowledge, our approach is 4-5 times faster than prior methods, which provide similar accuracy.

Realtime Multilevel Crowd Tracking using Reciprocal Velocity Obstacles

Aniket Bera, Nico Galoppo, Dillon Sharlet, Adam Lake, Dinesh Manocha
Conference PapersICRA 2014

Abstract

We present a novel realtime algorithm to compute the trajectory of each pedestrian in a crowded scene. Our formulation is based on an adaptive scheme that uses a combination of deterministic and probabilistic trackers to achieve high accuracy and efficiency simultaneously. Furthermore, we integrate it with a multi-agent motion model and local interaction scheme to accurately compute the trajectory of each pedestrian. We highlight the performance and benefits of our algorithm on well-known datasets with tens of pedestrians.

Line Based Robust Script Identification for Indian Languages

Bhupendra Kumar, Aniket Bera, and Tushar Patnaik
Journal PapersInternational Journal of Information and Electronics Engineering 2012

Abstract

In this paper a line based script identification using a hierarchical classification scheme is proposed to identify the Indian scripts includes Hindi, Gurumukhi and Bangla. We model the problem as topological, structural classification problem and examine the features inspired by human visual perception. Our basic algorithm uses different feature set at different level of classifier to optimize the tradeoff between accuracy and speed. The feature extraction is done on the subsets of image which in turn increases the performance of algorithm. The proposed system attains overall classification accuracy of 90% over the 2500+ text image data set.

Fast vectorization and upscaling images with natural objects using canny edge detection

Aniket Bera
Conference PapersInternational Conference on Electronics Computer Technology 2011

Abstract

In this paper a new, faster approach which is different from all the other conventional image vectorization techniques. Using canny edge detection we are able to find the sharp edges in the image and the assigning shades to each identifiable segment using random colour extraction from the original image. Finally mapping the colour blobs with the SVG Schema and generating a scalable vector image. This technique is efficient for natural and well as non-It can be directly used in security cameras for live image enhancement.

Scene flow estimation from stereo video sources

Aniket Bera
Conference PapersInternational Conference on Advances in Computing and Artificial Intelligence 2011

Abstract

There are various methods to estimate the scene flow. Most of the methods use motion estimation with stereo re-construction. This paper describes an interesting way to fuse the video from two camera's and create a 3D reconstruction. The proposed algorithm incorporates probabilistic distributions for optical flow and disparity. Multiple such re-created renderings can be put together to create re-timed movies of the event, with the resulting visual experience richer than that of a regular video clip, or switching between images from multiple cameras, do a head tracking of the viewer and change the view angle accordingly or view it on a mobile device using the accelerometer for camera tilting for the 3D effect.

Currrent Teaching

  • 2019 2020

    N/A

Teaching History

  • 2017 2017

    COMP 790-058 (Co-instructor): Multi-Agent Simulation for Crowds and Autonomous Driving

    Multi-agent systems are widely studied in different fields including robotics, AI, computer graphics and autonomous driving. In this course, we will cover the fundamentals of multi-agent simulation, mostly focusing on issues related to motion, behavior and navigation. The course will focus on three major applications: pedestrian and crowd simulation, real world pedestrian movement and analysis, and autonomous driving simulation. The course will consist of lectures by the instructors on the fundamental concepts in the areas, student lectures on selected topics of interests, and special guest lectures on recent research or work in progress. The goal of this class is to get students an appreciation of computational methods for motion planning and multi-agent simulation. We will discuss various considerations and tradeoffs used in designing various methodologies (e.g. time, space, robustness, and generality). This will include data structures, algorithms, computational methods, their complexity and implementation. Depending on the interests of the students, we may cover topics of interests in related areas. The course will include coverage of some software systems that are widely used to implement different motion planning and multi-agent simulation algorithms.

At My Office

You can find me at my office located at Stanford University Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua.

I am at my office every day from 7:00 until 10:00 am, but you may consider a call to fix an appointment.

At My Work

You can find me at my Work located at Stanford University Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua.

I am at my office every day from 7:00 until 10:00 am, but you may consider a call to fix an appointment.

At My Lab

You can find me at my office located at Stanford University Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua.

I am at my office every day from 7:00 until 10:00 am, but you may consider a call to fix an appointment.