Ph.D. Candidate & Graduate Student Researcher
I am currently a final-year Ph.D. candidate in Robotics at UC Berkeley under the supervision of Prof. Masayoshi Tomizuka. I am working at Mechanical Systems Control (MSC) Laboratory at UC Berkeley. I have also been doing research for multiple projects at Berkeley DeepDrive (BDD).
My research interest lies at the intersection of machine learning, computer vision, reinforcement learning, control and optimization approaches and their applications to scene understanding, relational reasoning/interaction modeling, motion prediction, sequential decision making and state estimation for intelligent autonomous systems (e.g., autonomous vehicles, mobile robots). In particular, my current research focuses on enabling effective and efficient relational representation learning and reasoning to model interactive behaviors for multi-agent systems in uncertain, dynamically evolving environments, which leverages both the agent state information and 2D/3D visual semantics of the environmental context.
- Graph Neural Network
- Relational Reasoning
- Motion Prediction
- Sequential Decision Making
- Scene Understanding
- Multi-Target Tracking
I am very open to research discussion and collaboration, please feel free to get in touch!
- 08/2021: One paper regarding continual learning for trajectory prediction is accepted to IEEE Robotics and Automation Letters!
- 08/2021: Organizing NeurIPS 2021 Workshop on Machine Learning for Autonomous Driving
- 07/2021: Two papers are accepted to ICCV 2021! Preprints are coming soon.
- 06/2021: One paper regarding graph-based multi-agent prediction and tracking is accepted to IEEE Transactions on Intelligent Transportation Systems!
- 06/2021: Keynote talk on "Multi-Agent Relational Reasoning for Autonomous Driving" scheduled at IV 2021 workshop!
- 06/2021: One paper regarding self-supervised learning for trajectory forecasting is accepted at ITSC 2021!
- 06/2021: One paper regarding a survey of autonomous driving strategies at intersections is accepted at ITSC 2021!
- 06/2021: One paper regarding motion planning of mobile robots is accepted at IROS 2021!
- 05/2021: Organizing IROS 2021 Workshop on Multi-Agent Interaction and Relational Reasoning
- 04/2021: Organizing ICCV 2021 Workshop on Multi-Agent Interaction and Relational Reasoning
- 03/2021: Invited talk titled "Relational Reasoning for Multi-Agent Systems" at Carnegie Mellon University!
- 02/2021: One paper regarding shared cross-modal trajectory forecasting is accepted at CVPR 2021 (Oral)!
- 02/2021: One paper regarding multi-agent decision making is accepted at ICRA 2021!
- 02/2021: One paper regarding multi-agent trajectory forecasting is accepted at ICRA 2021!
- 09/2020: One paper regarding multi-agent trajectory prediction and relational reasoning is accepted at NeurIPS 2020!
- 09/2020: Recognized as a Top Reviewer for ICML 2020!
- 06/2020: Organizing IEEE IV 2020 Workshop on Behavior Prediction and Generation for Autonomous Driving!
- 07/2019: One paper regarding a generic tracking and prediction framework is accepted to IEEE Transactions on Intelligent Transportation Systems!
- 06/2019: Organizing IEEE IV 2019 Workshop on Prediction and Decision Making for Autonomous Driving!
- 05/2019: One paper regarding deep generative models for behavior prediction is accepted at IROS 2019!
- 01/2019: One paper regarding generative adversarial networks is accepted at ICRA 2019!
Ph.D. Candidate, Robotics
Aug 2016 -- Present
University of California, Berkeley, CA, USA
Advisior: Prof. Masayoshi Tomizuka
Specialization: machine learning, graph neural network, prediction, tracking, decision making
Sep 2019 - May 2021
Honda Research Institute USA, San Jose, CA, USA
Jun 2019 - Aug 2019
Toyota Research Institute, Los Altos, CA, USA
The ultimate goal of my research is to build intelligent and autonomous systems that can understand and interact with the physical environment, efficiently collaborate with other agents, and safely coordinate with humans. The methodologies mainly cover Bayesian theory, probabilistic graphical models, deep learning, graph neural networks, (inverse) reinforcement learning as well as computer vision techniques.
- Dynamic Relational Reasoning for Multi-Agent Systems, 2019--Present.
- Interaction-Aware Decision Making for Multi-Agent Systems, 2019--Present.
- General Motion Forecasting with Dynamic Key Information Selection, 2020--Present.
- Scene Understanding and Human Behavior Modeling / Analysis/ Prediction, 2018--Present.
- Interpretable and Data-Efficient Driving Behavior Generation via Deep Generative Probabilistic and Logic Models, 2018--2019.
- Multi-Object Tracking with Learning Based Sequential Monte Carlo Methods, 2017--2019.
- Generic Motion Generation and Comprehension with Social Interactions, 2017--2018.
- Motion Prediction for Urban Autonomous Driving Based on Stochastic Policy Learned via Deep Neural Network, 2016--2017.
Dynamic Relational Reasoning for Multi-Agent Systems
Multi-agent interacting systems are prevalent in the world, from pure physical systems to complicated social dynamic systems. A key challenge in developing artificial intelligence systems with the flexibility and efficiency of human cognition is giving them a similar ability - to reason about entities and their relations from data.
The research aims at taking a step forward towards explicit relational reasoning by inferring a latent interaction graph with graph neural networks.
Interaction-Aware Decision Making for Multi-Agent Systems
Deep reinforcement learning (DRL) provides a promising way for learning navigation in complex and interactive scenarios. However, identifying the subtle cues that can indicate drastically different outcomes remains an open problem with designing autonomous systems that operate in human environments. In addition, multi-agent interaction modeling is another open challenge.
The research aims at explicitly inferring the latent state of interactive agents and encoding spatial-temporal relationships in a reinforcement learning framework for the sequential decision making task in multi-agent systems.
Motion Forecasting with Dynamic Key Information Selection
Motion forecasting has been widely studied in various domains, such as physical systems, human skeletons, and multi-agent interacting systems (e.g. traffic participants, sports players). The problem is formulated as to predict future state sequences based on the historical spatio-temporal observations. However, the observed information may be of different levels of significance and in some situations not all the information is relevant for the forecasting. Moreover, the key information may be varying as the situation evolves.
The research aims at developing a general motion forecasting framework with dynamic key information selection to address above issues.
Deep Generative Models for Motion Generation and Prediction
The objective of generative models is to approximate the true data distribution, with which one can generate new samples similar to real data points with a proper variance. Generative models have been widely employed in tasks of representation learning and distribution approximation in literature, which basically fall into two categories: explicit density models and implicit density models. In recent years, since deep neural networks have been leveraged as universal distribution approximators thanks to its high flexibility, two deep generative models have been widely studied: Variational Auto-Encoder (VAE) and Generative Adversarial Network (GAN).
The research aims at exploring how to utilize the power of deep generative models in motion generation/prediction tasks. The methods are also generalizable to time-series prediction.
Probabilistic Graphical Models for Behavior Recognition and Prediction
Probabilistic graphical models (PGM) are important approaches for Bayesian inference, which are probabilistic models for which a graph expresses the conditional dependence structure between random variables. Generally, probabilistic graphical models use a graph-based representation as the foundation for encoding a distribution over a multi-dimensional space and a graph that is a compact or factorized representation of a set of independences that hold in the specific distribution.
The research aims at exploring how to utilize the power of probabilistic graphical models in behavior recognition and motion generation tasks. The proposed methods are also generalizable to time-series modeling with an emphasis on probabilistic inference.
Scene Understanding and Behavior / Trajectory Prediction
Effective understanding of the environment and accurate trajectory prediction of surrounding dynamic obstacles are critical for intelligent autonomous systems to achieve safe and high-quality decision making, motion planning and control.
The research aims at recognizing driver behaviors and traffic situations as well as predicting future trajectories for multiple highly interactive agents jointly.
Constrained Mixture Sequential Monte Carlo and Its Application to Multi-Object Tracking
The research aims at realizing accurate and robust tracking of multivariate dynamic systems with a focus on multi-agent autonomous systems. The proposed method can handle multi-modality of system state as well as missing observations (e.g. sensor occlusion in autonomous driving). The tracking framework can incorporate arbitrary learning-based system transition models.
* co-first author
- Lead organizer of Workshop at International Conference on Computer Vision (ICCV), 2021
- Co-organizer of Workshop at IEEE Conference on Robotics and Systems (IROS), 2021
- Co-organizer of Workshops at IEEE Intelligent Vehicles Symposium (IV), 2019, 2020, 2021
- Co-organizer of Workshop at IEEE Intelligent Transporation Systems Conference (ITSC), 2021
- Associate Editor at IEEE Intelligent Vehicles Symposium, 2020, 2021
- Program Committee of Workshop on AI for Autonomous Driving at IJCAI, 2021
- IEEE Transactions on Pattern Analysis and Machine Intelligence (T-PAMI), 2020 -- Present
- IEEE Transactions on Robotics (T-RO), 2020 -- Present
- IEEE Transactions on Vehicular Technology, 2021 -- Present
- IEEE Intelligent Systems, 2021 -- Present
- Signal Processing, 2021 -- Present
- Neural Computation and Applications (NCAA), 2020 -- Present
- IEEE Transactions on Mechatronics (T-MECH), 2019 -- Present
- IEEE Transactions on Intelligent Vehicles (T-IV), 2018 -- Present
- IEEE Transactions on Intelligent Transportation Systems (T-ITS), 2018 -- Present
- IEEE Transactions on Industrial Electronics (T-IE), 2017 -- Present
- International Conference on Learning Representations (ICLR), 2022
- International Conference on Computer Vision (ICCV), 2021
- Adcances in Neural Information Processing Systems (NeurIPS), 2020, 2021
- International Conference on Machine Learning (ICML), 2020, 2021
- IEEE International Conference on Robotics and Automation (ICRA), 2019, 2020, 2021
- IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2019, 2020, 2021
- IEEE Intelligent Vehicles Symposium (IV), 2019, 2020, 2021
- IEEE Intelligent Transportation Systems Conference (ITSC), 2019, 2020, 2021
- IEEE International Conference on Systems, Man, and Cybernetics (SMC), 2021
- Undergraduate students, 2019 -- Present
- Master students, 2020 -- Present