Ph.D. Candidate & Graduate Student Researcher
I am currently a final-year Ph.D. candidate in Robotics at UC Berkeley under the supervision of Prof. Masayoshi Tomizuka. I am working at Mechanical Systems Control (MSC) Laboratory at UC Berkeley. I have also been doing research for multiple projects at Berkeley DeepDrive (BDD).
My research interest lies at the intersection of machine learning, computer vision, reinforcement learning, control and optimization approaches and their applications to scene understanding, relational reasoning/interaction modeling, motion prediction, sequential decision making and state estimation for intelligent autonomous systems (e.g., autonomous vehicles, mobile robots). In particular, my current research focuses on enabling effective and efficient relational representation learning and reasoning to model interactive behaviors for multi-agent systems in uncertain, dynamically evolving environments, which leverages both the agent state information and 2D/3D visual semantics of the environmental context.
- Graph Neural Network
- Relational Reasoning
- Motion Prediction
- Sequential Decision Making
- Scene Understanding
- Multi-Target Tracking
I am very open to research discussion and collaboration, please feel free to get in touch!
- 04/2021: Organizing ICCV 2021 Workshop on Multi-Agent Interaction and Relational Reasoning
- 03/2021: Invited talk titled "Relational Reasoning for Multi-Agent Systems" at Carnegie Mellon University!
- 02/2021: One paper regarding shared cross-modal trajectory forecasting is accepted at CVPR 2021 (Oral)!
- 02/2021: Two papers regarding multi-agent decision making / trajectory forecasting are accepted at ICRA 2021!
- 09/2020: One paper regarding multi-agent trajectory prediction and relational reasoning is accepted at NeurIPS 2020!
- 09/2020: Recognized as a Top Reviewer for ICML 2020!
- 06/2020: Organizing IEEE IV 2020 Workshop on Behavior Prediction and Generation for Autonomous Driving!
- 07/2019: One paper regarding a generic tracking and prediction framework is accepted at IEEE Transactions on Intelligent Transportation Systems!
- 06/2019: Organizing IEEE IV 2019 Workshop on Prediction and Decision Making for Autonomous Driving!
- 05/2019: One paper regarding deep generative models for behavior prediction is accepted at IROS 2019!
- 01/2019: One paper regarding generative adversarial networks is accepted at ICRA 2019!
Click here to download my full CV.
Ph.D. Candidate, Robotics
Aug 2016 -- Present
University of California, Berkeley, CA, USA
Advisior: Prof. Masayoshi Tomizuka
Specialization: machine learning, graph neural network, prediction, tracking, decision making
Aug 2012 -- Jul 2016
Harbin Institute of Technology, Harbin, China
Advisors: Prof. Huijun Gao and Prof. Shen Yin
Thesis: Partial Least Squares and Its Application to Process Control
Sep 2019 - Present
Honda Research Institute USA, San Jose, CA, USA
- Working on machine learning and computer vision algorithms for scene understanding and human behavior modeling
- Proposed and developed a generic multi-agent prediction framework with dynamic relational reasoning
- Proposed and developed a generic motion forecasting framework with dynamic key information selection
- Proposed and developed a interaction-aware decision making framework for autonomous driving
Jun 2019 - Aug 2019
Toyota Research Institute, Los Altos, CA, USA
- Proposed and developed a sample efficient, hybrid prediction system with graph neural network
The ultimate goal of my research is to build intelligent and autonomous systems that can understand and interact with the physical environment, efficiently collaborate with other agents, and safely coordinate with humans. The methodologies mainly cover Bayesian theory, probabilistic graphical models, deep learning, graph neural networks, (inverse) reinforcement learning as well as computer vision techniques.
- Dynamic Relational Reasoning for Multi-Agent Systems, 2019--Present.
- Interaction-Aware Decision Making for Multi-Agent Systems, 2019--Present.
- General Motion Forecasting with Dynamic Key Information Selection, 2020--Present.
- Scene Understanding and Human Behavior Modeling / Analysis/ Prediction, 2018--Present.
- Interpretable and Data-Efficient Driving Behavior Generation via Deep Generative Probabilistic and Logic Models, 2018--2019.
- Multi-Object Tracking with Learning Based Sequential Monte Carlo Methods, 2017--2019.
- Generic Motion Generation and Comprehension with Social Interactions, 2017--2018.
- Motion Prediction for Urban Autonomous Driving Based on Stochastic Policy Learned via Deep Neural Network, 2016--2017.
Dynamic Relational Reasoning for Multi-Agent Systems
Multi-agent interacting systems are prevalent in the world, from pure physical systems to complicated social dynamic systems. A key challenge in developing artificial intelligence systems with the flexibility and efficiency of human cognition is giving them a similar ability - to reason about entities and their relations from data.
The research aims at taking a step forward towards explicit relational reasoning by inferring a latent interaction graph with graph neural networks.
Interaction-Aware Decision Making for Multi-Agent Systems
Deep reinforcement learning (DRL) provides a promising way for learning navigation in complex and interactive scenarios. However, identifying the subtle cues that can indicate drastically different outcomes remains an open problem with designing autonomous systems that operate in human environments. In addition, multi-agent interaction modeling is another open challenge.
The research aims at explicitly inferring the latent state of interactive agents and encoding spatial-temporal relationships in a reinforcement learning framework for the sequential decision making task in multi-agent systems.
Motion Forecasting with Dynamic Key Information Selection
Motion forecasting has been widely studied in various domains, such as physical systems, human skeletons, and multi-agent interacting systems (e.g. traffic participants, sports players). The problem is formulated as to predict future state sequences based on the historical spatio-temporal observations. However, the observed information may be of different levels of significance and in some situations not all the information is relevant for the forecasting. Moreover, the key information may be varying as the situation evolves.
The research aims at developing a general motion forecasting framework with dynamic key information selection to address above issues.
Deep Generative Models for Motion Generation and Prediction
The objective of generative models is to approximate the true data distribution, with which one can generate new samples similar to real data points with a proper variance. Generative models have been widely employed in tasks of representation learning and distribution approximation in literature, which basically fall into two categories: explicit density models and implicit density models. In recent years, since deep neural networks have been leveraged as universal distribution approximators thanks to its high flexibility, two deep generative models have been widely studied: Variational Auto-Encoder (VAE) and Generative Adversarial Network (GAN).
The research aims at exploring how to utilize the power of deep generative models in motion generation/prediction tasks. The methods are also generalizable to time-series prediction.
Probabilistic Graphical Models for Behavior Recognition and Prediction
Probabilistic graphical models (PGM) are important approaches for Bayesian inference, which are probabilistic models for which a graph expresses the conditional dependence structure between random variables. Generally, probabilistic graphical models use a graph-based representation as the foundation for encoding a distribution over a multi-dimensional space and a graph that is a compact or factorized representation of a set of independences that hold in the specific distribution.
The research aims at exploring how to utilize the power of probabilistic graphical models in behavior recognition and motion generation tasks. The proposed methods are also generalizable to time-series modeling with an emphasis on probabilistic inference.
Scene Understanding and Behavior / Trajectory Prediction
Effective understanding of the environment and accurate trajectory prediction of surrounding dynamic obstacles are critical for intelligent autonomous systems to achieve safe and high-quality decision making, motion planning and control.
The research aims at recognizing driver behaviors and traffic situations as well as predicting future trajectories for multiple highly interactive agents jointly.
Constrained Mixture Sequential Monte Carlo and Its Application to Multi-Object Tracking
The research aims at realizing accurate and robust tracking of multivariate dynamic systems with a focus on multi-agent autonomous systems. The proposed method can handle multi-modality of system state as well as missing observations (e.g. sensor occlusion in autonomous driving). The tracking framework can incorporate arbitrary learning-based system transition models.
* co-first author
- Lead organizer of Workshop at International Conference on Computer Vision (ICCV), 2021
- Co-organizers of Workshops at IEEE Intelligent Vehicles Symposium, 2019, 2020, 2021
- Co-organizer of Workshop at IEEE Intelligent Transporation Systems Conference, 2021
- Associate Editor at IEEE Intelligent Vehicles Symposium, 2020, 2021
- Program Committee of Workshop on AI for Autonomous Driving at IJCAI, 2021
- Signal Processing, 2021 -- Present
- IEEE Transactions on Pattern Analysis and Machine Intelligence (T-PAMI), 2020 -- Present
- IEEE Transactions on Robotics (T-RO), 2020 -- Present
- Neural Computation and Applications (NCAA), 2020 -- Present
- IEEE Transactions on Mechatronics (T-MECH), 2019 -- Present
- IEEE Transactions on Intelligent Vehicles (T-IV), 2018 -- Present
- IEEE Transactions on Intelligent Transportation Systems (T-ITS), 2017 -- Present
- IEEE Transactions on Industrial Electronics (T-IE), 2017 -- Present
- International Conference on Computer Vision (ICCV), 2021
- Adcances in Neural Information Processing Systems (NeurIPS), 2020
- International Conference on Machine Learning (ICML), 2020, 2021
- IEEE Intelligent Vehicles Symposium (IV), 2019, 2020, 2021
- IEEE Intelligent Transportation Systems Conference (ITSC), 2019, 2020
- IEEE International Conference on Robotics and Automation (ICRA), 2019, 2020, 2021
- IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2019, 2020, 2021
- Undergraduate students, 2019 -- Present
- Master students, 2020 -- Present