Ph.D. Candidate & Graduate Student Researcher
I am currently a fifth year Ph.D. candidate in Robotics at UC Berkeley. My advisor is Prof. Masayoshi Tomizuka at Mechanical Systems Control (MSC) Laboratory at UC Berkeley. I have also been doing research on multiple projects at Berkeley DeepDrive (BDD) Laboratory. Prior to UC Berkeley, I received the B.Eng. degree in Automation from Harbin Institute of Technology (HIT), Harbin, China, in 2016. While at HIT, I was an undergraduate student researcher in Research Institute of Intelligent Control and Systems.
My research interest lies at the intersection of machine learning, graph neural network, computer vision and optimization approaches and their applications to relational reasoning, behavior prediction, decision making and motion planning for multi-agent intelligent systems (e.g. autonomous vehicles, robots).
- Graph Neural Network
- Relational Reasoning
- Motion Prediction
- Decision Making
- Scene Understanding
- Multi-Target Tracking
I am very open to research discussion and collaboration, please feel free to get in touch!
- 09/2020: One paper regarding multi-agent trajectory prediction and relational reasoning is accepted at NeurIPS 2020!
- 09/2020: I am honored to be recognized as a Top Reviewer for ICML 2020!
- 06/2020: I am co-organizing IEEE IV 2020 Workshop on Behavior Prediction and Generation for Autonomous Driving!
- 07/2019: One paper regarding a generic tracking and prediction framework is accepted at IEEE Transactions on Intelligent Transportation Systems!
- 06/2019: I am co-organizing IEEE IV 2019 Workshop on Prediction and Decision Making for Autonomous Driving!
- 05/2019: One paper regarding deep generative models for behavior prediction is accepted at IROS 2019!
- 01/2019: One paper regarding generative adversarial networks is accepted at ICRA 2019!
Click here to download my full CV.
Ph.D. Candidate, Robotics
Aug 2016 -- Present
University of California, Berkeley, CA, USA
Advisior: Prof. Masayoshi Tomizuka
Specialization: machine learning, graph neural network, prediction, tracking, decision making
Aug 2012 -- Jul 2016
Harbin Institute of Technology, Harbin, China
Advisors: Prof. Huijun Gao and Prof. Shen Yin
Thesis: Partial Least Squares and Its Application to Process Control
Sep 2019 - Present
Honda Research Institute USA, San Jose, CA, USA
- Working on machine learning and computer vision algorithms for scene understanding and human behavior modeling
- Proposed and developed a generic multi-agent prediction framework with dynamic relational reasoning
- Proposed and developed a generic motion forecasting framework with dynamic key information selection
- Proposed and developed a interaction-aware decision making framework for autonomous driving
Jun 2019 - Aug 2019
Toyota Research Institute, Los Altos, CA, USA
- Proposed and developed a sample efficient, hybrid prediction system with graph neural network
The ultimate goal of my research is to build intelligent autonomous robots (vehicles) that can think rationally, behave like human beings and interact with the world intelligently. The methodologies mainly cover Bayesian theories, probabilistic graphical models, deep learning, graph neural networks, (inverse) reinforcement learning as well as computer vision techniques.
- Relational Reasoning with Graph Neural Networks, 2019--Present.
- Interaction-Aware Decision Making for Multi-Agent Systems, 2019--Present.
- General Motion Forecasting with Dynamic Key Information Selection, 2020--Present.
- Continual and Lifelong Learning for Motion and Trajectory Prediction, 2020--Present.
- Scene Understanding and Human Behavior Modeling / Analysis/ Prediction, 2018--Present.
- Interpretable and Data-Efficient Driving Behavior Generation via Deep Generative Probabilistic and Logic Models, 2018--2019.
- Generic Motion Generation and Comprehension with Social Interactions, 2017--2018.
- Motion Prediction for Urban Autonomous Driving Based on Stochastic Policy Learned via Deep Neural Network, 2016--2017.
Relational Reasoning with Graph Neural Networks
Multi-agent interacting systems are prevalent in the world, from pure physical systems to complicated social dynamic systems. A key challenge in developing artificial intelligence systems with the flexibility and efficiency of human cognition is giving them a similar ability - to reason about entities and their relations from data.
The research aims at taking a step forward towards explicit relational reasoning by inferring a latent interaction graph with graph neural networks.
Interaction-Aware Decision Making for Multi-Agent Systems
Deep reinforcement learning (DRL) provides a promising way for learning navigation in complex and interactive scenarios. However, identifying the subtle cues that can indicate drastically different outcomes remains an open problem with designing autonomous systems that operate in human environments. In addition, multi-agent interaction modeling is another open challenge.
The research aims at explicitly inferring the latent state of interactive agents and encoding spatial-temporal relationships in a reinforcement learning framework for the sequential decision making task in multi-agent systems.
Motion Forecasting with Dynamic Key Information Selection
Motion forecasting has been widely studied in various domains, such as physical systems, human skeletons, and multi-agent interacting systems (e.g. traffic participants, sports players). The problem is formulated as to predict future state sequences based on the historical spatio-temporal observations. However, the observed information may be of different levels of significance and in some situations not all the information is relevant for the forecasting. Moreover, the key information may be varying as the situation evolves.
The research aims at developing a general motion forecasting framework with dynamic key information selection to address above issues.
Continual/Lifelong Learning for Motion and Trajectory Prediction
Multi-agent behavior prediction is essential in many real-world applications, such as autonomous driving and mobile robot navigation. Many recent works have been done to solve this problem and the prediction performance is becoming better and better in terms of accuracy. However, how to train a predictor with the incremental dataset in different scenarios remains a largely unexplored question.
The research aims at exploring a good and generalizable representation for different scenarios and employing continual learning techniques to enable learning with incremental data efficiently.
Deep Generative Models for Motion Generation and Prediction
The objective of generative models is to approximate the true data distribution, with which one can generate new samples similar to real data points with a proper variance. Generative models have been widely employed in tasks of representation learning and distribution approximation in literature, which basically fall into two categories: explicit density models and implicit density models. In recent years, since deep neural networks have been leveraged as universal distribution approximators thanks to its high flexibility, two deep generative models have been widely studied: Variational Auto-Encoder (VAE) and Generative Adversarial Network (GAN).
The research aims at exploring how to utilize the power of deep generative models in motion generation/prediction tasks. The methods are also generalizable to time-series prediction.
Probabilistic Graphical Models for Behavior Recognition and Prediction
Probabilistic graphical models (PGM) are important approaches for Bayesian inference, which are probabilistic models for which a graph expresses the conditional dependence structure between random variables. Generally, probabilistic graphical models use a graph-based representation as the foundation for encoding a distribution over a multi-dimensional space and a graph that is a compact or factorized representation of a set of independences that hold in the specific distribution.
The research aims at exploring how to utilize the power of probabilistic graphical models in behavior recognition and motion generation tasks. The proposed methods are also generalizable to time-series modeling with an emphasis on probabilistic inference.
Scene Understanding and Behavior / Trajectory Prediction
Effective understanding of the environment and accurate trajectory prediction of surrounding dynamic obstacles are critical for intelligent autonomous systems to achieve safe and high-quality decision making, motion planning and control.
The research aims at recognizing driver behaviors and traffic situations as well as predicting future trajectories for multiple highly interactive agents jointly.
Constrained Mixture Sequential Monte Carlo and Its Application to Multi-Object Tracking
The research aims at realizing accurate and robust tracking of multivariate dynamic systems with a focus on multi-agent autonomous systems. The proposed method can handle multi-modality of system state as well as missing observations (e.g. sensor occlusion in autonomous driving). The tracking framework can incorporate arbitrary learning-based system transition models.
* co-first author
- Co-organizers of Workshops at IEEE Intelligent Vehicles Symposium, 2019, 2020
- Associate Editor at IEEE Intelligent Vehicles Symposium, 2020, 2021
- IEEE Transactions on Pattern Analysis and Machine Intelligence (T-PAMI), 2020 -- Present
- IEEE Transactions on Intelligent Transportation Systems (T-ITS), 2017 -- Present
- IEEE Transactions on Intelligent Vehicles (T-IV), 2018 -- Present
- IEEE Transactions on Industrial Electronics (T-IE), 2017 -- Present
- IEEE Transactions on Robotics (T-RO), 2020 -- Present
- IEEE Transactions on Mechatronics (T-MECH), 2019 -- Present
- Neural Computation and Applications (NCAA), 2020 -- Present
- Adcances in Neural Information Processing Systems (NeurIPS), 2020
- International Conference on Machine Learning (ICML), 2020
- IEEE Intelligent Vehicles Symposium (IV), 2019, 2020
- IEEE Intelligent Transportation Systems Conference (ITSC), 2019, 2020
- IEEE International Conference on Robotics and Automation (ICRA), 2019, 2020, 2021
- IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2019, 2020
- Undergraduate students, 2019 -- Present
- Master students, 2020 -- Present
Coursework (Ph. D.)
- Mathematical Methods in Engineering
- Advanced Control Systems
- Nonlinear Systems
- Introduction to Artificial Intelligence
- Introduction to Machine Learning
- Deep Reinforcement Learning
- Optimization Models and Applications
- Convex Optimization
- Mathematical Programming
- Learning and Optimization
- Computer Vision
- Designing, Visualizing and Understanding Deep Neural Networks
- Teaching at the University Level