Jiachen Li

Ph.D. Candidate at University of California, Berkeley

About Me

Ph.D. Candidate & Graduate Student Researcher

I am currently a fifth year Ph.D. candidate in Robotics at UC Berkeley. My advisor is Prof. Masayoshi Tomizuka at Mechanical Systems Control (MSC) Laboratory at UC Berkeley. I have also been doing research on multiple projects at Berkeley DeepDrive (BDD) Laboratory. Prior to UC Berkeley, I received the B.Eng. degree in Automation from Harbin Institute of Technology (HIT), Harbin, China, in 2016. While at HIT, I was an undergraduate student researcher in Research Institute of Intelligent Control and Systems.

  • Education: UC Berkeley
  • Title: Graduate Student Researcher
  • E-mail: jiachen_li@berkeley.edu
  • Office: 2103 Etcheverry Hall, Berkeley


Research Interest

My research interest lies at the intersection of machine learning, graph neural network, computer vision and optimization approaches and their applications to relational reasoning, behavior prediction, decision making and motion planning for multi-agent intelligent systems (e.g. autonomous vehicles, robots).

  • Graph Neural Network
  • Relational Reasoning
  • Motion Prediction
  • Decision Making
  • Scene Understanding
  • Multi-Target Tracking

I am very open to research discussion and collaboration, please feel free to get in touch!


Curriculum Vitae

Click here to download my full CV.


Ph.D. Candidate, Robotics

Aug 2016 -- Present

University of California, Berkeley, CA, USA

Advisior: Prof. Masayoshi Tomizuka

Specialization: machine learning, graph neural network, prediction, tracking, decision making

B.Eng., Automation

Aug 2012 -- Jul 2016

Harbin Institute of Technology, Harbin, China

Advisors: Prof. Huijun Gao and Prof. Shen Yin

Thesis: Partial Least Squares and Its Application to Process Control

Work Experience

Research Intern

Sep 2019 - Present

Honda Research Institute USA, San Jose, CA, USA

  • Working on machine learning and computer vision algorithms for scene understanding and human behavior modeling
  • Proposed and developed a generic multi-agent prediction framework with dynamic relational reasoning
  • Proposed and developed a generic motion forecasting framework with dynamic key information selection
  • Proposed and developed a interaction-aware decision making framework for autonomous driving

Research Intern

Jun 2019 - Aug 2019

Toyota Research Institute, Los Altos, CA, USA

  • Proposed and developed a sample efficient, hybrid prediction system with graph neural network


The ultimate goal of my research is to build intelligent autonomous robots (vehicles) that can think rationally, behave like human beings and interact with the world intelligently. The methodologies mainly cover Bayesian theories, probabilistic graphical models, deep learning, graph neural networks, (inverse) reinforcement learning as well as computer vision techniques.

Selected Projects

  • Dynamic Relational Reasoning for Multi-Agent Systems, 2019--Present.
  • Interaction-Aware Decision Making for Multi-Agent Systems, 2019--Present.
  • General Motion Forecasting with Dynamic Key Information Selection, 2020--Present.
  • Scene Understanding and Human Behavior Modeling / Analysis/ Prediction, 2018--Present.
  • Interpretable and Data-Efficient Driving Behavior Generation via Deep Generative Probabilistic and Logic Models, 2018--2019.
  • Multi-Object Tracking with Learning Based Sequential Monte Carlo Methods, 2017--2019.
  • Generic Motion Generation and Comprehension with Social Interactions, 2017--2018.
  • Motion Prediction for Urban Autonomous Driving Based on Stochastic Policy Learned via Deep Neural Network, 2016--2017.

Dynamic Relational Reasoning for Multi-Agent Systems

Multi-agent interacting systems are prevalent in the world, from pure physical systems to complicated social dynamic systems. A key challenge in developing artificial intelligence systems with the flexibility and efficiency of human cognition is giving them a similar ability - to reason about entities and their relations from data.

The research aims at taking a step forward towards explicit relational reasoning by inferring a latent interaction graph with graph neural networks.

Related Publications:

Interaction-Aware Decision Making for Multi-Agent Systems

Deep reinforcement learning (DRL) provides a promising way for learning navigation in complex and interactive scenarios. However, identifying the subtle cues that can indicate drastically different outcomes remains an open problem with designing autonomous systems that operate in human environments. In addition, multi-agent interaction modeling is another open challenge.

The research aims at explicitly inferring the latent state of interactive agents and encoding spatial-temporal relationships in a reinforcement learning framework for the sequential decision making task in multi-agent systems.

Related Publications:

Motion Forecasting with Dynamic Key Information Selection

Motion forecasting has been widely studied in various domains, such as physical systems, human skeletons, and multi-agent interacting systems (e.g. traffic participants, sports players). The problem is formulated as to predict future state sequences based on the historical spatio-temporal observations. However, the observed information may be of different levels of significance and in some situations not all the information is relevant for the forecasting. Moreover, the key information may be varying as the situation evolves.

The research aims at developing a general motion forecasting framework with dynamic key information selection to address above issues.

Related Publications:

Deep Generative Models for Motion Generation and Prediction

The objective of generative models is to approximate the true data distribution, with which one can generate new samples similar to real data points with a proper variance. Generative models have been widely employed in tasks of representation learning and distribution approximation in literature, which basically fall into two categories: explicit density models and implicit density models. In recent years, since deep neural networks have been leveraged as universal distribution approximators thanks to its high flexibility, two deep generative models have been widely studied: Variational Auto-Encoder (VAE) and Generative Adversarial Network (GAN).

The research aims at exploring how to utilize the power of deep generative models in motion generation/prediction tasks. The methods are also generalizable to time-series prediction.

Related Publications:

Probabilistic Graphical Models for Behavior Recognition and Prediction

Probabilistic graphical models (PGM) are important approaches for Bayesian inference, which are probabilistic models for which a graph expresses the conditional dependence structure between random variables. Generally, probabilistic graphical models use a graph-based representation as the foundation for encoding a distribution over a multi-dimensional space and a graph that is a compact or factorized representation of a set of independences that hold in the specific distribution.

The research aims at exploring how to utilize the power of probabilistic graphical models in behavior recognition and motion generation tasks. The proposed methods are also generalizable to time-series modeling with an emphasis on probabilistic inference.

Related Publications:

Scene Understanding and Behavior / Trajectory Prediction

Effective understanding of the environment and accurate trajectory prediction of surrounding dynamic obstacles are critical for intelligent autonomous systems to achieve safe and high-quality decision making, motion planning and control.

The research aims at recognizing driver behaviors and traffic situations as well as predicting future trajectories for multiple highly interactive agents jointly.

Related Publications:

Constrained Mixture Sequential Monte Carlo and Its Application to Multi-Object Tracking

The research aims at realizing accurate and robust tracking of multivariate dynamic systems with a focus on multi-agent autonomous systems. The proposed method can handle multi-modality of system state as well as missing observations (e.g. sensor occlusion in autonomous driving). The tracking framework can incorporate arbitrary learning-based system transition models.

Related Publications:




EvolveGraph: Multi-Agent Trajectory Prediction with Dynamic Relational Reasoning
In Proceedings of the Neural Information Processing Systems (NeurIPS) 2020.
J. Li*, F. Yang*, M. Tomizuka, and C. Choi RAIN: Reinforced Hybrid Attention Inference Network for Motion Forecasting
arXiv preprint arXiv: TBD
J. Li, F. Yang, H. Ma, S. Malla, M. Tomizuka, and C. Choi Shared Cross-Modal Trajectory Prediction for Autonomous Driving
arXiv preprint arXiv: 2011.08436
C. Choi, J. H. Choi, J. Li, and S. Malla Reinforcement Learning for Autonomous Driving with Latent State Inference and Spatial-Temporal Relationships
arXiv preprint arXiv: 2011.04251
X. Ma, J. Li, MJ. Kochenderfer, D. Isele, and K. Fujimura Spectral Temporal Graph Neural Network for Trajectory Prediction
arXiv preprint arXiv: TBD
D. Cao*, J. Li*, H. Ma, and M. Tomizuka Orientation-Aware Planning for Parallel Task Execution of Omni-Directional Mobile Robot
arXiv preprint arXiv: TBD
C. Gong, X. Zhou, Z. Li, J. Li, J. Gong, and J. Zho CIA-TP: Continual Interaction-Aware Trajectory Prediction across Different Scenarios
arXiv preprint arXiv: TBD
H. Ma*, Y. Sun*, J. Li, L. Rong, and M. Tomizuka Social-WaGDAT: Interaction-Aware Trajectory Prediction via Wasserstein Graph Double-Attention Network
arXiv preprint arXiv: 2002.06241
J. Li, H. Ma, Z. Zhang, and M. Tomizuka



Generic Tracking and Probabilistic Prediction Framework and Its Application in Autonomous Driving
IEEE Transactions on Intelligent Transportation Systems
J. Li, W. Zhan, Y. Hu, and M. Tomizuka Interaction-aware Multi-agent Tracking and Probabilistic Behavior Prediction via Adversarial Learning
2019 IEEE International Conference on Robotics and Automation (ICRA)
J. Li*, H. Ma*, and M. Tomizuka Conditional Generative Neural System for Probabilistic Trajectory Prediction
2019 IEEE/RSJ International Conference on Robotics and Systems (IROS).
J. Li, H. Ma, and M. Tomizuka Coordination and Trajectory Prediction for Vehicle Interactions via Bayesian Generative Modeling
2019 IEEE Intelligent Vehicles Symposium (IV)
J. Li, H. Ma, W. Zhan, and M. Tomizuka Wasserstein Generative Learning with Kinematic Constraints for Probabilistic Interactive Driving Behavior Prediction
2019 IEEE Intelligent Vehicles Symposium (IV)
H. Ma, J. Li, W. Zhan, and M. Tomizuka



Generic Probabilistic Interactive Situation Recognition and Prediction: From Virtual to Real
2018 IEEE Intelligent Transportation Systems Conference (ITSC)
J. Li, H. Ma, W. Zhan, and M. Tomizuka Towards a Fatality-Aware Benchmark of Probabilistic Reaction Prediction in Highly Interactive Driving Scenarios
2018 IEEE Intelligent Transportation Systems Conference (ITSC)
W. Zhan, L. Sun, Y. Hu, J. Li, and M. Tomizuka Generic Vehicle Tracking Framework Capable of Handling Occlusions Based on Modified Mixture Particle Filter
2018 IEEE Intelligent Vehicles Symposium (IV) (Oral)
J. Li, W. Zhan, and M. Tomizuka



Safe and Feasible Motion Generation for Autonomous Driving via Constrained Policy Net
2017 Annual Conference of the Industrial Electronics Society (IECON)
W. Zhan, J. Li, Y. Hu, and M. Tomizuka



A Novel Variable Selection Approach for Redundant Information Elimination Purpose of Process Control
IEEE Transactions on Industrial Electronics
J. Li, C. Duan, and Z. Fei Finite-time H∞ Control of Switched Systems with Mode-dependent Average Dwell Time
Journal of the Franklin Institute
S. Shi, Z. Fei, and J. Li A Variable Selection Aided Residual Generator Design Approach for Process Control and Monitoring
C. Duan, Z. Fei, and J. Li


* co-first author

Academic Services

Program Committee

  • Co-organizers of Workshops at IEEE Intelligent Vehicles Symposium, 2019, 2020
  • Associate Editor at IEEE Intelligent Vehicles Symposium, 2020, 2021

Journal Reviewer

  • IEEE Transactions on Pattern Analysis and Machine Intelligence (T-PAMI), 2020 -- Present
  • IEEE Transactions on Intelligent Transportation Systems (T-ITS), 2017 -- Present
  • IEEE Transactions on Intelligent Vehicles (T-IV), 2018 -- Present
  • IEEE Transactions on Industrial Electronics (T-IE), 2017 -- Present
  • IEEE Transactions on Robotics (T-RO), 2020 -- Present
  • IEEE Transactions on Mechatronics (T-MECH), 2019 -- Present
  • Neural Computation and Applications (NCAA), 2020 -- Present

Conference Reviewer

  • Adcances in Neural Information Processing Systems (NeurIPS), 2020
  • International Conference on Machine Learning (ICML), 2020, 2021
  • IEEE Intelligent Vehicles Symposium (IV), 2019, 2020
  • IEEE Intelligent Transportation Systems Conference (ITSC), 2019, 2020
  • IEEE International Conference on Robotics and Automation (ICRA), 2019, 2020, 2021
  • IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2019, 2020

Research Mentor

  • Undergraduate students, 2019 -- Present
  • Master students, 2020 -- Present

Coursework (Ph.D.)

  • Mathematical Methods in Engineering
  • Advanced Control Systems
  • Nonlinear Systems
  • Introduction to Artificial Intelligence
  • Introduction to Machine Learning
  • Deep Reinforcement Learning
  • Optimization Models and Applications
  • Convex Optimization
  • Mathematical Programming
  • Learning and Optimization
  • Computer Vision
  • Designing, Visualizing and Understanding Deep Neural Networks
  • Teaching at the University Level