|
|
|
|
|
Abstract
|
Multi-agent interacting systems are prevalent in the world, from pure physical systems to complicated social dynamic systems. In many applications, effective understanding of the situation and accurate trajectory prediction of interactive agents play a significant role in downstream tasks, such as decision making and planning. In this paper, we propose a generic trajectory forecasting framework (named EvolveGraph) with explicit relational structure recognition and prediction via latent interaction graphs among multiple heterogeneous, interactive agents. Considering the uncertainty of future behaviors, the model is designed to provide multi-modal prediction hypotheses. Since the underlying interactions may evolve even with abrupt changes, and different modalities of evolution may lead to different outcomes, we address the necessity of dynamic relational reasoning and adaptively evolving the interaction graphs. We also introduce a double-stage training pipeline which not only improves training efficiency and accelerates convergence, but also enhances model performance. The proposed framework is evaluated on both synthetic physics simulations and multiple real-world benchmark datasets in various areas. The experimental results illustrate that our approach achieves state-of-the-art performance in terms of prediction accuracy. |
||
| |
||
Key Ideas and Contributions
 Generic Prediction Framework: We propose a generic trajectory forecasting framework with explicit interaction modeling viaa latent graph among multiple heterogeneous, interactive agents. Both trajectory information and context information (e.g. scene images, semantic maps, point cloud density maps) canbe incorporated into the system. Dynamic Relational Reasoning: We propose a dynamic mechanism to evolve the underlying interaction graph adaptivelyalong time, which captures the dynamics of interaction patterns among multiple agents. We also introduce a double-stage training pipeline which not only improves training efficiencyand accelerates convergence, but also enhances model performance in terms of prediction accuracy. Uncertainty and Multi-Modality: The proposed framework is designed to capture the uncertainty and multi-modality of future trajectories in nature from multiple aspects. |
||
| |
||
Particle Physics System
 |
||
|
Visualization of latent interaction graph evolution and particle trajectories: (a) The top two figures show the probability of the first edge type ("with link") at each time step. Each row corresponds to a certain edge (shown in the right). The actual times of graph evolution are 54 and 62, respectively. The model is able to capture the underlying criterion of relation change and further predict the change of edge types with nearly no delay. (b) The figures in the last row show trajectory prediction results, where semi-transparent dots are historical observations. Please refer to the paper for more details. |
|
||
| Quantitative results on various benchmark datasets in different areas: In this paper, we validated the proposed framework EvolveGraph on three benchmark datasets for real-world applications: H3D dataset (traffic scenarios), NBA dataset (sports games), and SDD dataset (university campus). The experimental results demonstrate that EvolveGraph achieves state-of-the-art performance in terms of minADE and minFDE. Please refer to the paper for more details. |
||
|
||
| Qualitative results of testing cases on the H3D dataset: Dashed lines are historical trajectories, solid lines are ground truth, and dash-dotted lines are prediction hypothesis. White areas represent drivable areas and gray areas represent sidewalks. We plotted the prediction hypothesis with the minimal ADE, and the heatmap to represent the distributions. (a) Intersection; (b) Roundabout. We can tell that our framework can generate accurate and plausible trajectories. These results also show that the evolving interaction graph can reinforce the multi-modal property of our model, since different samples of trajectories at the previous steps lead to different directions of graph evolution, which significantly influences the prediction afterwards. Please refer to the paper for more details. |
||
|
||
| Qualitative results of testing cases on the NBA dataset: Dashed lines are historical trajectories, solid lines are ground truth, and dash-dotted lines are prediction hypothesis. We plotted the prediction hypothesis with the minimal ADE, and the heatmap to represent the distributions. We can tell that in these cases the ball follows a player at most times, which implies that the predicted results represent plausible situations. Most prediction hypotheses are very close to the groundtruth, even if some predictions are not similar to the ground truth, they represent a plausible behavior. Moreover, the heatmaps show that our model can successfully predict most reasonable future trajectories and their multi-modal distributions. Please refer to the paper for more details. |
||
 |
J. Li*, F. Yang*, M. Tomizuka, C. Choi. EvolveGraph: Multi-Agent Trajectory Prediction with Dynamic Relational Reasoning NeurIPS 2020 [Paper] [Video](coming soon) [Bibtex] |
Acknowledgements |