Predicting multi-person dynamics

Abstract

Humans unconsciously model the dynamics of the world around them; for example, we predict the movement of surrounding traffic and pedestrians while driving, or forecast player positions in a game of soccer. Our work builds towards enabling computers with a facet of this ability. Given a video and corresponding bounding box tracks, we propose various methods to predict the future shape, pose, and position of people in unseen frames. Other works that also tackle video-based mesh prediction of humans focus on predicting the shape and pose, ignoring the position of the person in the scene. Additionally, they focus on predicting the future states of each individual in isolation, neglecting how interactions between individuals in a scene can inform their future actions. We present methods to address both of these limitations, and when evaluated on the Human3.6M and 3DPW datasets, we show favorable results to inform future directions of research.