DeepPhase: Periodic Autoencoders for Learning Motion Phases Manifolds

Principal Investigator: Professor Taku KOMURA (Professor of Computer Graphics from Department of Computer Science)

This project is showcased in the third exhibition – Technology for Future.

About the scholar

Professor Taku KOMURA

Professor Taku Komura is a professor of Computer Graphics in The University of Hong Kong. Before joining HKU in 2020, he worked at the University of Edinburgh (2006-2020), City University of Hong Kong (2002-2006) and RIKEN (2000-2002). He received his BSc,
MSc and PhD in Information Science from University of Tokyo. His research has focused on data-driven character animation, physically-based character animation, crowd simulation, 3D modelling, cloth animation, anatomy-based modelling and robotics. Recently, his main research interests have been on physically-based animation and the application of machine learning techniques for animation synthesis. He received the Royal Society Industry Fellowship (2014) and the Google AR/VR Research Award (2017).

Email: taku@cs.hku.hk
Website: https://www.cs.hku.hk/index.php/people/academic-staff/taku

Project information
Professor Komura’s research focuses on creating diverse human movements for computer animation, games, and virtual environments. His research team designed a new type of neural network called the Periodic Autoencoder that can identify repeating patterns in large sets of motion data without any additional information. This allows us to create unique movements in various styles, such as dance motions synchronized to music or dribbling movements in soccer. Additionally, the system can help find similar motions in large database, produce natural movements between a small number of key frame poses and estimate human movements even when the body is partially obscured in videos.
Football movements generated by Periodic Autoencoder.
Different type of dancing movements generated by Periodic Autoencoder.
Network architecture of the Periodic Autoencoder for extracting multi-dimensional phase manifolds from unstructured motion data.
Project video
Project images
3D Principal Component Analysis (PCA) embedding for periodic biped walking (top), stylized walking with waving arms (middle), and complex dance motion (bottom).
Generating movements by Interpolating keyframes using the phase manifold
Periodic Autoencoder can be applied for estimating human poses from the video even when the body is largely hidden.
Other projects
Exhibition Mainpage