Video content producers are often faced with video length requirements. One way to achieve the target duration is to speed up video scenes that exhibit a range of playback speeds that can be considered natural (like a car driving at a constant speed).
A recent paper published on arXiv.org describes a method to automatically find such temporary local video segments that can be sped up while maintaining realism in dynamic videos. It relies on a neural network trained through self-monitoring to recognize and localize artificially introduced changes in the playback rate of video frames.
First, slow predictions are calculated to identify frames that can speed up. The video is then optimized to reach the target length. Tests confirm that this method is more accurate at localizing speed changes, scales better with longer input videos, and is more computationally efficient than previous methods .
We propose a method to generate time-remapped videos that match the desired target duration while maintaining maximum natural video dynamics. Our approach trains the neural network through self-monitoring to accurately recognize and localize various transient changes in video playback speed. For video re-timing, we 1. use the model to infer the delay of each individual video frame, and 2. optimize the time frame subsampling to match the model’s slow prediction. We demonstrate that this model can detect playback rate variations more accurately while providing greater efficiency in magnitude than previous approaches. Furthermore, we suggest optimizing video reconstruction time allowing precise control over target duration and working more aggressively on longer videos than previous methods. We evaluate the model quantitatively on artificially accelerated videos, through transitions to action recognition, and qualitatively through user studies.
Research articles: Jenni, S., Woodson, M., and Caba Heilbron, F., “Video-ReTime: Learning Temporally Varying Speediness for Time Remapping”, 2022. Link: https://arxiv.org/abs/2205.05609
