What is FILM?

One of many challenges of body interpolation is dealing with complicated movement and occlusions within the scene. Conventional strategies typically produce artifacts similar to blurring, ghosting, or flickering. To beat these limitations, Google Analysis developed a novel method based mostly on deep studying and optical movement. The tactic consists of two steps: first, the optical movement community estimates the movement vector between two enter frames; second, the pixel synthesis community generates intermediate frames by warping and mixing the enter frames based mostly on the movement vectors. Optical movement networks are educated utilizing self-supervised losses and don’t require floor reality movement labels. The pixel synthesis community is educated with a perceptual loss to supply life like and clear outcomes. This methodology can deal with giant movement and complicated occlusions higher than earlier strategies. It additionally produces high-quality outcomes for difficult situations similar to water splash, fireplace, smoke and hair. The tactic is quick and environment friendly, working at 30 frames per second on a single GPU. Google Analysis has made the code and fashions obtainable on Replicate.com, a platform that permits anybody to run and reproduce machine studying experiments. Yow will discover the mission web page right here: https://replicate.com/google-research/frame-interpolation. You can too watch a video demonstration right here: https://www.youtube.com/watch?v=7oQn9yXxOaU. If you’re enthusiastic about studying extra about body interpolation and the way Google Analysis is advancing the cutting-edge in video processing, you’ll be able to learn their paper right here: https://arxiv.org/abs/2009.05113. You can too comply with their weblog and Twitter for extra updates and information.