DeepMind helps create animated movie headed for Sundance

In what is likely the first AI movie at Sundance, «Dear Upstairs Neighbors» was made with the creatives first. (Picture: Google)
The animated movie «Dear Upstairs Neighbors» was created by Pixar alumni Connie He, a group of animation veterans — and a team of researchers at DeepMind.

The movie was an experiment to see how AI tools could better help moviemakers, and ended up needing specialized tools.

Connie He provided the storyboard, the character design, the abstract background paintings, but a 45-person team at DeepMind soon realized that the off-the-shelf-versions of their software wasn’t good enough for this purpose.

They then fine-tuned the Veo and Imagen models to their artwork, and made a video-to-video model where the artists could sketch a scene for guidance and have the model add details and artwork, before upscaling it to 4K using Veo.

For precise detail, they also developed a tool that would let the artists fix detail in specific regions of an animation, without influencing the broader scene.

The movie will debut at the Sundance Institute’s Story Forum, a panel that focuses on tools and technologies in support of storytelling.

Read more: Google’s blog details the process, Announcement thread.