Image credit: Pawel Czerwinski

Google’s new tool is taking on the world’s film and TV composers by automatically producing the perfect score to visuals.

Google’s DeepMind lab are back with another revolutionary AI tool. They have unveiled the V2A (video-to-audio), a groundbreaking technology that automatically syncs audio to videos.

V2A is capable of analysing video content to develop an idea of the theme and feelings it provides. Google’s AI tool can then generate a soundtrack which it synchronises to the video.

DeepMind wrote in a blog post: “Video generation models are advancing at an incredible pace, but many current systems can only generate silent output. One of the next major steps toward bringing generated movies to life is creating soundtracks for these silent videos.”

Watch an example of a horror score generated to match a creepy video below:

This has huge potential for creators. Creators may soon be able to use this tool to create completely unique soundtracks to their content that matches it to a tee. It also opens up the potential to explore the vice-versa, in which AI can generate video automatically based on audio. This would revolutionise the potential for musicians to create ambitious and creative music videos regardless of budget or equipment.

Google’s V2A tool also purports to generate vocal narratives that are automatically generated over video. This could enhance educational content by allowing educators to create engaging and interesting videos with a simple workflow for adding a narrative on top.

V2A works not simply by generating music. The deep learning of the AI allows it to understand the video’s content and create a soundscape that fits it. They hope to empower creators with easier content creation, making it easier to create a whole product.

However, Google acknowledge the importance of protecting creators in responsible AI development. They’re integrating their SynthID tool to mark content generated with V2A to ensure transparency.

DeepMind wrote: “To make sure our V2A technology can have a positive impact on the creative community, we’re gathering diverse perspectives and insights from leading creators and filmmakers, and using this valuable feedback to inform our ongoing research and development.”

They are testing the tool before bringing it to the public.