This incredible tool turns recordings into authentic saxophone, flute, violin and more
Ever wished you could add a powerful sax solo into your mix? With this amazing tool from Google you can transform anything from your voice to other instruments into authentic sounding recordings.
Are you ready to have your world of musical synthesis opened up? Prepare to discover the incredible potential of music in a world with AI. From the powerful minds in Google’s Magenta team, Tone Transfer is an experiment in transform any sound into a recording that is as though it had come from these iconic instruments themselves.
The team went about training their AI by making it listen to authentic audio from instruments. They gave it 10 minute recordings of instruments, for example a violin, and told it to understand how the instrument creates it sound by measuring its pitch and loudness.
They show on their website how the AI slowly wrapped its head around the unique sound of each instrument. After one hour of training the results were still quite clearly synthesised sounds. After three hours the unique tone began to come through and then after 10 hours of training the AI was able to replicate an incredibly accurate violin sound.
It is taught to understand what makes these instruments unique beyond simply the sound is creating. This is where the output of this machine-learned tool is different to a lot of synthesised versions of instruments. It picks up elements like the bow scratching and the breath coming through a brass instrument and integrates them into the overall sound – making for a truly authentic model.
With the powerful understanding of how an instrument works, you can then give it any sound recording and the AI will transform it into how it would sound coming out of an array of instruments. Want to know something great? You can do it yourself!
The Tone Transfer website allows you to play around with their stock samples including birds tweeting, pans crashing, and a cello playing. But much more fun, you can record your own audio into it through a microphone or from an instrument. Who doesn’t want to sing along to a melody and turn themselves into a saxophone piece!
One thing I have noticed, and this may be solely an issue for me, is the tool stops recognising pitch about 10 seconds into the recording and you can only record for a maximum of 15 seconds. As you can hear below, in the very melancholy outcome of my attempt at recreating Giant Steps, the pitch trails off onto one note at the end – but it is still just as impressive before that point.