Don’t be afraid of the ‘AI-assisted’ Beatles song, ‘Now And Then’

There’s been a bit of FUD around the decision to use a form of AI to resurrect John Lennon’s voice in what Paul McCartney called “the last Beatles record.” What they’ve done is far from the sketchy AI imitations of artists we see cluttering Soundcloud today, and has much more in common with a more prosaic application of machine learning: noise reduction.

To hear people talk about it, you’d think this was an abject money-grab using the latest voice synthesis tech to ape one of the most famous songwriters of all time. But the real story is more simple and poignant than that, and the technology is far less fantastic.

As the members of the band recall in a sweet short film about the making of the song, “Now And Then” was originally a piano demo Lennon made shortly before he was killed in 1980. His widow, Yoko Ono, provided the tape on which it was recorded to the band afterwards, but the quality of the recording was not great — bad, even.

“When we listened to ‘Now And Then,’ it was very difficult because John was sort of hidden in a way,” says Ringo Starr in the making-of short.

“Every time I wanted a little more of John’s voice,” recalled Paul. “This piano came through and clouded the picture. And in those days, of course, we didn’t have the technology to do the separation.”

They “ran out of steam” in 1995 when they tried to rescue the song, but in 2022 they were working with Peter Jackson on the documentary Get Back. The filmmaker and his team were applying modern audio processing technology to archival footage of the band to isolate individual instruments and voices.

“We were paying a lot of attention to the technical restoration. That ultimately led us to develop a technology which allows us to take any soundtrack and split all the different components into separate tracks based on machine learning,” said Jackson in the short.

[youtube https://www.youtube.com/watch?v=APJAQoSCwuA?version=3&rel=1&showsearch=0&showinfo=1&iv_load_policy=1&fs=1&hl=en-US&autohide=2&wmode=transparent&w=640&h=360]

MAL, as they called it, is a version of audio isolation technologies that have come a long way in the last few years. Machine learning models can be trained on, say, many guitar tracks and learn what the waveform or spectral signature of a guitar is, and can with varying success pluck it right out of a mixed track.

It’s commonly used in video calls now as well, using models trained on human voices. By suppressing everything that isn’t the speaker’s voice, background noise like barking dogs or a loud cafe can be silenced in real time. Cruder versions of this were sometimes used to make karaoke versions of songs, identifying and removing the vocal tracks.

In the case of Lennon’s demo, it worked like a charm, as you can hear at this timestamp in the making-of short.

“There it was — John’s voice, crystal clear,” said Paul. “Now we could mix it and make a proper record of it.”

Some may question the ethics of making that record, but everyone involved seems to think John would have been all for it, as he loved tinkering with technology and had, of course, written and performed the song originally with the intention of recording it.

But more importantly, it seems to have acted as a bit of closure for the group. The vicissitudes of stardom and creativity they endured are more than adequately documented, but to lose a friend and creative partner of decades that way, and to have this last, lingering loose end dangling just out of reach must have been torturous.

As anyone who has lost someone can attest, every vestige of them becomes precious. “To hear John’s voice… that’s a thing we should cherish,” George had said back in 1995.

And now with a quarter century’s worth of technological improvements brought to bear, that’s exactly what they could do.

“It was the closest we’ll ever come to having him back in the room,” said Ringo.

You can listen to “Now And Then” right here.

[youtube https://www.youtube.com/watch?v=AW55J2zE3N4?version=3&rel=1&showsearch=0&showinfo=1&iv_load_policy=1&fs=1&hl=en-US&autohide=2&wmode=transparent&w=640&h=360]