K-pop: HYBE seeks to eliminate language barrier with AI – 07/19/2023 – Tech

In a dimly lit recording studio in Seoul, producers at the K-pop label that brought the world group BTS to the world are using artificial intelligence (AI) to fuse the voice of a South Korean singer with that of native speakers of five other countries. languages.

The technology allowed HYBE, South Korea’s biggest music label, to release a track by singer MIDNATT in six languages ​​— Korean, English, Spanish, Chinese, Japanese and Vietnamese — in May.

Some K-pop singers have already made songs in English and Japanese, as well as Korean, but this is the first time that the new technology has been applied for a simultaneous release in six languages, according to HYBE, and the move could pave the way. for the technology to be used by more popular artists.

“First we will listen to the reaction, the voice of the fans, and then we will decide what the next steps will be,” said Chung Wooyong, head of HYBE’s interactive media division, in an interview at the label’s studio.

Lee Hyun, 40, known artistically as MIDNATT, speaks only a little English and Chinese in addition to Korean, but he recorded the song “Masquerade” in each of the languages.

To do this, native speakers read the lyrics, and then their voices were combined with the singer’s with the help of HYBE’s in-house artificial intelligence music technology, Chung said.

The song is the latest sign of AI’s growing influence in the music industry, at a time when the Grammy Awards have introduced new rules for using the technology, while AI-generated song mash-ups are flooding social media. .

“We split a sound fragment into different components — pronunciation, timbre, pitch and volume,” Chung said. “We looked at pronunciation, which is associated with movement of the tongue, and used our imagination to see what kind of results we could get using our technology.”

In a before-and-after comparison shown to Reuters, a lengthened vowel sound was added to the word “twisted” in the song’s English lyrics, for example, to make it sound more natural, while no detectable changes were made to the singer’s voice.

MIDNATT said the use of artificial intelligence allowed for a “broader spectrum of artistic expression”.

“I feel the language barrier has been eliminated and it is much easier for global fans to have an immersive experience with my music,” he said in a statement.

While the technology isn’t new, it’s an innovative way to use artificial intelligence in music, said Valerio Velardo, director of The Sound of AI, a Spain-based consulting service for AI music and audio.

“It will lower the bar for creating music. It’s a bit like Instagram for photos, but in the case of music.”

For now, HYBE’s pronunciation correction technology takes “weeks or months” to do the job, but when the process is accelerated, it could serve a wider range of purposes, such as interpreting in video conferences, said Choi Jin-woo, producer from MIDNATT’s “Masquerade”, who goes by the name Hitchhiker.