Chinese Idol Girl Group SNH48 Creates Digital Clones Using AI

In the latest music video of a Chinese idol group debuted for Christmas, six members of the group SNH48 dance with their digital clones. These copies are based on their own vocals, looks, and body language. This video was co-produced with Tencent-backed AI start-up, called ObEN.

SNH48

This 4-minute music video is claimed to be the first commercially released song in the world to feature both artists and their 3D clones created by AI. In the video, artists SNH48 performed their solo parts and AI copies performed chorus segments. It is clear to see that the dance steps show the interaction between real singers and their AI copies.

SNH48's vice president, Xiong Wei, said this MV was the first step in developing the virtual idol market. Human idols have shortcomings, such as age and their personal lives. He said female idols will lose their youthful appeal after 8 years working.

These things will not happen to virtual idols. The Shanghai-based company also said it is planning to create smarter virtual idols in order to make movies and release albums.

The group SNH48 is like the Japanese AKB48 group, which consists of over 100 female members around the age of 20. They often perform at their own theater and meet with their fans through live events.

ObEN based in Pasadena, California, can create intelligent 3D representation images with images that are exactly the same as the human objects identified through computer vision. In addition, it can learn the pronunciation, singing, and behavior of the person to create an identical copy through speech technology and natural language processing.

Now fans are willing to pay for their idols' albums and products instead of using pirated albums as people used to do in the past. Therefore, the Chinese entertainment industry is developing and gaining a lot of profits. By 2020, according to Zhongtai Securities, China's pop economy will hit 100 billion yuan ($14.5 billion).

ObEN is not the first company to realize the potential of artificial intelligence in the entertainment industry. Before that, S.M. Entertainment (one of Korea's largest entertainment companies) also collaborated with AI Stars to combine artificial intellectual among the top K-pop idols with ObEN's AI technology. AI Stars claims that it is the first celebrity AI agency that uses this combination to give a new experience in interaction and entertainment products.

Last month, the Xinhua News Agency (owned by China’s state) introduced"composite anchors", which was created by Beijing-based Sogou using artificial intelligence technology combined with voices and images of actual anchors.

The "Now and Forever" music video 

The process of creating these copies is not easy, each singer had to record about 100 - 200 sentences so that their images and sounds were saved as data for AI algorithms. According to CEO and co-founder of ObEN, Adam Zheng, they managed to create a digital avatar capable of speaking and singing in accordance with the voice of each member. Moreover, the avatars have the ability to speak different languages by the singers' voices. A 3D body scan can create a more accurate copy of AI. However, SNH48 did not apply this method.

Zheng said the digital copy will open a bright future for the entertainment field. It could help the popular actors in the movie sing the theme song of some movie that they couldn't do. It can help artists do things that are not their expertise to serve the idea of the producer or fan.

Hatsune Miku is one of the famous examples of virtual star development. She was shaped like a high school girl with blue hair. She was designed and installed by a software company in Japan and was first introduced to advertise a speech synthesizer. Later, more people used this tool more to create original music, and Miku was noticed and followed by many people. They also held concerts using her hologram images and sang songs written by fans.

Last month, the Xinhua News Agency (is owned by China’s state) introduced AI technology capable of searching people through voice and image, called "composite anchors". This search technology is provided by Sogou, a search system operator based in Beijing.

Xiong said, what the company aims to is not to build digital avatars like the Xinhua newsreader and Miku. He wants the virtual idols of SNH48 to be able to interact with fans the most honestly.

The vice president of SNH48, Xiong also said that the idol group still have to move around the country to make nearly 1,000 shows every year. Apparently, digital members have an advantage of simultaneous appearing in many places. However, it is still limited to simpler functions such as performing personalized wake-up calls to fans.