Shimon Is A Robot That Can Sing And Write His Own Songs

Shimon is a robot that can sing and compose songs by collaborating with humans, he also has several other cool upgraded features. Created by Georgia Tech, the robot came out several years ago, but developers behind this robot are giving him new capacities.

Shimon now sings with a human voice, he can also play the marimba better, compose lyrics, even make facial expressions with his eyebrows.

A new song, “Into Your Mind,” indicates what the robot can do now. He wrote the song and lyrics in a collaboration with humans. The researchers will even release an album on Spotify featuring Shimon’s songs and there will also be a tour to support the material.

According to Gil Weinberg, Georgia Tech professor and creator of the robot, the main goal of this project is to create surprising, inspiring music through the collaboration of humans and robots. They hope that robots will be able to bring their best capabilities like unique algorithm-based ideas and mechanical abilities while humans offer their best features like expressions and emotions. Through this collaboration, the robots will hopefully connect with humans on an expressive and emotional level.

For the system to work, the professor gives Shimon a theme to create a new song. Then, by learning from more than 50,000 songs’ lyrics, the robot assembles a new lyric based around the given theme. For example, you give Shimon a “storm” theme, he will come up with a lyric including words like rain. Also, the robot can decide which words are more relevant to the theme and will create more lyrics from that. Instead of just using deep learning, the researchers combine it with semantic knowledge to generate lyrics with better rhythm and rhyme than other text generators.

In terms of his voice, the team collaborated with Pompeu Fabra University’s Music Technology Group. The Georgia Tech team gave the other team the melody and the lyrics. Then, using a deep learning-based voice synthesizer to create a voice for Shimon.

In addition, Shimon has some new hardware, including eyebrows, a mouth, as well as updated head movements to make the robot work in concert with the music. The purpose behind these updates is so that Shimon can connect with humans in the band during their performances.

In a press release, Lisa Zahray, who helped devise the robot's expressive eyebrow movements, said that they had to think about his role at each time in the song and the things he should be doing. She added that the robot should also be interacting with other people in the band.

Thanks to all the upgrades, Shimon can now perform music with a better dynamic range, where the loud parts are louder and the soft parts are quieter, allowing for more expressive music. The robot can now play as many as 30 hits per second, compared with 7 hits per second in the past, which is faster than the best humans can.

Now, enjoy "Into Your Mind" from Shimon and the band:

>>> This May Be The Best Humanoid Robot So Far, It Can Take A Selfie