Google's New Technology Will Not Only Let The Pixel 4 Sense Hand Motions But There's More

In the upcoming Pixel 4, Google will give its front cameras hands-free use, not only facial recognition but also hand gestures and motion sensing.

Originally dubbed Project Soli, the technology relies on radar to sense 3D movements in space in addition to a new infrared camera technology for scanning the face similar to Apple’s Face ID. The FCC approved of Soli tech earlier this year, which is anticipated to feature in the Pixel 4.

Project Soli means people can use air-swipes and waves to skip songs, silence calls, as well as operate other shortcuts. But there’s much more, Google may be attempting to create a new interface for its entire computing ecosystem, as implied in the latest Pixel-teasing blog post from Google.

Soli is part of Google’s Advanced Technologies and Projects Group (ATAP) that developed Project Jacquard – the smart fabric, Tango – the first phone-based augmented reality tech and Project Ara.

And the following are some future visions that Google holds for Soli.

Cameras no longer needed

While the Pixel 4 still features front-facing lenses with an infrared dot projector used for face scanning, Soli’s radar processor doesn’t require a camera for its work. In fact, then Google debuted Project Soli in 2015, it was used for controlling sound through in-air finger gestures, no need for remotes.

With Pixel 4, users can technically rely on its infrared-equipped camera to recognize gestures and hand, but the camera must be on and looking. On the contrary, theoretically, Soli would be able to work anytime. That means the technology could be applied for devices without cameras and still enable motion control. Hence, you may no longer need cameras in your connected home tech.

The ability to work in the dark or through clothes

Google talks about how Soli chip’s radar tech can work through the fabric. Maybe it would operate through pants in case of the phone, which means it can be embedded in walls, or even in other projects. Looks like we will have a technology to use for a door panel or light switch.

Smart home devices as the next step

If Google can combine voice and gesture, maybe we will be able to make commands for the Assistant without a wake word, or lower lights by moving our hand in a certain direction.

In 2016, Google has already indicated Soli operating with a smart speaker, which turned out to be the first use case for the technology. Now, as it has come to a phone, the next step is highly likely to involve the speaker or even the Nest.

Other Soli-powered products

During Google I/O 2016, the tech giant also tried Soli with hands-free control for a watch. Yes, the existing touch screen is cool, but imagine wearing a Pixel watch that you can control simply with near-screen gestures. You can rely on that to answer or silence calls, launch a workout, etc.

Or maybe we will even have wearables without screens, in which Soli serves as the interface. Or remember Project Jacquard, the smart fabric that needs touching to work? Soli could work with that too.

Car dashboards

Soli could be useful for controlling things inside the car without looking. While we can’t say for sure it’s safe to use such technology while driving, it’s completely possible to rely on Soli for controlling volumes or taking a new route suggestion.

A new tool for AR headsets

One outstanding ability of Soli is recognizing finger gestures from a distance. This is the exact thing that a lot of mixed-reality headsets like Microsoft Hololens have always wanted.

For AR, hand gestures are still extremely hard with Leap Motion never fulfilling the prototype headset for Project North Star. If Google is looking to evolve its Google Glass, Soli could be the key.