Unable To Type, Google Engineer Creates His Own Voice Recognition Software

When we talk about software developers, we often visualize them sitting at their computer desks, typing continuously to deal with some of the most complex IT problems. But with James Stout, things don’t work that way. Working at Google for 10 years, every time he works, Jame doesn’t use the keyboard. Instead, he uses voice recognition to command his computer. His working desk setup is quite special with an eye tracker, a monitor, a micro, and a specifically-designed keyboard. Under his desk, there is a foot pedal that he steps on. Each step will account for a number to mark each line.

It may sound simple and easy but behind this setup is an inspiring story. James suffers from what is known as the Ehlers-Danlos syndrome, a very rare condition. With this syndrome, Stout can’t type and he is more vulnerable to muscular injuries.

At the moment, James and his colleagues are researching methods that allow users to easily access and use products of Gooogle, like control apps with their voices, for example. He also has a blog named Hands-Free Coding where he shares about his process of coding using his voice instead of typing on the keyboard.

Talking with Business Insider, he said:

And time has proved Stout’s thinking as there are a lot of virtual assistants available on the market at the moment including Amazon Alexa, Google Assistant, and Apple Siri. These virtual assistants make life a lot easier for users when all they need to do is give out commands while the assistants will do the rest, from booking movie tickers, to buy groceries, etc. Stout’s voice recognition system not only helped him continue his dream but also gave other people inspiration.

There were times when Stout felt desperate, thinking that he would not become a software developer with his condition.

He said:

But after everything, Google and his colleagues have always supported him. His manager also allowed him to code using his voice recognition system.

Stout began working at Gooogle under the infrastructure team. After that, he joined Waymo, the self-driving car project. But this was also when his condition got worse. His fingers didn’t just feel sore anymore, as it turned into a sharp pain every time he tried to tap a key.

And as a software engineer, this terrifies Stout. Although he suffers from EDS for all his life, it did not become a real problem until he had to type regularly. When he was young, he found out that his joints are loose. When he started studying in college, he noticed that he was more prone to injuries.

There were some points that James thought he would have to give up his job. Some doctors had advised him to change his job to keep his health in check.

Stout said:

Put aside what the doctors said, Stout started testing voice recognition to control the computer, starting from simple tasks like surfing the Internet to a more complicated one like writing code. He said that he got his inspiration from Tavis Rudd, a developer who shared his own experience in coding in Python using his voice. Rudd even emphasized that combing typing and using voice commands would make the coding process a lot faster.

Stout continued:

Stout then ask his manager whether could use one day a week to use voice to type. And he was extremely thankful that his manager allowed him to do so. His manager quoted, saying that:

And then one day turned into five days a week.

His blog became a useful tool for him to interact with his readers. A lot of them commented that what he shared on his blog has helped them save their careers.

For example, one commented that his 12-year-old sister also shared the same syndrome as Stout. Stout then video-called them to share what he had learned with his research on coding using voice recognition.

Understanding how his blog could affect other people, Stoud decided to leave Waymo and joined Google’s Accessibility team. He is also specifically interested in machine learning. Although his specialty is voice recognition, James believes that machine learning is the key to help other disabled people.

He shared that:

At the moment, besides his full-time job, James is still working on his software called Dragon. He plans on continue sharing what he has done and discovered for free. He hopes that what he has been doing at Google would help other people around the world, not only software developers. He also hopes that his ideas and ambition for machine learning can inspire coders globally.

Stout’s ability to control the computer using his voice has reached its finest when he can write emails and type codes even faster than using a keyboard. However, with photo-editing tools like Photoshop or office suite like Powerpoint, there are still some limitations.

Another difficulty that Stout is facing is that he can’t command the computer or plan what he is working on at the same time since he uses the same part of the brain for both tasks. He hopes that he can create a voice system to minimize distraction in the future.

In contrast, a normal can still think of other things while typing.

However, despite these difficulties, James’ productivity has improved significantly. He not only can write code faster but he also gains a deeper understanding of every aspect of his work.

However, despite these difficulties, James’ productivity has improved significantly. He not only can write code faster but he also gains a deeper understanding of every aspect of his work. And most importantly, Stout would never have to give up his dream of working as a software engineer. And what he is working on has the potential of developing more in the future.