Speech Recognition in VR/AR Applications

Finnish startup ZOAN is a VR studio that specializes in building VR/AR services for marketing, travel, education and real estate. ZOAN is a company of over 30 professionals spanning three continents. ZOAN is cooperating with Speechly to build next-generation VR and AR services for both industry professionals and consumers.

Sami Heinonen, the project manager of ZOAN still sees issues in the human-computer interaction in VR services. "We want to remove the barriers for natural interaction and start using voice and touch, the most natural interfaces for humans in our solutions.", says Heinonen.

Currently virtual environments usually require hand controllers for user input. To build more immersive experiences, better input technologies are needed. Voice would be the most natural option for both user input and as a feedback system from computer to human.

"There's a lot of industry specific tasks that require employees to look for meaningful data from many different displays. This could be better facilitated with voice technology and VR, Heinonen says.

One of the key differentiators of Speechly’s technology against current offering is it's real-time natural speech understanding. This improves user experience, because the feedback to user can be almost instanteous. Another important feature that's not available on smart speaker platforms is the ability to train the AI model for specific use cases.

"When user wants to accomplish a task by using Google Assistant or Amazon Alexa, first the device sends the audio for complete speech-to-text transcription to cloud. After the sentence is transformed into text, it's sent to another service that understands what is being requested and actually performs the task. Speechly is doing these two tasks simultaneously, just like a human. This makes it faster and smarter than current providers", tells Otto Söderlund, the CEO of Speechly.

The first application by ZOAN and Speechly is a virtual reality real estate tool that allows users to paint walls and change materials of the house by using voice. Future plans include bringing voice control into industry services such as airport maintenance.

"We are still in closed beta and can only accomodate certain amount of new developers on our platform. ZOAN is an interesting developer partner because the cooperation allows us to build our integrations for our technology to platforms such as Unity and Unreal 3D", Söderlund says.