Insight
What if you could make things appear in front of you with the power of your voice?
Idea
Skyfall
Execution
I had an idea of making a virtual reality (VR) demo where you can say things and make them fall from the sky.
For the initial demo, the only objects that you can magically summon are:
- a cube
- a white car – Ferrari
- a red car – Dodge Viper
- a house
- Deadpool
- a Star Destroyer
The “car” command randomly picks between one of 2 cars.
You can also say the word “clear” to remove all the objects from the scene.
Additionally, you can also move around the 3D space using an XBox controller or keyboard and mouse.
All this works within the Oculus Rift DK2.
To take this project further I can:
- add more objects and commands
- add Leap Motion support so you can interact with the objects using your hands
Tech Details
Problem
One of the roadblocks I hit early on was that Unity 5.3.2 does not support the .NET speech recognition DLL’s since it is using Mono.
Solution
In order to get around this, I created 2 applications and used a TCP port for them to communicate with each other.
Components
1) Server: C# .NET application
The voice recognition is being done using System.Speech.Recognition.
A GrammarBuilder object is created with the following words.
private string[] VALID_COMMANDS = {“clear”, “cube”, “car”, “house”, “deadpool”, “star destroyer” };
Once a word is recognised, a message is sent to a specified port via TCP.
2) Client: Unity application
On the client side, there is a TCP listener running on a thread that listens for TCP messages.
If a word is received (e.g. cube) the model named “Cube” is then cloned and added to the scene in front of and above where the user is looking.
First person controls and VR support was also added to make the experience more immersive.
benxtan 