Speech is an interesting input modality that has been used in many ways in the real world as an efficient modality for thousands of years. Put simply, speech is the essence of communication. Although not all applications are suited for this modality, many can use speech in a variety of ways. It is essential to match speech input to the task, as it is with any input modality. Extreme examples could include writing a letter with a mouse, and a freehand drawing tool that uses speech input.
To do this, I will be designing a computer game based on speech, rather than using the tried, tested, and failed approach of attempting to design a speech based game based on a computer game.
While analysing speech and communication, the most important fact is that speech exists as a means to interact with other people. The current method of controlling computer controlled bots in computer games involves keypresses and mouse driven menus. And when you have teams, the menu depth of commands can be high enough to be distracting. And in first player shooter games, this could make you lose a life. If there was another modality, such as speech, that could be used to command your team, this drawback can be eliminated.
This project aims to create a computer based game that has characters you can interact with. The interaction is based on queries and commands, and is inputted by speech.
The project also includes:
1. A method for creating speech operated AI.
and
2. A method for creating speech based computer games in general.
Top down view of the world and the players in it:
The setup of a 'Wizard of Oz' experiment to discover what the user would want to say to the other players:
The project is complete. Bots may be controlled by the user by certain speech commands and respond accordingly.
The dissertation may now be downloaded in Word (181 Kb) or Pdf (216 Kb), or viewed online in HTML.
A low quality movie file of the game in action may be downloaded in an Avi file. (1.79Mb)
1. There will be a structured method to follow to create more speech based computer games.
2. Speech input can be easily added to any previously created computer bots.
3. The level of interaction may be increased, allowing the computer players to follow more abstract commands or respond in a more nested discourse environment, eg:
Player: Ok, computer... Attack.
Computer: Hey, there's a sentry over there, want me to take him out?
Player: No, let Bert to do it, he's stronger.
Computer: Yes sir!
moments later...
Computer: I need help!
Player: Bert, help him!
Bert (bot): Yes sir!