Voice GUI layout system
Original Publication Date: 2004-Jan-23
Included in the Prior Art Database: 2004-Jan-23
Current voice control solutions for PC's rely on providing voice commands to interact with a system that is fundamentally designed to be used with a keyboard and a mouse. This means that a number of problems are presented when you try to use voice control of a current graphical user interface (GUI) desktop. Things such as switching window context, moving windows around, and re-sizing windows are practically impossible to do using voice control. A solution is needed that moves away from these difficult concepts and provides easy desktop management for voice control systems to produce a control mechanism which would allow a computer to be used without any access to a mouse or a keyboard.
Voice GUI layout system
Window management systems have in the past provided the ability to automatically "arrange" windows on the screen, however these system are simply designed around bringing order to the desktop, and as such do not enforce window layouts to prevent overlapping. Once the windows have been re-arranged (in a similar way to currently multiple document interface for programs such as Microsoft Word) they are still resizeable and stackable and it is easy to get back into a situation where mouse control is required to select the window which is currently hidden.
The main difficulty with a voice controlled system is dealing with overlapping windows. This concept is fundamental to the standard windows desktop, and works well when you can use a device such as a mouse to interact with the display and move things around. However this does not work well when trying to control the desktop using you voice. There is no way to conveniently and easily select windows that are layered, or to move them around and provide resizing once they are selected. To this end moving and resizing windows to arbitrary sizes and positions is removed from this proposed desktop management solution.
Instead the desktop allows only a fixed number of layout positions and sizes. Applications, by default, fill the whole screen, and can be split both horizontally and vertically into a maximum of 4 sections. This allows each screen section to be easily referenced by voice command, and with a screen split into quarters, each section will still be easily viewable.
Launching applications, and managing applications that are currently running: The system would initially present a main control page. This would be the starting point for all application launching, the selection of different displays, and selection of an active application.
Applications are initially start as full screen. The display can then split using a split voice command on either the horizontal or vertical eg. "Split top verticall...