IMNIP

The Integration Manager and Natural Interaction Processor (IMNIP) merges multiple streams of input data and derives the intent represented by that data. It can be implemented in conventional desktop and notebook computers or embedded within small microprocessor-based control devices. While specifically designed for the Archimedes intelligent Total Access System (ITAS), the IMNIP can be used in any situation requiring a person to command and control a computer-based device. The Integration Manager (IM) is able to integrate inputs from multiple sources such as keyboards, speech recognizers, touch panels, Morse code, gestures and special sensors. The Natural Interaction Processor (NIP) functions operate independently of the input modalities. The strategies developed for the IMNIP provide significant advantages over earlier agent based NIP designs.

The IMNIP merges and decodes any number and combination of simultaneous input message streams.
Individual characters are parsed as they are received thereby eliminating the need to buffer strings of input data.
Pointers into tables and arrays drive all operations in the IMNIP. This eliminates string manipulations and provides instantaneous translation from an input command to an output event.
The IMNIP can be implemented equally well on large computers or small embedded microprocessors.
The IMNIP design is scaleable and can support any size corpus of command words.
The IMNIP can simultaneously parse multiple word lists.
The IMNIP recognizes synonyms.
The IMNIP decodes single word commands, multi word commands, macro commands and embedded multi word device names.
The IMNIP filters out unwanted words that precede or follow a macro or are embedded within a macro.
The IMNIP can separate and act upon macros and parameters contained within a single message.
The IMNIP enables simple devices to communicate efficiently with each other using natural text messages.
The IMNIP makes "Intention Driven Technology" feasible and practical.
The system can support multiple languages

The IMNIP performs all of the message merging and decoding functions for the new Intelligent Total Access System Modules (iTASK™).

One way of explaining the operation of the IMNIP is to think of it as a word filter, as depicted in the following diagram.

As depicted in this diagram, the IMNIPs distribute intelligence throughout a system.User IMNIPs use a corpus matched to the types of user input expected from its input data. User IMNIPs know about users but not about devices, and device IMNIPs know about the commands for the devices to which they are connected but nothing about users. Each IMNIP extracts predefined keywords and macros from messages that are received at its inputs. This type of word filter is necessary in practical systems since a device designer has no control over the words that may be chosen by a user to express an intended action. The designer does have full control, however, over the choice of words used to specify each function a target device is able to perform. The traditional solution to this problem is to force the user to learn scripts containing the exact words and phrases that have been predefined for each command on each device. Fundamental flaws with this traditional solution include:

Interaction is not intuitive.
Every command requires a separate script.
There is a steep learning curve for complex devices.
People are reluctant to learn scripts and they forget them when under pressure.
It is difficult to combine multiple input modalities.
Scripts that initiate an operation must be entered separately from scripts that provide parameters.

The IMNIP overcomes these problems by allowing a user to describe an intended outcome in his or her own words and gestures as if asking another person to perform the action. Whenever an IMNIP recognizes an intended action, it instructs the system to perform all of the necessary steps without requiring the user to participate in their execution or to even know what the steps are.

While a single IMNIP can perform all of the necessary merging and decoding operations for a particular combination of user inputs and output devices this results in very specialized and inflexible system. It is much more versatile when the IMNIP functions are distributed between input devices that handle the human centered functions and output devices that handle all of the the machine-dependent functions. A serial, or cascaded configuration of the IMNIPs enables systems to handle ad hoc combinations of users and target devices with minimum overhead. The first IMNIP handles user dependent details independently of the device requirements and the second IMNIP handles device specific details independently of the human factors. Parallel operation of IMNIPs as depicted by the four parallel input IMNIPs in the preceding figure, adds another level of flexibility by allowing different processing and different constraints to be independently applied to each of the input and output channels.

The ability for the IMNIP in each device controller to combine multiple inputs messages allows accessors to be used in collaborative modes without the need for channeling all of the information through a single central processor. The first IMNIP translates intuitive user messages into a compact, well-defined intermediate format that can be efficiently and unambiguously decoded by a second IMNIP to generate the control codes required by a particular device.
As depicted in the figure, a user can generate any input word or phrase either directly, such as by typing or talking to a speech recognizer, or indirectly such as by touching the image of an object on a touch screen tablet that generates a text message. After passing each input though an input IMNIP, only words or synonyms that are included in the ITAS corpus will survive. Filtered messages from multiple accessors can be directed to a single iTASK Module in which the IMNIP will integrate the messages into a single stream, extract command words, parameters, and macros, and generate the stream of output control codes necessary to achieve the intended outcome.