27-12-2012, 03:30 PM
THE NRI RICH TOUCH PAD Technology and Implementations
[attachment=45443]
ABSTRACT
This document describes the technology for, and implementations of, the NRI® rich touchpad,
a novel touchpad controller, based on U.S. Patent 6,570,078, which can be used for a
wide variety of real-time applications. The touchpad is very powerful, provides an unprecedented
capability to enter large amounts of information at high speed and is extremely
flexible in the kinds of input it can process, the kinds of output it can produce and in how it
is configured. The touchpad creates images of the pressure exerted on it and can recognize
images created by contact with different parts of the hand (e.g. a fingertip, flat finger, palm,
wrist). It can process multiple areas of contact simultaneously and can extract the values
of a large number (typically three to six) of continuous parameters from each area of contact.
The touchpad is simple to use, and its capabilities can easily be extended. It can be
favorably compared to conventional computer pointing devices, such as the mouse, trackball
and conventional touchpad, which typically provide control of only two continuous parameters
at any one time and can process only a single region of contact.
The general-purpose nature of the touchpad permits the same basic system to be used in a
wide range of applications. They include CAD/CAE workstation control, real-time machine
control, human-machine interfaces for the physically disabled and electronic musical
instruments. The touchpad can also be used in intelligent machine sensing and robotics applications.
The touchpad can be implemented in a variety of ways. In one implementation, it incorporates
a two-dimensional pressure-sensor array, a data acquisition and compression stage,
an image processing and recognition stage, and an application interface. Special hardware
and algorithms permit the data processing and image processing to be carried out in real
time. The system can be modularized to support partitions of the sensor array into functionally
discrete regions and aggregations of sensor arrays to form larger arrays.
Introduction
This document, based on U.S. Patent 6,570,078 [1], describes the technology for, and implementations
of, the NRI® rich touchpad, a novel controller that can be used for a wide
range of data entry and real-time applications. The touchpad was originally envisioned as
a human-machine interface, though it can also be used in machine sensing and robotics.
When used by the hand, the touchpad provides an unprecedented capability to enter large
amounts of information at high speed.
The touchpad incorporates a pressure-sensor array for gathering information in the form of
real-time images of the pressure exerted on it [2]. When used as a human-machine interface,
these images are typically produced by contact with the user's hand, though they can
be produced using other parts of the body, such as the foot. The pressure images are presented
to a data acquisition and compression stage, whose output is sent to an image processing
and recognition stage. The image processing and recognition stage is used to
identify the shape of particular types of pressure images (e.g. images of a fingertip, flat finger,
thumb, palm, wrist). The touchpad can process multiple regions of contact simultaneously
and can extract the values of a large number (typically three to six) of continuous
parameters from each region of contact. An application interface assigns these values to
control signals, which can be used to control arbitrary external systems. Special hardware
and algorithms enable the data acquisition, the image processing and recognition, and the
derivation of parameter values to be carried out in real time [2,3].
Illustrations of User-Level Operation
Before we consider the technology of the touchpad, it will be useful to give a sense of how
it operates at the user level. We will first show how the touchpad makes it possible to control
six parameters at once by inducing variations in a single contiguous image, and will
then briefly discuss the touchpad’s capability to process compound images, images consisting
of multiple, non-overlapping contiguous images.
Controlling Six Parameters with One Finger
Assume the sensor array is contacted by the end joint of a single finger, as suggested in Figure
1a. The pressure image produced by such contact will be similar to that shown in Figure
1b. Note that the darker the image, the higher the pressure.
Compound Images
A notable feature of the touchpad is its capability to process, and extract the values of parameters
from, compound images. Figure 3 shows a compound image created by pressing
the left part of the left hand against the sensor array with the palm raised. The image consists
of four non-overlapping contiguous regions or “blobs.”
System Architecture
We turn now to the architecture of the touchpad. We will begin by considering, in the first
two sections, some aspects of the hardware design. We will next consider the overall information
flow of the touchpad, and then some aspects of the software (or firmware) design.
This section is concerned with the design of the basic system. In section 4, we will consider
some enhancements of, and alternatives to, the basic system.
Physical Formats
Figures 4a-d illustrate some possible physical formats for the touchpad. In Figures 4a-b the
pressure-sensor array and its supporting hardware share the same housing. In Figure 4c the
sensor array and at least some of its supporting hardware are separately housed and connected
by a flexible cable to permit a smaller, more portable housing for the sensor array.
In Figure 4d the supporting hardware is incorporated into a larger housing that also contains
the system the touchpad controls (e.g. a computer workstation, a robotics system, an electronic
musical instrument, a lighting control system). Many other formats are possible. For
instance, a pressure-sensor array can be installed inside or on the surface of a glove. Similarly,
a pressure-sensor array can be installed inside or on the bottom of a shoe or sandal,
or inside or on the surface of a sock.
Data Acquisition and Data Processing
The touchpad is intended as a real-time device with a fast response time. A problem that
must be faced in implementing such a system -- i.e. one that must process a large number
of measurements very quickly -- is memory access within the data processing hardware. By
using special hardware and algorithms, the functions involved can be carried out in real
time in an inexpensive implementation using existing, low-cost commodity microprocessors.
The hardware and algorithms permit a large sensor array to be scanned and processed
at rates of 10-100 (nominally 20-40) times per second. In this subsection we sketch such an
algorithm, and in the next subsection we describe the algorithm in more detail.
The amount of information required to characterize each scan of the sensor array can be
minimized by exploiting the expected characteristics of pressure images. A record comprising
only a few entries is allocated for each “blob” (i.e. contiguous region of sensors with
non-zero pressure values) when it is first detected. (Strictly, the relevant pressure values are
ones that exceed a noise threshold, though we will say that such pressure values are nonzero.)
As the scan progresses, the entries in the record undergo running updates. At the end
of a scan, or upon confirmation that the record is complete for a given blob, simple postscan
processing is done on the record to calculate values of parameters useful for image
processing and recognition. This technique considerably reduces the amount of information
required to characterize each scan of the sensor array.
[attachment=45443]
ABSTRACT
This document describes the technology for, and implementations of, the NRI® rich touchpad,
a novel touchpad controller, based on U.S. Patent 6,570,078, which can be used for a
wide variety of real-time applications. The touchpad is very powerful, provides an unprecedented
capability to enter large amounts of information at high speed and is extremely
flexible in the kinds of input it can process, the kinds of output it can produce and in how it
is configured. The touchpad creates images of the pressure exerted on it and can recognize
images created by contact with different parts of the hand (e.g. a fingertip, flat finger, palm,
wrist). It can process multiple areas of contact simultaneously and can extract the values
of a large number (typically three to six) of continuous parameters from each area of contact.
The touchpad is simple to use, and its capabilities can easily be extended. It can be
favorably compared to conventional computer pointing devices, such as the mouse, trackball
and conventional touchpad, which typically provide control of only two continuous parameters
at any one time and can process only a single region of contact.
The general-purpose nature of the touchpad permits the same basic system to be used in a
wide range of applications. They include CAD/CAE workstation control, real-time machine
control, human-machine interfaces for the physically disabled and electronic musical
instruments. The touchpad can also be used in intelligent machine sensing and robotics applications.
The touchpad can be implemented in a variety of ways. In one implementation, it incorporates
a two-dimensional pressure-sensor array, a data acquisition and compression stage,
an image processing and recognition stage, and an application interface. Special hardware
and algorithms permit the data processing and image processing to be carried out in real
time. The system can be modularized to support partitions of the sensor array into functionally
discrete regions and aggregations of sensor arrays to form larger arrays.
Introduction
This document, based on U.S. Patent 6,570,078 [1], describes the technology for, and implementations
of, the NRI® rich touchpad, a novel controller that can be used for a wide
range of data entry and real-time applications. The touchpad was originally envisioned as
a human-machine interface, though it can also be used in machine sensing and robotics.
When used by the hand, the touchpad provides an unprecedented capability to enter large
amounts of information at high speed.
The touchpad incorporates a pressure-sensor array for gathering information in the form of
real-time images of the pressure exerted on it [2]. When used as a human-machine interface,
these images are typically produced by contact with the user's hand, though they can
be produced using other parts of the body, such as the foot. The pressure images are presented
to a data acquisition and compression stage, whose output is sent to an image processing
and recognition stage. The image processing and recognition stage is used to
identify the shape of particular types of pressure images (e.g. images of a fingertip, flat finger,
thumb, palm, wrist). The touchpad can process multiple regions of contact simultaneously
and can extract the values of a large number (typically three to six) of continuous
parameters from each region of contact. An application interface assigns these values to
control signals, which can be used to control arbitrary external systems. Special hardware
and algorithms enable the data acquisition, the image processing and recognition, and the
derivation of parameter values to be carried out in real time [2,3].
Illustrations of User-Level Operation
Before we consider the technology of the touchpad, it will be useful to give a sense of how
it operates at the user level. We will first show how the touchpad makes it possible to control
six parameters at once by inducing variations in a single contiguous image, and will
then briefly discuss the touchpad’s capability to process compound images, images consisting
of multiple, non-overlapping contiguous images.
Controlling Six Parameters with One Finger
Assume the sensor array is contacted by the end joint of a single finger, as suggested in Figure
1a. The pressure image produced by such contact will be similar to that shown in Figure
1b. Note that the darker the image, the higher the pressure.
Compound Images
A notable feature of the touchpad is its capability to process, and extract the values of parameters
from, compound images. Figure 3 shows a compound image created by pressing
the left part of the left hand against the sensor array with the palm raised. The image consists
of four non-overlapping contiguous regions or “blobs.”
System Architecture
We turn now to the architecture of the touchpad. We will begin by considering, in the first
two sections, some aspects of the hardware design. We will next consider the overall information
flow of the touchpad, and then some aspects of the software (or firmware) design.
This section is concerned with the design of the basic system. In section 4, we will consider
some enhancements of, and alternatives to, the basic system.
Physical Formats
Figures 4a-d illustrate some possible physical formats for the touchpad. In Figures 4a-b the
pressure-sensor array and its supporting hardware share the same housing. In Figure 4c the
sensor array and at least some of its supporting hardware are separately housed and connected
by a flexible cable to permit a smaller, more portable housing for the sensor array.
In Figure 4d the supporting hardware is incorporated into a larger housing that also contains
the system the touchpad controls (e.g. a computer workstation, a robotics system, an electronic
musical instrument, a lighting control system). Many other formats are possible. For
instance, a pressure-sensor array can be installed inside or on the surface of a glove. Similarly,
a pressure-sensor array can be installed inside or on the bottom of a shoe or sandal,
or inside or on the surface of a sock.
Data Acquisition and Data Processing
The touchpad is intended as a real-time device with a fast response time. A problem that
must be faced in implementing such a system -- i.e. one that must process a large number
of measurements very quickly -- is memory access within the data processing hardware. By
using special hardware and algorithms, the functions involved can be carried out in real
time in an inexpensive implementation using existing, low-cost commodity microprocessors.
The hardware and algorithms permit a large sensor array to be scanned and processed
at rates of 10-100 (nominally 20-40) times per second. In this subsection we sketch such an
algorithm, and in the next subsection we describe the algorithm in more detail.
The amount of information required to characterize each scan of the sensor array can be
minimized by exploiting the expected characteristics of pressure images. A record comprising
only a few entries is allocated for each “blob” (i.e. contiguous region of sensors with
non-zero pressure values) when it is first detected. (Strictly, the relevant pressure values are
ones that exceed a noise threshold, though we will say that such pressure values are nonzero.)
As the scan progresses, the entries in the record undergo running updates. At the end
of a scan, or upon confirmation that the record is complete for a given blob, simple postscan
processing is done on the record to calculate values of parameters useful for image
processing and recognition. This technique considerably reduces the amount of information
required to characterize each scan of the sensor array.