27-09-2016, 12:14 PM
[attachment=71767]
Abstract—Over the past decade, many laboratories have
begun to explore brain–computer interface (BCI) technology as
a radically new communication option for those with neuromuscular
impairments that prevent them from using conventional
augmentative communication methods. BCI’s provide these users
with communication channels that do not depend on peripheral
nerves and muscles. This article summarizes the first international
meeting devoted to BCI research and development. Current BCI’s
use electroencephalographic (EEG) activity recorded at the scalp
or single-unit activity recorded from within cortex to control
cursor movement, select letters or icons, or operate a neuroprosthesis.
The central element in each BCI is a translation algorithm
that converts electrophysiological input from the user into output
that controls external devices. BCI operation depends on effective
interaction between two adaptive controllers, the user who encodes
his or her commands in the electrophysiological input provided to
the BCI, and the BCI which recognizes the commands contained
in the input and expresses them in device control. Current BCI’s
have maximum information transfer rates of 5–25 b/min. Achievement
of greater speed and accuracy depends on improvements in
signal processing, translation algorithms, and user training. These
improvements depend on increased interdisciplinary cooperation
between neuroscientists, engineers, computer programmers,
psychologists, and rehabilitation specialists, and on adoption
and widespread application of objective methods for evaluating
alternative methods. The practical use of BCI technology depends
on the development of appropriate applications, identification of
appropriate user groups, and careful attention to the needs and
desires of individual users. BCI research and development will
also benefit from greater emphasis on peer-reviewed publications, and from adoption of standard venues for presentations and
discussion.
Index Terms—Brain–computer interface (BCI), electroencephalography
(EEG), augmentative communication.
INTRODUCTION
Brain–computer interfaces (BCI’s) give their users communication
and control channels that do not depend on the brain’s
normal output channels of peripheral nerves and muscles. Current
interest in BCI development comes mainly from the hope
that this technology could be a valuable new augmentative communication
option for those with severe motor disabilities—disabilities
that prevent them from using conventional augmentative
technologies, all of which require some voluntary muscle
control. Over the past five years, the volume and pace of BCI research
have grown rapidly. In 1995 there were no more than six
active BCI research groups, now there are more than 20. They
are focusing on brain electrical activity, recorded from the scalp
as electroencephalographic activity (EEG) or from within the
brain as single-unit activity, as the basis for this new communication
and control technology.
In recognition of this recent rapid development and its potential
importance for those with motor disabilities, the National
Center for Medical Rehabilitation Research of the National Institute
of Child Health and Human Development of the National
Institutes of Health sponsored a workshop on BCI technology.
This workshop, also supported by the Eastern Paralyzed Veterans
Association and the Whitaker Foundation and organized
by the Wadsworth Center of the New York State Department of
Health, took place in June of 1999 at the Rensselaerville Institute
near Albany, New York. Fifty scientists and engineers participated.
They represented 22 different research groups from
the United States, Canada, Great Britain, Germany, Austria, and
Italy. Their principal goals were: 1) to review the current state
of BCI research; 2) to define the aims of basic and applied BCI
research; 3) to identify and address the key technical issues; and
4) to consider development of standard research procedures and
assessment methods.
On the first day, one person from each group gave a brief summary
of his or her group’s current work and future plans. The
substance of these talks is presented in the peer-reviewed papers
that follow this article. They range from descriptions of a
variety of functioning EEG-based or single-unit based BCI’s, to
analyses of the correlations between EEG or single-unit activity
and the brain’s conventional motor outputs, to investigations of issues important for BCI applications, to BCI software development.
Together they constitute a comprehensive review of the
present state of BCI research.
The following two days were devoted to six discussion sessions,
each led by a panel of five to seven people; and each addressing
a set of questions focused on a single important aspect
of BCI research and development. Evenings were occupied
with demonstrations of BCI technology and by poster presentations.
The discussion sessions were designed to cover the full
range of crucial issues, from the essential features of any BCI,
to the brain activity it uses, to the algorithms that translate that
activity into control signals, to user–system interactions, to research
methods and standards, to practical applications in rehabilitation
settings. The sections that follow, written by the panel
chairmen, summarize the contents and conclusions of these discussions.
Taken together, these summaries touch on each key
issue at least once and often more than once and in different
ways.
SESSION 1: DEFINITION AND ESSENTIAL FEATURES OF A
BRAIN–COMPUTER INTERFACE (BCI)
Since the EEG was first described by Hans Berger in 1929
[1], people have speculated that it might be used for communication
and control, that it might allow the brain to act on the environment
without the normal intermediaries of peripheral nerves
and muscles. In the 1970’s, several scientists developed simple
communication systems that were driven by electrical activity
recorded from the head. Early in that decade, the Advanced Research
Projects Agency (ARPA, which also sponsored the initial
development of the internet) of the U.S. Department of Defense
became interested in technologies that provided a more
immersed and intimate interaction between humans and computers
and included so-called “bionic” applications. A program
proposed and directed by Dr. George Lawrence focused initially
on autoregulation and cognitive biofeedback. It sought to develop
biofeedback techniques that would improve human performance,
especially the performance of military personnel engaged
in tasks that had high mental loads. The research produced
some valuable insights on biofeedback, but made minimal
progress toward its stated goals. A new direction, under
the more general label of “biocybernetics,” was then defined and
became the main source of support for bionics research in the
ensuing years. One of the directives of the biocybernetics program
was to evaluate the possibility that biological signals, analyzed
in real-time by computer, could assist in the control of vehicles,
weaponry, or other systems. The most successful project
in this area was that headed by Dr. Jacques Vidal, Director of the
Brain–Computer Interface Laboratory at UCLA. Using computer-generated
visual stimulation and sophisticated signal processing,
the research showed that single-trial (i.e., not averaged)
visual evoked potentials (VEP’s) could provide a communication
channel by which a human could control the movement of
a cursor through a two-dimensional maze [2].
Vidal’s studies and other less well-controlled early work
brought out the importance of the distinction between control
systems that use actual EEG activity and those that use EMG
(electromyographic) activity from scalp or facial muscles Because scalp-recorded EMG activity can be much more
prominent than EEG activity at the same locations, EMG-based
communication can masquerade as EEG-based communication.
To the extent that EMG-based communication is mistaken
for EEG-based communication, it can hamper the latter’s
development. Careful spectral and topographical analysis may
be needed to distinguish one from the other. The early work
also served to bring out the fundamental distinction between
EEG-based communication that depends on muscle control
(e.g., visual evoked potentials that depend on where the eyes
are directed), and EEG-based control that does not depend on
muscle control.
These distinctions shaped the definition of the term BCI put
forward in this session: “A brain–computer interface is a communication
system that does not depend on the brain’s normal
output pathways of peripheral nerves and muscles.” This definition
also reflects the principal reason for recent interest in
BCI development—the possibilities it offers for providing new
augmentative communication technology to those who are paralyzed
or have other severe movement deficits. All other augmentative
communication technologies require some form of
muscle control, and thus may not be useful for those with the
most severe motor disabilities, such as late-stage amyotrophic
lateral sclerosis, brainstem stroke, or severe cerebral palsy.
As a number of the presentations at this conference demonstrated,
several different true BCI’s, that is, communication systems
that do not appear to depend on nerves and muscles, have
been achieved (e.g., [3]–[9]). These systems use either EEG activity
recorded from the scalp or the activity of individual cortical
neurons recorded from implanted electrodes. While these
are exciting developments, with considerable theoretical significance
and practical promise, they are relatively low bandwidth
devices, offering maximum information transfer rates of
5–25 bits/min at best. Furthermore, improvement is likely to be
gradual, and to require continued careful and laborious investigation.
BCI development requires recognition that a “wire-tapping”
analogy probably does not apply—that the goal is not simply
to listen in on brain activity (via EEG, intracortical recording,
or some other method) and thereby determine a person’s intentions.
A BCI is a new output channel for the brain, and, like the
brain’s normal output channels of peripheral nerves and muscles,
is likely to engage the brain’s adaptive capacities, which
adjust output so as to optimize performance. Thus, BCI operation
depends on the interaction of two adaptive controllers, the
user’s brain, which produces the activity measured by the BCI
system, and the system itself, which translates that activity into
specific commands. Successful BCI operation is essentially a
new skill, a skill that consists not of proper muscle control but
rather proper control of EEG (or single-unit) activity.
Like any communication and control system, a BCI has an
input, an output, and a translation algorithm that converts the
former to the latter. BCI input consists of a particular feature (or
features) of brain activity and the methodology used to measure
that feature. As the BCI and pre-BCI studies presented at this
workshop illustrate, BCI’s may use frequency-domain features
(such as EEG or rhythms occurring in specific areas of
cortex) [6]–[9] and [11]–[14], or time-domain features (such as slow cortical potentials, P300 potentials, or the action potentials
of single cortical neurons) [3]–[5], [10], [12], [15]–[18]. The
methodology includes the scalp electrode type and locations,
the referencing method, the spatial and temporal filters, and
other signal-processing methods used to detect and measure the
features. The distinction between a feature as a reflection of a
specific aspect of nervous system physiology and anatomy and
a methodology as a technique for measuring the feature is more
clear for some features (e.g., the firing rate of a single cortical
neuron, which is presumably the same however it is measured)
than for others (e.g., autoregressive parameters, which depend
on the details of the analysis algorithm). Nevertheless, the
distinction is important because attention to features as reflections
of nervous system anatomy and physiology, rather than as
merely products of particular analysis methods, helps guide improvements
in BCI technology, and also encourages continued
attention to the problem of artifacts such as EMG activity
(which can, for example, affect autoregressive parameters).
Each BCI uses a particular algorithm to translate its input
(e.g., its chosen EEG features) into output control signals. This
algorithm might include linear or nonlinear equations, a neural
network, or other methods, and might incorporate continual
adaptation of important parameters to key aspects of the input
provided by the user. BCI outputs can be cursor movement,
letter or icon selection, or another form of device control, and
provides the feedback that the user and the BCI can use to
adapt so as to optimize communication.
In addition to its input, translation algorithm, and output,
each BCI has other distinctive characteristics. These include its
On/Off mechanism (e.g., EEG signals or conventional control);
response time, speed and accuracy and their combination into
information transfer rate; type and extent of user training
required, appropriate user population; appropriate applications;
and constraints imposed on concurrent conventional sensory
input and motor output (e.g., the need for a stereotyped visual
input, or the requirement that the user remain motionless).
Because BCI operation depends on the user encoding his or
her wishes in the EEG (or single-unit) features that the system
measures and translates into output control signals, progress
depends on development of improved training methods. Future
studies should evaluate the effects of the instructions given to
users, and analyze the relationships between user reports of
strategies employed and actual BCI performance. For example,
some BCI protocols ask that the user employ very specific
motor imagery (e.g., imagery of right or left hand movement)
or other mental tasks to produce the EEG features the system
uses as control signals (e.g., [7], [9]). Others may leave the
choice of imagery, or the decision to use any imagery at all,
up to the user (e.g., [3], [8]). Analysis of the similarities and
differences between acquisition of BCI control and acquisition
of conventional motor or nonmotor skills could lead to
improvements in training methods. The impacts of subject
motivation, fatigue, frustration, and other aspects of mental
states also require exploration. Users’ reports might help in
assessing these factors. At the same time, the value of such
reports is not clear. Users’ reports of their strategies may not
accurately reflect the processes of achieving and maintaining
Abstract—Over the past decade, many laboratories have
begun to explore brain–computer interface (BCI) technology as
a radically new communication option for those with neuromuscular
impairments that prevent them from using conventional
augmentative communication methods. BCI’s provide these users
with communication channels that do not depend on peripheral
nerves and muscles. This article summarizes the first international
meeting devoted to BCI research and development. Current BCI’s
use electroencephalographic (EEG) activity recorded at the scalp
or single-unit activity recorded from within cortex to control
cursor movement, select letters or icons, or operate a neuroprosthesis.
The central element in each BCI is a translation algorithm
that converts electrophysiological input from the user into output
that controls external devices. BCI operation depends on effective
interaction between two adaptive controllers, the user who encodes
his or her commands in the electrophysiological input provided to
the BCI, and the BCI which recognizes the commands contained
in the input and expresses them in device control. Current BCI’s
have maximum information transfer rates of 5–25 b/min. Achievement
of greater speed and accuracy depends on improvements in
signal processing, translation algorithms, and user training. These
improvements depend on increased interdisciplinary cooperation
between neuroscientists, engineers, computer programmers,
psychologists, and rehabilitation specialists, and on adoption
and widespread application of objective methods for evaluating
alternative methods. The practical use of BCI technology depends
on the development of appropriate applications, identification of
appropriate user groups, and careful attention to the needs and
desires of individual users. BCI research and development will
also benefit from greater emphasis on peer-reviewed publications, and from adoption of standard venues for presentations and
discussion.
Index Terms—Brain–computer interface (BCI), electroencephalography
(EEG), augmentative communication.
INTRODUCTION
Brain–computer interfaces (BCI’s) give their users communication
and control channels that do not depend on the brain’s
normal output channels of peripheral nerves and muscles. Current
interest in BCI development comes mainly from the hope
that this technology could be a valuable new augmentative communication
option for those with severe motor disabilities—disabilities
that prevent them from using conventional augmentative
technologies, all of which require some voluntary muscle
control. Over the past five years, the volume and pace of BCI research
have grown rapidly. In 1995 there were no more than six
active BCI research groups, now there are more than 20. They
are focusing on brain electrical activity, recorded from the scalp
as electroencephalographic activity (EEG) or from within the
brain as single-unit activity, as the basis for this new communication
and control technology.
In recognition of this recent rapid development and its potential
importance for those with motor disabilities, the National
Center for Medical Rehabilitation Research of the National Institute
of Child Health and Human Development of the National
Institutes of Health sponsored a workshop on BCI technology.
This workshop, also supported by the Eastern Paralyzed Veterans
Association and the Whitaker Foundation and organized
by the Wadsworth Center of the New York State Department of
Health, took place in June of 1999 at the Rensselaerville Institute
near Albany, New York. Fifty scientists and engineers participated.
They represented 22 different research groups from
the United States, Canada, Great Britain, Germany, Austria, and
Italy. Their principal goals were: 1) to review the current state
of BCI research; 2) to define the aims of basic and applied BCI
research; 3) to identify and address the key technical issues; and
4) to consider development of standard research procedures and
assessment methods.
On the first day, one person from each group gave a brief summary
of his or her group’s current work and future plans. The
substance of these talks is presented in the peer-reviewed papers
that follow this article. They range from descriptions of a
variety of functioning EEG-based or single-unit based BCI’s, to
analyses of the correlations between EEG or single-unit activity
and the brain’s conventional motor outputs, to investigations of issues important for BCI applications, to BCI software development.
Together they constitute a comprehensive review of the
present state of BCI research.
The following two days were devoted to six discussion sessions,
each led by a panel of five to seven people; and each addressing
a set of questions focused on a single important aspect
of BCI research and development. Evenings were occupied
with demonstrations of BCI technology and by poster presentations.
The discussion sessions were designed to cover the full
range of crucial issues, from the essential features of any BCI,
to the brain activity it uses, to the algorithms that translate that
activity into control signals, to user–system interactions, to research
methods and standards, to practical applications in rehabilitation
settings. The sections that follow, written by the panel
chairmen, summarize the contents and conclusions of these discussions.
Taken together, these summaries touch on each key
issue at least once and often more than once and in different
ways.
SESSION 1: DEFINITION AND ESSENTIAL FEATURES OF A
BRAIN–COMPUTER INTERFACE (BCI)
Since the EEG was first described by Hans Berger in 1929
[1], people have speculated that it might be used for communication
and control, that it might allow the brain to act on the environment
without the normal intermediaries of peripheral nerves
and muscles. In the 1970’s, several scientists developed simple
communication systems that were driven by electrical activity
recorded from the head. Early in that decade, the Advanced Research
Projects Agency (ARPA, which also sponsored the initial
development of the internet) of the U.S. Department of Defense
became interested in technologies that provided a more
immersed and intimate interaction between humans and computers
and included so-called “bionic” applications. A program
proposed and directed by Dr. George Lawrence focused initially
on autoregulation and cognitive biofeedback. It sought to develop
biofeedback techniques that would improve human performance,
especially the performance of military personnel engaged
in tasks that had high mental loads. The research produced
some valuable insights on biofeedback, but made minimal
progress toward its stated goals. A new direction, under
the more general label of “biocybernetics,” was then defined and
became the main source of support for bionics research in the
ensuing years. One of the directives of the biocybernetics program
was to evaluate the possibility that biological signals, analyzed
in real-time by computer, could assist in the control of vehicles,
weaponry, or other systems. The most successful project
in this area was that headed by Dr. Jacques Vidal, Director of the
Brain–Computer Interface Laboratory at UCLA. Using computer-generated
visual stimulation and sophisticated signal processing,
the research showed that single-trial (i.e., not averaged)
visual evoked potentials (VEP’s) could provide a communication
channel by which a human could control the movement of
a cursor through a two-dimensional maze [2].
Vidal’s studies and other less well-controlled early work
brought out the importance of the distinction between control
systems that use actual EEG activity and those that use EMG
(electromyographic) activity from scalp or facial muscles Because scalp-recorded EMG activity can be much more
prominent than EEG activity at the same locations, EMG-based
communication can masquerade as EEG-based communication.
To the extent that EMG-based communication is mistaken
for EEG-based communication, it can hamper the latter’s
development. Careful spectral and topographical analysis may
be needed to distinguish one from the other. The early work
also served to bring out the fundamental distinction between
EEG-based communication that depends on muscle control
(e.g., visual evoked potentials that depend on where the eyes
are directed), and EEG-based control that does not depend on
muscle control.
These distinctions shaped the definition of the term BCI put
forward in this session: “A brain–computer interface is a communication
system that does not depend on the brain’s normal
output pathways of peripheral nerves and muscles.” This definition
also reflects the principal reason for recent interest in
BCI development—the possibilities it offers for providing new
augmentative communication technology to those who are paralyzed
or have other severe movement deficits. All other augmentative
communication technologies require some form of
muscle control, and thus may not be useful for those with the
most severe motor disabilities, such as late-stage amyotrophic
lateral sclerosis, brainstem stroke, or severe cerebral palsy.
As a number of the presentations at this conference demonstrated,
several different true BCI’s, that is, communication systems
that do not appear to depend on nerves and muscles, have
been achieved (e.g., [3]–[9]). These systems use either EEG activity
recorded from the scalp or the activity of individual cortical
neurons recorded from implanted electrodes. While these
are exciting developments, with considerable theoretical significance
and practical promise, they are relatively low bandwidth
devices, offering maximum information transfer rates of
5–25 bits/min at best. Furthermore, improvement is likely to be
gradual, and to require continued careful and laborious investigation.
BCI development requires recognition that a “wire-tapping”
analogy probably does not apply—that the goal is not simply
to listen in on brain activity (via EEG, intracortical recording,
or some other method) and thereby determine a person’s intentions.
A BCI is a new output channel for the brain, and, like the
brain’s normal output channels of peripheral nerves and muscles,
is likely to engage the brain’s adaptive capacities, which
adjust output so as to optimize performance. Thus, BCI operation
depends on the interaction of two adaptive controllers, the
user’s brain, which produces the activity measured by the BCI
system, and the system itself, which translates that activity into
specific commands. Successful BCI operation is essentially a
new skill, a skill that consists not of proper muscle control but
rather proper control of EEG (or single-unit) activity.
Like any communication and control system, a BCI has an
input, an output, and a translation algorithm that converts the
former to the latter. BCI input consists of a particular feature (or
features) of brain activity and the methodology used to measure
that feature. As the BCI and pre-BCI studies presented at this
workshop illustrate, BCI’s may use frequency-domain features
(such as EEG or rhythms occurring in specific areas of
cortex) [6]–[9] and [11]–[14], or time-domain features (such as slow cortical potentials, P300 potentials, or the action potentials
of single cortical neurons) [3]–[5], [10], [12], [15]–[18]. The
methodology includes the scalp electrode type and locations,
the referencing method, the spatial and temporal filters, and
other signal-processing methods used to detect and measure the
features. The distinction between a feature as a reflection of a
specific aspect of nervous system physiology and anatomy and
a methodology as a technique for measuring the feature is more
clear for some features (e.g., the firing rate of a single cortical
neuron, which is presumably the same however it is measured)
than for others (e.g., autoregressive parameters, which depend
on the details of the analysis algorithm). Nevertheless, the
distinction is important because attention to features as reflections
of nervous system anatomy and physiology, rather than as
merely products of particular analysis methods, helps guide improvements
in BCI technology, and also encourages continued
attention to the problem of artifacts such as EMG activity
(which can, for example, affect autoregressive parameters).
Each BCI uses a particular algorithm to translate its input
(e.g., its chosen EEG features) into output control signals. This
algorithm might include linear or nonlinear equations, a neural
network, or other methods, and might incorporate continual
adaptation of important parameters to key aspects of the input
provided by the user. BCI outputs can be cursor movement,
letter or icon selection, or another form of device control, and
provides the feedback that the user and the BCI can use to
adapt so as to optimize communication.
In addition to its input, translation algorithm, and output,
each BCI has other distinctive characteristics. These include its
On/Off mechanism (e.g., EEG signals or conventional control);
response time, speed and accuracy and their combination into
information transfer rate; type and extent of user training
required, appropriate user population; appropriate applications;
and constraints imposed on concurrent conventional sensory
input and motor output (e.g., the need for a stereotyped visual
input, or the requirement that the user remain motionless).
Because BCI operation depends on the user encoding his or
her wishes in the EEG (or single-unit) features that the system
measures and translates into output control signals, progress
depends on development of improved training methods. Future
studies should evaluate the effects of the instructions given to
users, and analyze the relationships between user reports of
strategies employed and actual BCI performance. For example,
some BCI protocols ask that the user employ very specific
motor imagery (e.g., imagery of right or left hand movement)
or other mental tasks to produce the EEG features the system
uses as control signals (e.g., [7], [9]). Others may leave the
choice of imagery, or the decision to use any imagery at all,
up to the user (e.g., [3], [8]). Analysis of the similarities and
differences between acquisition of BCI control and acquisition
of conventional motor or nonmotor skills could lead to
improvements in training methods. The impacts of subject
motivation, fatigue, frustration, and other aspects of mental
states also require exploration. Users’ reports might help in
assessing these factors. At the same time, the value of such
reports is not clear. Users’ reports of their strategies may not
accurately reflect the processes of achieving and maintaining