13-08-2012, 02:43 PM
Voice Recognition Evaluation Report
[attachment=31291]
Introduction
This study investigated and compared several voice translator units. Three units were tested, which were:
the Phraselator, Voice Response Translator (VRT), and the Universal Translator (UT-103). The intent of
the study was to provide a comparison report that outlines the units’ performance capabilities, including
operation within noisy environments, ease of use/learning and other operational characteristics such as
battery life.
In 1997, the National Institute of Justice (NIJ) and NAVAIR ORL Training Systems Division (formerly
the Naval Air Warfare Center Training Systems Division) signed an Interagency Agreement (IAA) to
establish “ … a collaborative relationship to share modeling, simulation, and training technology to
address law enforcement and military requirements.” Under this IAA, NIJ requested that NAVAIR ORL
TSD develop training support and conduct field evaluations for the Voice Response Translator (VRT), a
language translation device developed by Integrated Wave Technologies specifically for law enforcement
officers. As NAVAIR ORL TSD and NIJ worked to expand and improve the utility of the VRT for law
enforcement, potential military applications were identified. Simultaneously, information regarding other
translation devices such as the Phraselator, developed by the Defense Advanced Research Projects
Agency (DARPA), and the Universal Translator (UT-103), a commercial product from Ectaco, Inc,
emerged. While there were testimonials regarding the utility and potential applications of all these
devices, there was no known existing laboratory test to evaluate the devices. In 2002, NIJ requested that
NAVAIR ORL conduct these tests in order to determine the speech accuracy of the units and include
degradation in noisy environments. As plans for the testing evolved, representatives from the Special
Operations Command (SOCOM) contacted NIJ regarding potential use of the VRT. There was already
substantial military interest in translation devices; the Phraselator had already been involved in several
field tests. SOCOM agreed to supply Phraselator units for the NAVAIR ORL laboratory tests and
requested specifically that an out-of-the-box test be conducted for the Phraselator, because this could have
direct bearing on SOCOM’s plans for fielding units.
The VRT is a system currently used by law enforcement and the Coast Guard. The phraseology covers
search, questioning, and commands. The target audience is military and law enforcement.
The Phraselator was a part of the DARPA one-way translation effort. Marine Acoustics, Inc. was awarded
a DARPA SBIR grant in January 2001 to develop the Phraselator. Phraselators were built and delivered
to military units in support of operation Enduring Freedom. The Phraselator covers medical and military
phrases (search, checkpoint and interrogation). The target audience is medical personnel and military
personnel.
The Universal Translator UT-103 is a commercial system designed to provide translation capability
centered on transportation and other tourist type interactions. It translates English to common European
languages: Spanish, German and French. The areas covered are plane, luggage, rail, taxi, and restaurant
ordering type phrases. The target audience is English-speaking tourists on a European vacation.
The physical testing was performed in 3 parts. The first was an “out of the box” test in which just the
Phraselator device was used. It consisted of 2 different subjects opening the device and trying to learn to
use it to perform simple translation task without any prior contact or familiarization with the unit. The
second was to test the units with 5 different phrases, picked from the units phrase list, first with ambient
room noise (<50Db), then with different levels of pink and white background noise. The sound level meter
7 October 2003
5
used is only sensitive to 50 Db, so measurements referred to as “<50 Db” mean that it’s below the meter’s
ability to make a measurement. The last test was to determine the battery life of the unit and recharge
[attachment=31291]
Introduction
This study investigated and compared several voice translator units. Three units were tested, which were:
the Phraselator, Voice Response Translator (VRT), and the Universal Translator (UT-103). The intent of
the study was to provide a comparison report that outlines the units’ performance capabilities, including
operation within noisy environments, ease of use/learning and other operational characteristics such as
battery life.
In 1997, the National Institute of Justice (NIJ) and NAVAIR ORL Training Systems Division (formerly
the Naval Air Warfare Center Training Systems Division) signed an Interagency Agreement (IAA) to
establish “ … a collaborative relationship to share modeling, simulation, and training technology to
address law enforcement and military requirements.” Under this IAA, NIJ requested that NAVAIR ORL
TSD develop training support and conduct field evaluations for the Voice Response Translator (VRT), a
language translation device developed by Integrated Wave Technologies specifically for law enforcement
officers. As NAVAIR ORL TSD and NIJ worked to expand and improve the utility of the VRT for law
enforcement, potential military applications were identified. Simultaneously, information regarding other
translation devices such as the Phraselator, developed by the Defense Advanced Research Projects
Agency (DARPA), and the Universal Translator (UT-103), a commercial product from Ectaco, Inc,
emerged. While there were testimonials regarding the utility and potential applications of all these
devices, there was no known existing laboratory test to evaluate the devices. In 2002, NIJ requested that
NAVAIR ORL conduct these tests in order to determine the speech accuracy of the units and include
degradation in noisy environments. As plans for the testing evolved, representatives from the Special
Operations Command (SOCOM) contacted NIJ regarding potential use of the VRT. There was already
substantial military interest in translation devices; the Phraselator had already been involved in several
field tests. SOCOM agreed to supply Phraselator units for the NAVAIR ORL laboratory tests and
requested specifically that an out-of-the-box test be conducted for the Phraselator, because this could have
direct bearing on SOCOM’s plans for fielding units.
The VRT is a system currently used by law enforcement and the Coast Guard. The phraseology covers
search, questioning, and commands. The target audience is military and law enforcement.
The Phraselator was a part of the DARPA one-way translation effort. Marine Acoustics, Inc. was awarded
a DARPA SBIR grant in January 2001 to develop the Phraselator. Phraselators were built and delivered
to military units in support of operation Enduring Freedom. The Phraselator covers medical and military
phrases (search, checkpoint and interrogation). The target audience is medical personnel and military
personnel.
The Universal Translator UT-103 is a commercial system designed to provide translation capability
centered on transportation and other tourist type interactions. It translates English to common European
languages: Spanish, German and French. The areas covered are plane, luggage, rail, taxi, and restaurant
ordering type phrases. The target audience is English-speaking tourists on a European vacation.
The physical testing was performed in 3 parts. The first was an “out of the box” test in which just the
Phraselator device was used. It consisted of 2 different subjects opening the device and trying to learn to
use it to perform simple translation task without any prior contact or familiarization with the unit. The
second was to test the units with 5 different phrases, picked from the units phrase list, first with ambient
room noise (<50Db), then with different levels of pink and white background noise. The sound level meter
7 October 2003
5
used is only sensitive to 50 Db, so measurements referred to as “<50 Db” mean that it’s below the meter’s
ability to make a measurement. The last test was to determine the battery life of the unit and recharge