As indicated above, the SR program did not recognize a few words, and subject 1 achieved a word error rate of 2.1% and a BLEU score (Papineni, et al., 2002) of 95.6 (maximum 100). The program did not recognize several of subject 2's words, however, and she achieved a BLEU score of only 70.5. Speech recognition errors are common in MI research, as one study (Okumura, et al., 2002) had a 14%-15% word error rate, and another (Waibel, 1996) had a word error rate of 29%. These errors can significantly affect the following translation phase.

Three native Korean speakers were asked to evaluate the translations for comprehension using the scale:

1=do not understand at all, 2=mostly do not understand, 3=misunderstand more than understand, 4=no opinion, 5=understand more than misunderstand, 6=mostly understand, and 7= understand very clearly. For subject 1, the mean score for comprehension was 6.02 with a standard deviation of 1.25. Using the same scale, they were also asked how well all of the comments together should be understood, and the mean score was 4.67. Using this threshold, the comprehension was significantly higher than that required (t = 6.68, p < 0.001). For subject 2, the mean comprehension was only 4.96 with a standard deviation of 2.08, primarily because so many words were not recognized during the SR phase. Nevertheless, the comprehension was still higher than that required.

For the speech synthesis phase, the three were asked to evaluate the sentences in Table 3, spoken by the system in Korean.

Table 3: Comments for evaluating the speech synthesis phase

For each spoken sentence, each user gave the system a perfect score of 7 on the 1-to-7 scale, but they noted that the female voice sounded like someone reading a book rather than talking in a conversation. Even unknown words such as the sound animals might make, when written in Korean, were heard correctly.

The study suffers from several limitations. First, only two English speakers were used and only three Koreans evaluated the results. Other speakers might generate different SR errors, and other evaluators could have different opinions about comprehension. Second, only one source language (English) and one target language (Korean) were tested. Accuracies with other language combinations are likely to be different (Aiken & Park, 2010; Aiken, et al., 2009). Third, the system was not tested in the field. Background noise can severely degrade the quality of the speech recognition. Finally, the system is not self-contained, as it relies upon Internet access for the Google Translate service. Therefore, the system is not completely portable.

Nevertheless, we believe this pilot study demonstrates the potential of the prototype. It is capable of generating speech with adequate comprehension for English to Korean, and a large number of language combinations are possible. With extra programming, even more languages can be accommodated. By converting text into phonemes, adjusting the pitch and pauses, Swahili can be spoken using the English text-to-speech module, for example. Thus, the other 30 or so languages currently available with Google Translate could be accommodated. With the use of 11 different SR language modules from Via Voice (Chinese, Dutch, English, French, German, Italian, Japanese, Korean, Portuguese, Spanish, and Swedish), up to 11 x 50 = 550 language-pair combinations can be achieved.

In addition, even if the system cannot currently speak some languages, such as Arabic and Hindi, it is capable of generating text in these tongues. People can generate more words speaking with SR than typing, and the translated text could be read rather than heard.

This paper has described a prototype system for machine interpretation that currently supports 20 language-pair combinations, but has the potential for up to 550. In a test of English to Korean interpretation, there were several errors due to poor speech recognition and translation, but the spoken result was still comprehensible enough. Other language combinations could produce worse results, but many are likely to produce better, especially when interpreting between Western European languages.

1. Aiken, M. and Park, M. (2010). The Efficacy of Round-trip Translation for MT Evaluation. Translation Journal, 14(1), January.
2. Aiken, M., Park, M., Simmons, L., and Lindblom, T. (2009). Automatic Translation in Multilingual Electronic Meetings. Translation Journal, 13(9), July.
3. Aiken, M. and Wong, Z. (2001). The influence of textual complexity on automatic speech recognition. Proceedings of the 32nd Annual Meeting of the Decision Sciences Institute, San Francisco California, November 17-20.
4. Broughton, M. (2002). Measuring the accuracy of commercial automated speech recognition systems during conversational speech. Virtual Conversational Characters (VCC) Workshop, Human Factors Conference, Melbourne, Australia, December.
5. Carter, D., Eklund, R., Kirchmeier-Andersen, S., Becket, R., MacDermid, C., Philp, C., Rayner, M., Wiren, M. (1997). Translation Methodology in the Spoken Language Translator: An Evaluation, Proceedings of a Workshop Sponsored by the Association of Computational Linguistics and by the European Network in Language and Speech (ELSNET), 73-82.
6. Dragon (2009). Dragon Naturally Speaking.
7. Garreau, J. (2009). Tongue in check. Washington Post (DC) (05/24/09).
8. Martins, C. (2006). Translation software put to the test. PC World.
9. Matsubara, S., Toyama, K., and Inagaki, Y. (1999). Sync/Trans: Simultaneous machine interpretation between English and Japanese. Advanced Topics in Artificial Intelligence. Springer: Berlin.
10. Morimoto, T., Takezawa, T., Yato, F., Sagayama, S., Tashiro, T., Nagata, M., and Kurematsu, A. (1993). ATR's speech translation system: ASURA. EUROSPEECH 1993 Third European Conference on Speech Communication and Technology. Berlin, Germany, September 22-25.
11. Nakamura, S., Markov, K., Nakaiwa, H., Kikui, G., Kawai, H., Jitsuhiro, T., Jin-Song Z., Yamamoto, H., Sumita, E., and Yamamoto, S. (2006). The ATR multilingual speech-to-speech translation system. IEEE Transactions on Audio, Speech, and Language Processing, 14(2), 365-376.
12. Och, F. (2009). 51 Languages in Google Translate. Google Research Blog.
13. Ogden, B. (2005). Computer mediated multilingual translation. Computing Research Lab, New Mexico State University.
14. Okumura, A., Iso, K., Doi, S., Yamabana, K., Hanazawa, K., and Watanabe, T. (2002). An automatic speech translation system for travel conversation. NEC Research and Development, 43(1), 37-40.
15. Papineni, K., Roukos, S., Ward, T., and Zhu, W. (2002). BLEU: A method for automatic evaluation of machine translation. Proceedings of the 40^th Annual Meeting of the Association for Computational Linguistics (ACL), Philadelphia, July, 2002, 311-318.
16. Pastor, M., Sanchis, A., Casacuberta, F., and Vidal, E. (2001). EUTRANS: A speech-to-speech translator prototype. Proceedings of EUROSPEECH 2001, 2385-2388.
17. Wahlster, W. (2009). Mobile speech-to-speech translation of spontaneous dialogs: An overview of the final Verbmobil system.
18. Waibel, A. (1996). Interactive translation of conversational speech. Computer, 29(7), 41-48.