Advances in Distributed Computing and Artificial Intelligence Journal

Salamanca University Press Journal

Developing multimodal conversational agents for an enhanced e-learning experience

General information

Authors:
David GRIOL, Jose Manuel MOLINA, Araceli SANCHÍS DE MIGUEL

DOI:
10.14201/ADCAIJ2014381326

Volume:
Regular Issue 3 (1), 2014

Keywords:

Conversational agents; Multimodal interaction; Chatbots; Speech; E-learning

Conversational agents have become a strong alternative to enhance educational systems with intelligent communicative capabilities, provide motivation and engagement, and increment significant learning and helping in the acquisition of meta-cognitive skills. In this paper, we present Geranium, a multimodal conversational agent that helps children to appreciate and protect their environment. The system, which integrates an interactive chatbot, has been developed by means of a modular and scalable framework that eases building pedagogic conversational agents that can interact with the students using speech and natural language.

References

Abu-Shawar, B., Atwell, E. Fostering language learner autonomy via adaptive conversation tutors. Proc. of Corpus Linguistics, 2007, pp. 1-8.

Aleven, V., Ogan, A., Popescu, O., Torrey, C., Koedinger, K. Evaluating the Effectiveness of a Tutorial Dialog System for Self-Explanation. Proc. of 7th Int. Conference on Intelligent Tutoring Systems (ITS’04), 2004, pp. 443- 454

Bickmore, T-W. Relational Agents: Effecting Change through Human- Computer Relationships. PhD Thesis, Media Arts & Sciences, Massachusetts Institute of Technology, 2003

Bos, J., Klein, E., Lemon, O., Oka, T. The Verbmobil prototype system - a software engineering perspective. Journal of Natural Language Engineering. 5(1), 1999, 95-112

Cassell, J., Sullivan, J., Prevost, S., Churchill, E-F. Embodied Dialog systems, The MIT Press, 2001

Cavazza, M., de la Camara, R-S., Turunen, M. How Was Your Day? a Companion ECA. Proc. of AAMAS’10 Conference, 2010, pp. 1629-1630.

Chou, C-Y., Chan, T-W., Lin, C-J. Redefining the Learning Companion: the Past, Present and Future of Educational Agents. Computers & Education, 40, 2003, 255-269

Corchado, J., Tapia, D., Bajo, J.: A multi-agent architecture for distributed services and applications. Computational Intelligence 24(2), 2008, 77–107

De Rosis, F., Cavalluzzi, A., Mazzotta, I., Noviell,i N. Can embodied dialog systems induce empathy in users? Proc. of AISB'05 Virtual Social Characters Symposium, 2005, pp. 1-8

Dominguez, K. VoiceXML 31 Success Secrets - 31 Most Asked Questions On VoiceXML - What You Need To Know. Emereo Publishing, 2014

Dowding, J., Clancey, W-J., Graham, J. Are You Talking to Me? Dialogue Systems Supporting Mixed Teams of Humans and Robots. Proc. of AIAA Fall Symposium "Annually Informed Performance: Integrating Machine Listing and Auditory Presentation in Robotic Systems, 2006, pp. 22-27

Edlund, J., Gustafson, J., Heldner, M., Hjalmarsson, A. Towards human- like spoken dialog systems. Speech Communication, 50(8-9), 2008, 630- 645

Forbus, K-D., Whalley, P-B., Evrett, J-O., Ureel, L., Brokowski, M., Baher, J., Kuehne, S-E. CyclePad: An articulate virtual laboratory forengineering thermodynamics. Artificial Intelligence, 114(1-2), 1999, 297- 347

Fryer, L., Carpenter, R. Bots as Language Learning Tools. Language Learning and Technology, 10(3), 2006, 8-14

Gorostiza, J-F., Salichs, M-A. End-user programming of a social robot by dialog. Robotics and Autonomous Systems, 59(12), 2011, 1102–1114.

Graesser, A-C., Chipman, P., Haynes, B-C., Olney, A. AutoTutor: An Intelligent Tutoring System with Mixed-initiative Dialog. IEEE Trans. in Education, 48, 2005, 612-618

Graesser, A-C., Person, N-K., Harter, D. Teaching Tactics and Dialog in AutoTutor. International Journal of Artificial Intelligence in Education. 12, 2001, 23-39

Gratch, J., Rickel, J., Andre, J., Badler, N., Cassell, J., Petajan, E. Creating Interactive Virtual Humans: Some Assembly Required. Proc. of IEEE Conference on Intelligent Systems, 2002, pp. 54-63

Grigoriadou, M., Tsaganou, G., Cavoura, T. Dialog-Based Reflective System for Historical Text Comprehension. Proc. of Workshop on Learner Modelling for Reflection at Artificial Intelligence in Education, 2003, pp. 238-247

Griol, D., Molina, J., Sanchis, A., Callejas, Z. A Proposal to Create Learning Environments in Virtual Worlds Integrating Advanced Educative Resources. JUCS Journal 18(18), 2012, 2516–2541

Griol, D., Molina, J., Sanchis, A., Callejas, Z. Bringing together commercial and academic perspectives for the development of intelligent AmI interfaces. Journal of Ambient Intelligence and Smart Environments, 4(3), 2012, 183-207

Heffernan, N-T. Web-Based Evaluations Showing both Cognitive and Motivational Benefits of the Ms. Lindquist Tutor. Proc. of Int. Conference on Artificial Intelligence in Education, 2003, pp. 115-122

Kerly, A., Ellis, R., Bull, S. Dialog systems in E-Learning. Proc. of AI’08, 2008, pp. 169-182

Kerly, A., Ellis, R., Bull, S. CALMsystem: A Dialog system for Learner Modelling. Knowledge Based Systems, 21(3), 2008, 238-246

Kerly, A., Bull, S. Children's Interactions with Inspectable and Negotiated Learner Models. Proc. of Int. Conf. on Intelligent Tutoring Systems, 2008, pp. 132-141.

Kumar, R., Rose, C-P. Architecture for Building Dialog systems that Support Collaborative Learning. IEEE Trans. Learn. Technol. 4(1), 2011, 21-34

Latham, A., Crockett, K-A., McLean, D., Edmonds, B. A conversational intelligent tutoring system to automatically predict learning styles. Computers & Education. 59(1), 2012, 95-109

Li, S., Wrede, B. Why and how to model multi-modal interaction for a mobile robot companion. Proc. of AAAI Spring Symposium on Interaction Challenges for Intelligent Assistant, 2007, pp. 71-79

Litman, D-J., Silliman, S. ITSPOKE: An Intelligent Tutoring Spoken Dialog System. Proc. of Human Language Technology Conference: North American Chapter of the Association for Computational Linguistics, 2004, pp. 5-8

López-Cózar, R., Araki, M. Spoken, Multilingual and Multimodal Dialog Systems. Development and Assessment. John Wiley and Sons, 2005.

Marsella, S-C., Johnson, W-L., Labore, C-M. Interactive pedagogical drama for health interventions. Artificial Intelligence in Education: Shaping the Future of Learning through Intelligent Technologies, 2003, pp. 341-348

Matsui, T., Asoh, H., Asano, F., Fry, J., Hara, I., Motomura, Y., Itoh, K. Spoken Language Interface of the Jijo-2 Office Robot. Springer Tracts in Advanced Robotics, 6/2003, 2003, 307-317

Mctear, M-F. Spoken dialog technology. Springer, 2004

Mctear, M.F., Callejas, Z. Voice Application Development for Android Packt Publishing, 2013

Mostow, J. Why and How Our Automated Reading Tutor Listens. Proc of Int. Symposium on Automatic Detection of Errors in Pronunciation Training (ISADEPT), 2012, pp. 43-52

Nakano, M., Hasegawa, Y., Funakoshi, K., Takeuchi, J., Torii, T., Nakadai, K., Kanda, N., Komatani, K., Okuno, H-G., Tsujino, H. A multi- expert model for dialogue and behavior control of conversational robots and agents. Knowledge-Based Systems. 24(2), 2011, 248-256

Pieraccini, R. The Voice in the Machine: Building Computers that Understand Speech. The MIT Press, 2012

Pieraccini, R., Suendermann, D., Dayanidhi, K., Liscombe, J. Are we there yet? Research in Commercial Spoken Dialog Systems, Lecture Notes in Computer Science, 5729, 2009, 3-13

Pinzón, C., Bajo, J., de Paz, J., Corchado, J. S-MAS: An adaptive hierarchical distributed multi-agent architecture for blocking malicious SOAP messages within Web Services environments. Expert Systems with Applications 38(5), 2011, 5486-5499

Pon-Barry, H., Schultz, K., Bratt, E-O., Clark, B., Peters, S. Responding to student uncertainty in spoken tutorial dialog systems. International Journal of Artificial Intelligence in Education. 16, 2006, 171-194

Roda, C., Angehrn, A., Nabeth, T. Dialog systems for Advanced Learning: Applications and Research. In: Proc. of BotShow’01 Conference, 2001, pp. 1-7

Rodríguez, S., de Paz, Y., Bajo, J., Corchado, J.M. Social-based Planning Model for Multi-agent Systems. Expert Systems with Applications 38(10) , 2011, 13005–13023

Rosé, C-P., Moore, J-D., VanLehn, K., Allbritton, A. A Comparative Evaluation of Socratic versus Didactic Tutoring. Proc of Cognitive Sciences Society, 2001, pp. 869-874

Sidner, C-L., Kidd, C-D., Lee, C., Lesh, N. Where to look: a study of human-robot engagement. Proc. of 9th Int. Conference on Intelligent user interfaces (IUI’04), 2004, pp. 78-84

Vaquero, C., Saz, O., Lleida, E., Marcos, J., Canalís, C. Vocaliza: An application for computer-aided speech therapy in Spanish language. Proc. of IV Jornadas en Tecnología del Habla, 2006, pp. 321–326

Wang, N., Johnson, L-W. The Politeness Effect in an intelligent foreign language tutoring system. Proc. of Intelligent Tutoring Systems (ITS’08), 2008, pp. 270 - 280

Wang, Y., Wang, W., Huang, C. Enhanced Semantic Question Answering System for e-Learning Environment. Proc of AINAW’07 Conference, 2007, pp. 1023-1028

Williams, J., Young, S. Partially Observable Markov Decision Processes for Spoken Dialog Systems. Computer Speech and Language, 21(2), 2007, 393–422

How to cite item?