Acquiring Grounded Representations of Words with Situated….pdfVIP

Advances in Cognitive Systems 2 (2012) 113-130 Submitted 09/2012; published 12/2012 Acquiring Grounded Representations of Words with Situated Interactive Instruction Shiwali Mohan SHIWALI @UMICH .EDU Aaron H. Mininger MININGER @UMICH .EDU James R. Kirk JRKIRK @UMICH .EDU John E. Laird LAIRD @UMICH .EDU Computer Science and Engineering, University of Michigan, Ann Arbor, MI USA Abstract We present an approach for acquiring grounded representations of words from mixed-initiative, situated interactions with a human instructor. The work focuses on the acquisition of diverse types of knowledge including perceptual, semantic, and procedural knowledge along with learn- ing grounded meanings. Interactive learning allows the agent to control its learning by requesting instructions about unknown concepts, making learning efficient. Our approach has been instantiated in Soar and has been evaluated on a table-top robotic arm capable of manipulating small objects. 1. Introduction A goal of our research is to develop autonomous agents that can dynamically extend not only their knowledge of the world and their ability to interact with it, but also their use of language for inter- acting with humans. We describe initial progress in developing such an agent that learns grounded representations of adjectives, nouns, prepositions, and verbs through interactive situated instruction with a human mentor while performing a task. The agent is developed in Soar (Laird, 2012) using the same architectural mec