evaluation methods for learning environments.pptVIP

Evaluation Methods for Learning Environments Shaaron Ainsworth School of Psychology Learning Sciences Research Institute University of Nottingham Contentious Claim? AIED systems die, the only thing you can hand on to the next generation is information about the success (or lack of) of a current system Without evaluation, there is no point in doing anything….. Me Why am I doing this? Background Bsc (Hons) Psychology MSc Artificial Intelligence PhD Educational Cognitive Science Research Interests The unique properties of learning with more than one representation ITS authoring tools Conducted circa 20 evaluation studies in the last 10 yrs Main interest – how can we understand more about human complex information processing by studying learners interacting with innovative technologies Today Why evaluate What questions should you ask to design an evaluation What do I want to achieve What can I measure What is an appropriate design? What should I compare my system to? What is an appropriate context in which to evaluate Misc issues Summary and Conclusions Why evaluate? To improve usability To enhance learning outcomes To increase learning efficiency To inform theory To increase user acceptance To sell your ILE To help develop the field Times they are a changing Questions to answer What do I want to do with the information Informing design Choosing between alternatives Credit assignment problem Informing about context of use What are appropriate forms of measurement? What is an appropriate design? What is an appropriate form of comparison? What is an appropriate context Two main types To inform design Formative evaluation E.g. Heuristic Evaluation, Cognitive Walkthrough http://www.psychology.nottingham.ac.uk/staff/sea/c8cxce/handout4.pdf Should the same usability heuristics be used in educational systems as are used in other computer-based systems E.g. Squires Preece (1999), Gilmore (1996) To assess end product To assess end product or discover how it should be