Usability Evaluation slides.pptVIP

Formative v. Summative evaluation III. What is a Usability Experiment? How many test users? Cost-benefit analysis Summative EvaluationWhat to measure and it’s relationship to usability elements Measuring User Performance Measuring User Performance cont. Reliability and Validity Formative EvaluationWhat is a Usability Problem?? Qualitative methods for collecting usability problems Conducting a usability experiment –steps and deliverables The planning phaseOutput: written plan or proposal Outline of Test Protocol Designing Test Tasks Detailed Test Protocol Execution phase Execution phase: ethics of human experimentation Data collection - usability labs and equipment What can statistics tell us? Testing usability in the field Testing Usability in the Field cont. What tasks? Criteria for completion? User aids What will users be asked to do thinking aloud studies ? Interaction with experimenter What data will be collected? Tasks: Are representative Cover most important parts of UI Don’t take too long to complete Goal or result oriented possibly with scenario Not frivolous or humorous unless part of product goal First task should build confidence Last task should create a sense of accomplishment Deliverables from detailed test protocol *What test tasks? written task sheets *What user aids? written manual *What data collected? include questionnaire How will results be analyzed/evaluated? sample tables/charts All materials to be given to users as part of the test, including detailed description of the tasks. Pilot test protocol with a few users Prepare environment, materials, software Introduction should include: purpose evaluating software voluntary and confidential explain all procedures recording question-handling invite questions During experiment give user written task description s , one at a time only one experimenter should talk De-briefing Users feel exposed using unfamiliar tools and making erros Guidelines: Re-assur