User Experience not Metrics.pdfVIP

User Experience, not Metrics Page 1 www.PerfTestP ? 2006 PerfTestPlus, Inc. All rights reserved. User Experience, not Metrics; Performance Engineering Methodology First Presented for: Northern Virginia Rational Users Group Reston, VA 2002 Scott Barber Chief Technology Officer PerfTestPlus, Inc. User Experience, not Metrics Page 2 www.PerfTestP ? 2006 PerfTestPlus, Inc. All rights reserved. Agenda What are Performance Metrics? What are User Experience Measurements? Modelling Real Users and User Communities Collecting Meaningful Times and Statistics Interfacing with Developers and Architects Iterative Tuning Reporting to Stakeholders Summary / Questions User Experience, not Metrics Page 3 www.PerfTestP ? 2006 PerfTestPlus, Inc. All rights reserved. What are Performance Metrics? System Component Statistics Often misleading Generally impressive sounding Generally NOT useful in making “Go Live” decisions Often Misused and Misunderstood Not an indication of interaction with other system/application tiers No direct correlation to user experience Not intuitively valuable User Experience, not Metrics Page 4 www.PerfTestP ? 2006 PerfTestPlus, Inc. All rights reserved. What are Performance Metrics? Common Metrics Include – Throughput ? Number of bytes a Web Server can serve in a unit of time ? Useful in comparing Web Server Hardware and Software – Hits per Second ? How many requests can a Web Server process in a second ? Useful in comparing Web Server Hardware and Software – Bandwidth ? Same as Throughput – usually referring to networks or internet pipelines User Experience, not Metrics Page 5 www.PerfTestP ? 2006 PerfTestPlus, Inc. All rights reserved. What are Performance Metrics? Common Misconceptions – Scenario: Throughput is 200kbps. Average page size is 50kb. ? Assumption: Site can process 4 page requests per second (or 14,400 per hour) with sub-second response time. ? Truth: Thro