AdvancedModelingModelBasedTestGenerationand.ppt

SASQAG 16Jan03 AsmL Modelling Advanced ModelingModel Based Test Generationand Abstract state machine Language Keith Stobie Test Architect XML Web Services Microsoft Corporation  What’s a Model? A model: Is an abstraction or simplified representation of the system from a particular perspective Supports investigation, discovery, explanation, prediction, or construction May be expressed as a description, table, graphical diagram, or quantitative mathematical model Is not necessarily comprehensive Models in Everyday Life Examples of Models Flow Charts Data Flow Diagrams Entity- Relationship Diagrams State Diagrams Deployment Diagrams Class Diagrams Use Cases Activity Diagrams State Transition Tables Model Based Test Generation NOT just model based input generators (pair-wise, random data, grammars, etc.) NOT modeling tools (e.g. UML) Modeling Benefits (GOTCHA) Testing pilot amounted to 10 person months, including the GOTCHA learning curve. This time is less than the time used to originally test the subsystem (12 person months). revealed 2 documentation changes defects 15 of the 18 defects found by the original function test effort would also have been found by this pilot test effort.”From: Using a model-based test generator to test for standard conformance Benefits (TestMaster) “In a typical application of this approach, test engineer productivity has increased by a factor of five to ten over conventional manual approaches (Figure 5)”.From: Model Based Testing FSM: Start State action other States Simple FSM Model File-in-a-directory Model Walk Every State-Change File-in-a-directory Model Test Every Walk 5 Steps Test Model Toolkit Test Model Toolkit - Impact Used by over 20 teams at MS Time to automate fell by as much as 88% BizTalk - Took 1 week to generate a set of test cases that took 8 weeks by hand Code coverage increased by as much as 50% One subteam in Shell – In 2 weeks increased code coverage from 20% to 75%, while the number of test ca