Session2SpecifyingtheConceptualandOperationalModels.pptVIP

IES Workshop on Evaluating State and District Level Interventions Mark W. Lipsey Director, Center for Evaluation Research and Methodology Vanderbilt University David Holdzkom Assistant Superintendent for Evaluation and Research Wake County Public School System, North Carolina Purpose To help schools, districts, and states design and implement rigorous evaluations of the effects of promising practices, programs, and policies on educational outcomes. Why encourage locally initiated impact evaluation? Many interventions are not effective; users and interested others need to know. The interventions most relevant to improving outcomes are those that schools and districts believe are promising and feasible. IES has funding to support research initiated by schools, districts, and states. What kinds of interventions might be evaluated? Practices, e.g., one-on-one tutoring, educational software, acceleration of high ability students, cooperative learning. Programs, e.g., Reading Recovery, Ladders to Literacy, Cognitive Tutor algebra, Saxon Math, Caring School Community (character education). Policies, e.g., reduced class size, pre-K, alternative high schools, all year calendar. Key Issues in Designing Impact Evaluations for Education Interventions Logic Models, Variables, and Evaluation Questions Logic model: 1. Specifying the problem the intervention addresses Nature of the need: What and for whom (e.g., kindergarten students who aren’t ready for school). Why (e.g., poor pre-literacy skills, inappropriate school behavior). Rationale/evidence supporting the intervention target (e.g., at entry K students need to be ready to learn or they will begin to fall behind; research shows school readiness can be enhanced for at-risk 4 year olds). Logic model: 2. Specifying the planned intervention What the intervention does that addresses the need: Content: What the students should know or be able to do; why this meets the need. Pedagogy: Instructional techniques and meth