Can you hear me now Keeping threats to validity from muffling 你在能听到我保持威胁的有效性从消声.pptVIP

Can you hear me now? Keeping threats to validity from muffling assessment messages Maureen Donohue-Smith, Ph.D., RN Elmira College Plan for Discussion: Review common “threats to internal validity” Identify how these threats might affect assessment at participants’ own institutions Identify assessment strategies that minimize the impact of these threats on assessment Internal Validity Your ability to say that your program produced the changes that you observed Infers a causal relationship between the program or intervention and the outcome that was NOT caused by other alternative factors Educational Assessment Program Outcomes Threats to Internal Validity What are some of the common alternative explanations that obscure the real relationship between your educational program and students’ learning outcomes? History: Specific events occur between the first and second measurement of program outcomes that could have independent effects on your program outcomes Maturation Changes in program participants that result from individual development and experience over time and not from the program intervention Testing Participation in the first assessment affects performance during later assessments (also known as sensitization or “practice effects”) Instrumentation Measurement tool is inaccurate Wrong instrument selected for the construct you want to measure Pretest and posttest assess different domains Questionnaire not standardized for reliability Inconsistency in data collection procedures or coding Statistical regression: (regression to the mean) Extreme scores on the first test tend to move toward the center or average on the second test Selection Bias Students participating in assessment are not representative of the overall student population  Program Mortality: Program participants drop out differentially