EvaluatingTrustworthinessinEducationalResearchThatis.docVIP

NCITE: National Center to Improve the Tools of Educators Funded by the US Office of Special Education Programs Evaluating Trustworthiness in Educational Research That is Used to Support a Change in Teaching or School Practice A. Find a research review of the practice. Guidelines for Evaluating the Trustworthiness of a Research Review (A research review is more trustworthy than an individual study)A reformer’s claim that one teaching practice should replace another requires research that compares the recommended practice with the one it is to replace.Are the comparative studies reviewed with each individual study described in sufficient detail to clarify the specific nature of the comparison and the results?If the answer is no, disregard the studyIf “effect sizes” are compared, were the effect sizes calculated using the same procedures across all studies? (Reporting effect sizes from other meta-analyses can be misleading. Different rules may be used by different researchers.) If no, disregardDoes the review seem complete? If not all studies were reviewed, are the criteria for the inclusion and the exclusion of studies listed and reasonable?If no, disregardIs there clear evidence that things compared were the same across all studies? (e.g., the “nongraded,” primary refer to either mixed-ability or homogeneous grouping)If no, disregardAre the researchers independent, receiving no financial benefit from the sale of the program or services being evaluated?Be suspicious B. If no research reviews are available, refer to individual studies: Essential Factors in Evaluating a Single StudyWas there a comparison? (Gains in percentile points qualify as a comparison)If no, disregardDo the measures evaluate performance that is important to your school? (e.g., academic or self-esteem goals)If no, disregardAre the measures fair to both treatments? (e.g., if one approach is used to teach arithmetic and the other geometry, a measure of arithmetic would bias