Being an Informed Consumer of Quantitative Educational Research

Author: Fashola, O.S.
Publisher: Phi Delta Kappa
Publication Date: 2004, March
Journal: Phi Delta Kappan
Journal Volume: 85(7)
Pages: 532-538

Abstract (written by WestEd)

Fashola describes in simple terms what scientific research is, how to judge its quality, and how to relate findings to the educator's own school or district context. The article starts with the author making a case for the importance of looking to the evidence on the effectiveness of existing programs for improving the achievement of students at all schools, but she cautions the consumer to look for outcomes at schools and districts with similar student and teacher demographics, staff capacity, etc.

The No Child Left Behind (NCLB) act encourages the use of school reform programs based on research that provides evidence of effectiveness, such as replicability, rigorous evaluation designs, fair measures, and improvement in student academic outcomes. The ideal evaluation design is a true experimental design with treatment and control groups, which employs random assignment of students to study conditions. An alternative is a quasi-experimental design, where students are matched on relevant variables (e.g., race, gender, socioeconomic status, English-language proficiency) and assigned to treatment or control groups. [Statistical control of pretreatment differences between groups during data analysis is an alternative to matching during group assignment but is not mentioned.]

Fashola also describes what to look for in terms of student assessments used in the study, characteristics of the research subjects, and how to interpret differences in scores. She explains in clear language both "statistical significance" (e.g., p<.05), which indicates the level of confidence that the results are not due to chance, and "effect size," which is an indicator of the degree to which the groups differ. A general guide for the latter is that .30 is a small effect size, .50 is a medium effect size, and .80 is a large effect size.

Last, factors contributing to program effectiveness are discussed, including clear goals, regular assessments, professional development, and support.