          |
A Meta-Analysis of the Effectiveness of Teaching and Learning With Technology on Student OutcomesIntroductionEducation often has been characterized as the only field where personal experience and ideology are relied on to make policy choices because the research base is inadequate and rarely used (National Research Council, 1999). The federal No Child Left Behind Act of 2001, however, is placing a new emphasis on scientifically based research and is requiring states and school districts to choose "evidence-based" programs for their schools and classrooms. This change is providing support to the growing numbers of researchers (Glass, 2000) and organizations, such as the Campbell Collaboration (2002), which use the statistical technique of meta-analysis to synthesize findings from research. It is argued that these systematic reviews of the research will firm up the "soft science" of education and finally begin to provide empirical evidence that certain programs or approaches are effective in improving student outcomes (Viadero, 2002). During the past three decades, a large number of meta-analyses have systematically examined the effects of technology on student outcomes. Several meta-analyses, for example, have investigated the impact of computer-assisted instruction on student outcomes (Lipsey & Wilson, 1993). Other meta-analyses have examined aspects such as the effects of microcomputer applications in elementary schools (Ryan, 1991) and the effects of computer programming on student outcomes (Liao & Bright, 1991). Niemiec and Walberg (1992) summarized the findings on 13 quantitative research syntheses that were conducted between 1975 and 1987 and found that the average effect size was .42, which indicated that the average student who received computer-based instruction scored at the 66th percentile of the control group distribution (i.e., the 50th percentile). Overall, these meta-analyses—along with some recent, major studies and narrative reviews of the research—have documented the positive effects of educational technology on student achievement (Schacter, 2001; Sivin-Kachala, 1998; Wenglinsky, 1998). These studies, reviews, and meta-analyses, however, typically look at different aspects or types of technology. Furthermore, this knowledge base has not really provided information on how to appropriately integrate and use technology in schools and classrooms. In addition, recent improvements regarding the quality and quantity of technology in schools suggest that technology in schools today is dramatically different from the technology that used in schools several years ago. This rapid growth and improvement in technology exceeds current knowledge of how to effectively use technology in schools (Allen, 2001) and suggests that the impact of technology is different today than it was in the past. Although many of the meta-analyses examining the effects of technology on student outcomes were conducted more than a decade ago, several recent meta-analyses have focused on specific aspects of technology. Blok, Oostdam, Otter, and Overmaat (2002), for example, examined the effectiveness of computer-assisted instruction (CAI) programs in supporting beginning readers. Their review included 42 studies from 1990 onward, and they found the corrected overall effect size estimate was .19. Their findings were similar to earlier meta-analyses by Kulik and Kulik (1991) and Ouyang (1993), which also examined the effects of CAI and found it to have positive but small effects. Lou, Abrami, and d'Apollonia (2001) examined the effects of students working in a small group versus working individually when students were using computer technology. They found that small-group learning had more positive effects than individual learning. Other recent meta-analyses in technology have examined topics such as the effectiveness of interactive distance education (Cavanaugh, 2001), computer-assisted instruction in science education (Bayraktar, 2001-2002), and computer-based instructional simulation (Lee, 1999). Furthermore, other recent meta-analyses have examined the effects of computer-assisted instruction on student achievement in differing science and demographic areas (Christmann & Badgett, 1999), microcomputer-based computer-assisted instruction within differing subject areas (Christmann, Badgett, & Lucking, 1997), gender differences in computer-related attitudes and behavior (Whitley, 1997), and the effectiveness of computer-assisted instruction on the academic achievement of secondary students (Christmann, Lucking, & Badgett, 1997). Some recent meta-analyses that have not yet been published have focused on the uses of educational technology in home and school (Penuel et al., 2002) and discrete educational software (Murphy, Penuel, Means, Korbak, Whaley, & Allen, 2002). Table 1 presents a summary of nine recent meta-analyses in the area of educational technology that have been published in peer-reviewed journals. The median effect size of the seven reported effect sizes is .21, which represents a small positive effect with the experimental group scoring at the 58th percentile of the control group distribution. These meta-analyses, however, also found that their particular treatments had several differential effects on their outcomes. Lee (1999), for example, found that although computer-based simulation had a modest, positive effect size of .41 on student achievement, it had a negative effect size of -.04 on student attitudes. The ability to examine differential effects of the treatment is one of the many advantages of meta-analysis as a meaningful method to aggregate and report educational findings.
One area in which there have not been many meta-analyses and systematic reviews of the research is how teaching and learning with technology impacts student outcomes. This area is important because some studies have found that technology can change teachers' pedagogic practices from a teacher-centered or teacher-directed model to a more student-centered classroom where students work cooperatively, have opportunities to make choices, and play an active role in their learning. Swan and Mitrani (1993), for example, compared the classroom interactions between high school students and teachers involved in (a) computer-based instruction and (b) traditional instruction. They found that student-teacher interactions were more student-centered and individualized during computer-based teaching and learning than in traditional teaching and learning. In another study that examined changes in classroom instruction as a result of technology, Sandholtz, Ringstaff, and Dwyer (1992) found that high access to computers enabled teachers to individualize instruction more. In a national study, Worthen, Van Dusen, and Sailor (1994) found that students using a computerized integrated learning system (ILS) in both laboratory and classroom settings were more actively engaged in learning tasks than students in the non-ILS classrooms.
Waxman and Huang (1996) similarly found that instruction in classroom settings where technology was not often used tended to be whole-class approaches, in which students generally listened or watched the teacher. Instruction in classroom settings where technology was moderately used had much less whole-class instruction and much more independent work. Another important finding from the Waxman and Huang (1996) study is that students in classrooms where technology was moderately used (more than 20 percent of the time) were found to be on task significantly more of the time than students from the other two groups-in which technology was infrequently used (less than 10 percent of the time) or in which technology was slightly used (11 percent to 19 percent of the time). These findings are similar to prior studies that found that computer-based instruction increases students' time-on-task (MacArthur, Haynes, & Malouf, 1986; Schofield & Verban, 1988; Worthen, Van Dusen, & Sailor, 1994). Although these individual studies have examined how technology impacts the teaching and learning process, little is known about how this intervention impacts student outcomes.
|
|||||||||||||||||||||||||||||||||||||||||||||||
 |
 Contact Us | Privacy Policy |