achievement measures, mathematics, graphing technology, graphing calculators, college level mathematics, mathematics classes, achievement, experimental group students, control groups, mathematics instruction, the control group, Dissertation Abstracts International, procedural knowledge, treatment group, conceptual understanding, student achievement, implementing technology, technology, results, References Dunham
What Does the Research Say about Achievement of Students Who Use Calculator Technologies and Those Who Do Not? Susan Barton, Ph.D. Brigham Young University Brigham Young University-Hawaii Campus Department of Mathematics [email protected]
Introduction For more than 30 years calculators or computers have been available to teachers and students in the classroom. And for almost as long, research has been conducted to assess the effect of the calculator or computer on student learning. Starting in the late 1970s, groups of studies were gathered and analyzed to determine the effect of calculator use in mathematics classrooms. For example, Suydam (1976, 1980) located 75 studies from the late1960s through the 1970s relating the effects of calculator use on mathematics education. The studies addressed the areas of achievement in traditional instruction, achievement within a special curriculum and student attitude toward mathematics. Since some of the studies yielded more than one finding, over 95 comparisons were made. In 47 of the comparisons, no significant difference was found. The treatment group (calculators) scored higher on test scores than the control group (non-calculator) in 43 of the comparisons, while the control group scored higher than the calculator group in only five of the comparisons. The studies were primarily at the elementary school level. Suydam's findings suggest the use of calculators do not adversely affect student achievement, and can actually result in higher achievement than with non-calculator usage. Suydam also found when calculators are used that some mathematical concepts such as estimation and long division simplification could be introduced to students at an earlier age. She also noted resequencing of topics may be possible, such as decimals before fractions, with the use of calculators. Hembree and Dessart (1986) combined the information from Suydam and other studies into one meta-analysis. Over 70 studies with quantitative data comparing calculator-based instruction to traditional instruction were used in the analysis. About half of the studies found no significant difference in the achievement of students who use calculators compared to those who did not use calculators. However, the results of the analysis on overall achievement found that most grade levels were significantly and positively affected by the use of calculators even though many of the studies did not allow calculators on the exam. With the exception of 4th grade, results of the metaanalysis found average ability students at all grade levels who used calculators performed significantly better than the non-calculator group on computation and problem solving. For average ability students in the 4th grade, the non-calculator group performed slightly better than the calculator group in computation. However, the average ability students in 4th grade in the calculator group outperformed the non-calculator group in problem solving. No apparent effects were observed for low or high ability fourth-grade students in either computation or problem solving. Smith (1996) found over 30 studies that were completed after Hembree's review. Smith's review included studies in grades K - 12 from 1984 to 1995. Results found significantly higher achievement for students who used calculators for problem solving, computation, and conceptual understanding compared to students who did not use calculators. A significant difference also existed in the attitudes of students favoring those who used calculators in mathematics classes
when compared to the attitudes of those who did not use calculators. Positive significant differences were found in the overall achievement of students in grades three, seven, eight, nine and 10 who used calculators in mathematics classes as opposed to those who did not use calculators. No significant difference was demonstrated in the overall achievement of students in grades four, five, six and 11. By the late 1980s graphing calculators began to appear more frequently in mathematics classrooms; so, Smith included eight secondary school comparison studies involving the graphing calculator. Analysis of the studies found no significant difference in achievement between students who use a graphing calculator to graph mathematical functions and those who did not. One study of 12th graders found the students not using graphing calculators scored higher on achievement measures than the graphing calculator group. Following the model of Hembree, King (1997) performed a meta-analysis to determine the effect of computer-enhanced instruction on college level mathematics. Thirty studies (with 68 effect sizes) were collected from dissertations and journal articles published from 1986 to 1995, which met the criteria for a meta-analysis. Mathematics topics included functions, algebra, linear programming, finite mathematics, statistics, and business, applied, and science calculus. Computer enhanced instruction in the studies included teacher demonstration using a single computer and a classroom display unit, student use of a graphing or programmable calculator, or students (singly or in pairs) using microcomputers in a laboratory setting. Results of the analysis include: (a) a statistically significant positive influence was found on overall achievement when the computer or graphing calculator were used (b) no significant effect was found between technology and control groups on procedural achievement, however, a significant favorable effect resulted on procedural achievement when the experimental group students were allowed to use technology during testing (d) when experimental group students were denied use of technology on tests, procedural achievement (though not statistically significant) was adversely affected, (e) instructional use of computers and graphing calculators as both tool and for demonstration was the most beneficial to all achievement, (f) access to graphing calculator only in the classroom or lab had a slightly adverse effect on conceptual achievement. Since Smith's (1996) analyses and King's (1997) review were completed, more than 60 studies investigating the impact of graphing utilities on mathematics instruction have been conducted. Those studies, from the past decade, that examined the effect of graphing technology (including computer algebra systems CAS) with control groups not using the technology were compiled. Mathematical concepts of algebra through calculus in both high school settlings and college courses were included in the review. Results discussed in this paper include student overall achievement, conceptual understanding, and procedural knowledge. Criteria for inclusion in the review included: (a) comparison of experimental and control groups on achievement measures, (b) more than 10 students participated in each of the treatment and comparison groups, (c) at least part of the treatment included computer or graphing calculator use, (d) the mathematical content assessed came from courses of algebra through calculus, and (e) the study must have gone through a refereeing process. Eight studies for this review came from Smith (1996), sixteen appropriate studies came from King (1997), and 28 more were gathered from computer assisted searches of Dissertation Abstracts International, Education Abstracts, British Education Abstract, ERIC, and Humanities and Social Science Abstracts. Fifty-two studies were found to meet all the criteria for this review; 5 at the beginning algebra level, 4 high school Algebra II, 9 high school precalculus, 3 high school calculus, 4 college level elementary or intermediate algebra, 14 college algebra, 5 college
precalculus (including trigonometry), and 8 college calculus studies. The studies included 40 dissertations, 3 master theses, 7 journal articles, and 2 proceedings articles. For purposes of this review, treatment and /or experimental group is defined to be the group in which at least part of the treatment included the use of graphing technologies or CAS. Control and/or comparison group is defined to be the group that was typically taught in a traditional manner and did not use the graphing technology. Results for Overall Achievement One question of major interest concerning the use of technology in mathematics courses is how the overall achievement of students who use graphing technology or CAS as an aid to learning compared with students not using the technology. To address this issue, 46 of the studies located for this review contained information on overall achievement of the students in the treatment groups and comparison groups. Twenty-nine of the studies found statistically significant overall achievement favoring the treatment group while only one study found statistically significant overall achievement in favor of the comparison group. Thirteen of the studies found no significant differences in overall achievement between the technology enhanced courses and the control courses. See Table 1. Table 1. Overall Achievement Results
Course MS/HS Begin. Alg. HS Algebra II HS Precalculus HS Calculus Coll. Elem./Int. Alg. College Algebra College Precalculus College Calculus TOTAL
Num. of Studies 5 4 9 3 4 14 5 8 52
Treatment Higher 4 3 6 2 6 4 4 29
No Sign. Difference 1 1 3 1 2 2 1 2 13
Control Higher 1 1
Not Interaction Reported
Six of the studies specifically mentioned comparing teaching mathematics courses using graphing calculators with courses using only scientific calculators. Three of the six found no significant difference in overall achievement. Two of the studies found significant differences favoring the treatment group. The other study found an interaction between instructor and calculator type; therefore the researcher gave separate results for the two instructors. Results showed one instructor's class had no significant difference between scientific calculator and graphing calculator groups on overall achievement while results for the other teacher were statistically significant favoring the graphing calculator group (neither were included in Table 1). Interactions were found in two other studies where additional treatments (besides graphing calculator versus non-graphing calculator) were investigated. Because of the interactions, the researchers were unable to obtain an overall effect concerning achievement when using graphing technology. Six other studies did not report overall achievement, but did provide achievement measures for conceptual understanding and/or procedural knowledge.
Conceptual Understanding and Procedural Knowledge Results One of the prevalent claims for the use of graphing technology in mathematics courses is the improvement of conceptual understanding and visualization of mathematical concepts. Thirty-two studies investigated conceptual understanding and/or spatial visualization of mathematical concepts. Eighty-eight different results were provided concerning conceptual understanding (including problem solving and visual thinking). There were 66 statistically significant results favoring the experimental / treatment group while one study reported two results on conceptual understanding that favored the control group. Twenty results indicated no significant difference between the experimental group and the comparison group on conceptual understanding. Table 2. Results of Conceptual and Procedural Assessment Measures
Treatment No Sign. Control Treatment No Sign. Control
Greater Diff. Greater Greater
MS/HS Beg. Algebra
HS Algebra II
Coll. Elem/Inter Alg.
*Each number in Table 2 represents a study; its value represents the number of results reported within the study.
E.g., 1, 2, 3, 3 means there were four studies in that category with respectively, 1, 2, 3, and 3 results reported.
Another area of paramount interest when technology is used in mathematics courses is the effect it may have on students' ability to acquire the paper and pencil skills often referred to as procedural knowledge. Twenty-eight studies examined procedural knowledge or skills acquisition. There were 51 different results given. Nine results (from 7 studies) favored the treatment group, nine results (5 studies) favored the control group and 33 results (18 studies) found no significant difference in symbolic manipulation of algebra or calculus procedures. See Table 2. Conclusion
If the descriptive statistics supplied by the 52 comparison studies are considered at face value, the benefits of the use of technology on student achievement are evident. More than two-thirds of the studies compiled for this paper reported better overall achievement for the treatment group (graphing technology and/or CAS) and 75% of the results on measures testing for conceptual understanding favored the treatment group while nearly two-thirds of the results on procedural knowledge indicated no significant difference between the control group and the treatment group. Results from the review of research are encouraging. Although the evidence supporting the use of graphing technology and computer algebra systems is not unanimous, it does strongly suggest that
when used appropriately these technologies do assist in increasing conceptual understanding without adversely affecting procedural knowledge. It is interesting to note that a few of the studies attempting to isolate the effect of technology on student achievement found the graphing technology group demonstrated better conceptual understanding of the topic being tested. Yet, the majority of the studies seeking to isolate the technology variable, by controlling curriculum, text, homework, exams, and teacher variables, did not find a significant difference in overall achievement between the treatment group and the control group. These findings suggest that simply having access to technology does not insure it will be used to enhance learning. From the results of his study, Ruthven (1990) suggested the impact of the technology in the secondary classroom might depend as much on the ways in which the technology is used to mediate mathematics in the classroom as on simple access to the technology. Dunham and Dick (1994) also noted the mere presence of graphing technology may not account for the positive results that have been found in studies. Rather, the combination of changes in curriculum and instruction with the use of graphing technology should be examined. Regarding teaching practices, Stick (1997) noted that when implementing technology in teaching college mathematics, those instructors who regularly put some emphasis on class discussion had fewer adjustments to make than those who used a lecture-only format. Space restrictions will not permit elaboration on teaching strategies facilitated with access to graphing technology. Suffice it to say that approaches to teaching and learning which emphasize problem solving, foster visualization and exploration of concepts, student participation, and which allow students to actively construct meaning for the mathematics they encounter, find in graphing technology a natural and mathematically powerful partner. References Dunham, P. H. & T. Dick (1994). Research on graphing calculators. Mathematics Teacher, 87(6), 440-445. Hembree R. and D. Dessart. (1986). Effects of hand-held calculators in precollege mathematics education: A meta-analysis. Journal for Research in Mathematics Education, 17, 83-99. King H. J. (1997). Effects of computer-enhanced instruction in college level mathematics as determined by a meta-analysis. (Doctoral dissertation, The University of Tennessee, 1997). Dissertation Abstracts International, 59(1), 114A. Ruthven, K. (1990). The influence of graphic calculator use on translation from graphic to symbolic forms. Educational Studies in Mathematics, 21, 431-450. Smith, B. A. (1996). A meta-analysis of outcomes from the use of calculators in mathematics education. (Doctoral dissertation, Texas A & M University, 1995). Dissertation Abstracts International, 57, 787A. (University Microfilms No. DA9724626). Stick, M. E. (1997). Calculus reform and graphing calculators: A university view. Mathematics Teacher, 90(5), 356-60, 63. Suydam, M. N. (1980). The use of calculator in precollege education: A state of the art review. Columbus, OH: Calculator Information Center. Suydam, M. N. (1976). Electronic hand calculators: The implication for precollege education. Final Report. Washington, D. C: National Science Foundation. Space constraints do not allow all of the studies reviewed in this paper to be listed. Please contact the author for a complete list of the studies.