reading achievement were positive, which emphasizes the fact that reducing print distortions, whether by optometric prescription or color, is likely to be only one factor in reading development." (Robinson, 1994, p.10) Model, assumptions, and variables: The program calls for screening an entire classroom of students in general education classrooms, interviewing all of the students, selecting students that meet the criteria of this visual perceptual condition, identifying the corrective color needed to use with all print-based media, and collecting achievement score data on the students in two time periods (once previous to the use of the color screen and once after the introduction of this intervention). Hypotheses: It is hypothesized that the screening for the visual perceptual skills debility and using the colored sheets for reading and for test taking will lead to higher scores relative to the higher scores experienced by the general population of students and the students that were screened as not having the debilitating condition. Background variables that are normally collected in the schools will be held constant. Thus, it is not enough that the scores rise in the treatment group. The scores of this group must rise significantly higher than the scores of the other groups. Tests will be made of the achievement scores of reading, language, and math components of the CTBS/4 and SABE II achievement tests. Moreover, it is also hypothesized that the increase in scores will be greater for the reading and language achievement tests components than for the math computation sub-test components. This differential effect is due the different text spacing on the tests. The reading text is generally in single-spaced lines across the page. The math test questions are shorter in total word count and shorter across the page, usually providing a visual separation between problems. Thus, a student using a colored plastic transparency to provide accurate visual input will gain relatively more clarity, stability, and accuracy in the language and writing portions of the test. Sample of students: A convenience sample of students was selected in the second and third grades at Kingsley and Montvue elementary Schools as part of the Light Sensitivity Project in the Pomona Unified School district. The notes that are regularly taken during the screening exams were used to collect some of the student data, notably, which students were screened with the condition and which wore glasses. Normal curve equivalent achievement scores, other demographic variables, and program participation variables were then selected, matching the students that had both the previous year's achievement scores from the second grade and the present year's score. Thus, the numbers in the samples reflect the continuation of students in the schools over this two year period. approximately 15% of the students had scores in the first year but not the second. There was no attempt to randomly select students for screening; there was also no attempt to randomly assign students to the treatment group. Time frame of data collection, analyzes, and reports: Achievement data collection occurred in April, 1996, and April, 1997. Data collection from the district database occurred in October, 1997. Students were provided the plastic transparencies in late February, 1997. thus, the "treatment" group used the plastic transparencies for a maximum of two months. statistical analysis and preliminary reports were conducted and written in December, 1997. page 2

Statistical tests: A simple t-test is considered inappropriate in this case, as is an analysis of variance procedure. Generally, both of these methods rely exclusively on random assignment of students to the treatment and control groups, which is usually a part of the design of the evaluation. In this case, students were chosen for the treatment group because they were diagnosed as having the condition. A regression analysis is needed to control statistically for all of the possible "blocking conditions" that could not be controlled in the design of the "experiment." Thus, in the presence of all of the group differences among the students, the addition of the diagnosis and treatment using the colored acetates is expected to be significant and positive. This test can be done using the 1997 achievement test scores as the relevant dependent test score, using the group categories (treatment, diagnosis but no treatment, and all other regular education students at that school in that grade level) as independent dummy variables, and controlling for the level of the achievement scores by including the previous year's score as an inDependent variable. The test for the first hypothesis is that the coefficient for the treatment dummy variable is positive, significant and significantly greater than the coefficient for the other dummy variables of interest. The test for the second hypothesis is that the coefficients for the treatment dummy variables for the reading and writing regression models are positive, significant, and greater than the coefficient for the math computation regression model. Results: Overall, this study found supporting evidence for the use of plastic transparencies to aid reading learning. As can be seen from the second and third grade means of students' math, reading, and language scores (see Tables 1 and 2), students in the treatment classes had higher mean NCE scores before and after the use of color transparencies. Moreover, these students gained relatively more during the year in which the use of color transparencies occurred. It is unclear whether the treatment "caused" this increased learning in these subjects, as both the students with plastic transparencies and those without plastic transparencies increased their NCE scores. A better designed evaluation study could clear up this question. The models used identical sets of explanatory (but non-experimental) variables, and the set of models as a whole provided robust results for these explanatory variables. Reviewing Tables 3 and 4, we find that the lagged NCE values were always positive and strong explainers of the dependent variables. Being female had a negative effect for math scores, and were only significant for the computation subtest. LEP categorization had a negative effect for language scores and a positive effect or math scores, principally the computation subtest. Third graders scored higher for language, math, and reading, as expected. GATE identified children scored higher only on reading. Finally, Title I students scores significantly lower only on the math sections, although the signs were negative for the other two other subject areas. Thus, the tables show expected results for all the subject areas, making the models robust in their explanatory results, aside from the experimental variables. As to the first hypothesis, this study found that there was an independent effect for the students who were assigned the plastic transparencies. consider Tables 3 and 4. Using regression analysis, which controls for the independent effects of previous score levels, program participation, and student demographics, we find that participation in the treatment increases scores on the achievement tests for every subject area. This is prima facie evidence that the treatment increases student achievement scores. page 3

Unfortunately, the control group of students, who were in the same classroom as the treatment group but who did not receive the plastic transparencies, also increased their achievement scores. This suggests that the teacher or type of student in the classroom was at least partly responsible for the increased scores. In the table below, all of the relative contributions (beta values) of participating with transparencies were positive, but so are the relative contributions of being in the same class as the "treatment" students but not receiving the transparencies. Participation in the treatment was about two-thirds as important as prior math scores and as important (but in the opposite direction) as participation in the Title I program. A better designed and implemented evaluation study could tease out the effects of these confounding variables.

Model Quality and the Relative Contributions of the Transparencies in Each Model Adjusted R-squares and Beta Values in Each Model

Total Language

Total Reading

Concepts & Applications Math

Total Math

Computation Math

Adjusted R-

.809

.437

.453

.429

.242

squared

Relative

.164

.162

.183

.231

.222

Contribution of

Transparencies,

Beta Value

Relative

.088

.153

.281

.223

.122

Contribution of

only being in

the class with

experimental

group, Beta

Value

As to the second hypothesis, this study found that betas, which are the regression equivalents for effect sizes (standardized coefficients), for the total math battery were nearly 50 % larger than those of either reading or language. (See Tables 3 and 4). The standardized coefficients for reading and language were approximately the same; the standardized coefficient for the Math Computation subtest was higher. This finding was not expected, as we hypothesized that the effects for reading and language would be higher. The standardized coefficient for the Math Concepts and Applications subtest was more like the reading and language coefficients. Moreover, the effect for the "classroom control" students (those students not assigned a transparency but residing in the same classroom as the treatment students) was as high for math and reading but only half as much for language. Again, this points out that something else is happening to the students in the class besides participation in the Light Sensitivity Project.

page 4

We are reluctant to give up on the second hypothesis. It may be that the higher error in the math models is leading to the higher estimates of the relative contributions of the participation in the experimental classrooms. In the table above, as the quality of the model decreases, i.e., as the R-squared decreases, the relative contributions of being in the experimental classrooms increases. The absence of better explanatory variables for the math models increases the variability of the experimental variables. While the relative contributions of being in an experimental classroom are greater for the math scores, the models have a great deal more variability. This reasoning is only suggested as one possible explanation of the contrary results. Generally, the transparency treatment group gained higher achievement scores than the control group of students. This leads to the conclusion that the filter treatment did contribute to increased learning, including increased language achievement in the treated students, although many other things, including design effects and model effects, confound this finding. Threats to internal and external validity: This is an ex post facto research design with two treatment groups (screened students, ones issued plastic transparencies and ones not issued plastic transparencies) and a control group (all other non-screened students at these grade levels). Threats to internal validity may lead to finding the same results that were found using the present methodology. Better studies will include responses to these threats, taking account of any additional effects before the treatment tests are made. Selection threats: Selecting the students using a diagnostic technique precludes the use of random assignment of students to the treatment and control groups. Educational research often has this threat to internal validity. We do not know how much bias was introduced by allowing the screening staff to include children from the class into one group or another. Our results suggest that the control classes and the other students had different pre-treatment achievement scores. We used the entire non-screened classes as the control group, but this is unsatisfactory for the selection of the control. One way to attempt a better control of this threat is to randomly select classes for the program and then to randomly assign treatment and control status. First, we could give the diagnostic to each of the classes of students. In one of the classes, we assign them the plastic transparencies, giving a placebo to the students not needing a transparency. In the control class, we keep track of the students diagnosed as needing the filter and those not needing it. The test of increased reading learning could then be between the condition-identified treatment group, the placebo treatment group, the filter-needing control group, and the filter-not-needing control group. Without the random assignment, which is the design presented in this study, the evaluation could have inadvertently included the biases of the screening staff. Instrumentation Threats: The CTBS/4 or SABE II are regularly given once a year to these grade level students, and the pre- and post-testing does not seem to affect the results. However, did the filter treatment students actually use the sheet during the time between the prescription and the second testing? No formal attempt was made in this study to follow-up on the students to determine this fact, although calls to teachers were made to check on what was happening. A better design would be to observe the students using these sheets, page 5

recording the frequency of the use of the acetates during class work and the test. While every student considered has, indeed, taken both the "pre" and "post" CTBS/4 or SABE II exams, we have no formal evidence that the students have used the sheets of acetate throughout the prescription period and during the second testing. This stronger design, one including the collecting of filter use data for students during their classroom activities, would be able to show the direct effect of student use of plastic transparencies and increased achievement test scores. Hawthorne Effect Threats: These effects confound the results by supplying the treatment classroom students with a "special" feeling that they are more important than other students. Consider the students in the treatment classes. Most are using plastic transparencies, and they may be getting "special" treatment from teachers. The students who were not using plastic transparencies could have felt the need to "compensate" or "compete" with these students. This could have accounted for the elevated score gains in these classes. A possible solution could be to assign a "real" or placebo filter to every student in the class, making the filter a part of normal classroom operation and not placing relatively any more emphasis on the types of filter users. A double blind study could even include both types of plastic transparencies without the teacher's knowledge of the differences between them. Implementation Threats: Gottfredson and Gottfredson (1990) comment that the original study at the General Electric plant included special organizational training, special incentives for increasing production, and the developing of a team spirit among the Hawthorne lightingtreatment subjects. These are implementation effects that were not controlled for in the Hawthorne study. Similar problems could have occurred in this case. The use of the plastic transparencies by the treatment subjects could have had similar implementation effects: special training by the screening staff besides the initial training, team or group identity building by the treatment subjects that was not recorded, or internalized positive feedback incentives (higher scores leading to more parental praise or school incentives leading to higher incentives). there appears to be some teacher effects in these results. We could tease out these effects if we collect teacher specific data. This would allow us to glean a "true" estimate of the effect of the filter treatment. Conclusions: This was an ex post facto evaluation study of the Light Sensitivity Project. Evidence was found supporting the value of the treatment. The results, however, were not completely satisfying, as the design could not completely rule out alternative explanations for the findings. Specifically, while strong effect sizes were found for the training in the presence of past achievement score levels, of student demographic variables, and of student program participation, the data also strongly suggest other mechanisms may be at work. These include a possible biased choice of classrooms for the treatment, a possible Hawthorne effect among the filter users and non-users in the classroom, possible teacher differences, and no formal evidence of a direct link between the use of the plastic transparencies and increased scores. While the results of this evaluation were generally positive for the value of the treatment, a future study with a stronger design would yield better information. page 6

Bibliography: Blaskey, P. et al, (1990), "The Effectiveness of Irlen Filters for Improving Reading Performance, A pilot study," Journal of Learning Disabilities, 23. Cardinal, D. H. et al, (1993), "Do Tinted Lenses Really Help Students with Reading Disabilities?", Intervention in School and Clinic, 28: 275-279. Fletcher, J. and G. Martinez (1994), "An eye Movement Analysis of the Effects of Scotopic Sensitivity Correction on Parsing and Comprehension," Journal of Learning Disabilities, 27: 67-70. Gottfredson, G. D. (1996), "The Hawthorne Misunderstanding (and How to get the Hawthorne Effect in Action Research)," Journal of Research in Crime and Delinquency, 33: 28-48. Henk, W. A. (1992), "Scotopic Sensitivity Syndrome and It's Treatment by Colored Overlays and Lens Filters: an Update," in Padack, N. D. et al, eds., Literacy research and Practice: Foundations for the Year 2000: 14th Yearbook of the College Reading Association. Henson-Parker, D. (1997), See it Right!, Assessment Manual for Scotopic Sensitivity Syndrome, Dorothy Henson-Parker, Rancho Cucamonga, CA. O'Connor, P. D. et al, (1990), "Reading Disabilities and the Effects of Colored Filters," Journal of Learning Disabilities, 23: 597-603. Otto, W. (1994), "Tinted Polyester Overlays and Other Stories (Views and Reviews)," Journal of Reading. 37: 694-697. Parker, R. M. (1990), "Power, Control, and Validity in Research," Journal of Learning Disabilities, 23: 613-620. Robinson, G. L., and R. N. Conway (1990), "The Effects of Irlen Colored Lenses on Students' Specific reading skills and Their Perception of Ability: a 12 Month Validity Study," Journal of Learning Disabilities, 23: 589-596. Robinson, G. L. (1994), "Coloured Lenses and Reading: A Review of Research into Reading Achievement, Reading Strategies, and Causal Mechanisms," Australasian Journal of special education, 18: 3-14. Spafford, C. S., et al (1995), "Contrast Sensitivity Differences between Proficient and Disabled Readers Using Colored Lenses," Journal of Learning Disabilities, 28: 240-252. Solan, H. A. (1990), "An Appraisal of the Irlen Technique of Correcting Reading Disorders Using Tinted Overlays and Tinted Lenses," Journal of Learning Disabilities, 23: 621-623. Tyrrell, R. et al (1995), "Coloured Overlays, Visual Discomfort, Visual Search, and Classroom Reading, Journal of Research and Reading, 18: 10-23. page 7

Statistical Appendix:

Table 1: Normal Curve Equivalent Values for CTBS/4 Survey and SABE II Students in grade 2, 1997

Total Language Battery

Mean; St. Dev.; N Transparencies, Yes

1996 54.20; 24.14; 10

1997 66.00; 15.90;10

Transparencies, No 60.50; 13.65; 8

63.75; 18.46; 8

Not Screened

24.72; 25.31;140

23.48; 24.99; 140

Total

24.80; 26.75; 158

28.21; 27.53; 158

Total Math Battery

Mean; St. Dev.; N Transparencies, Yes

1996 58.50; 25.15; 10

Transparencies, No 56.88; 19.98; 8

Not Screened

39.10; 22.06;140

Total

41.23; 22.82; 158

1997 73.90; 18.13;10 82.75; 15.92; 8 36.62; 17.58; 140 41.32; 21.87; 158

Total Reading Battery

Mean; St. Dev.; N

1996

Transparencies, Yes 53.00; 23.82; 10

Transparencies, No 53.00; 19.01; 8

Not Screened

41.31; 18.30;140

Total

42.64; 18.95; 158

1997 66.90; 19.87;10 60.00; 18.07; 8 36.48; 15.62; 140 39.59; 18.18; 158

JD, 12/9/97

page 8

Table 2: Normal Curve Equivalent Values for CTBS/4 Survey and SABE II Students in grade 3, 1997

Total Language Battery

Mean; St. Dev.; N Transparencies, Yes

1996 35.24; 21.74; 29

1997 52.17; 18.78; 29

Transparencies, No 43.55; 16.61; 11

54.09; 21.46; 11

Not Screened

27.66; 27.20; 108

29.34; 26.30; 108

Total

30.32; 25.89; 148

35.66; 26.66; 148

Total Math Battery

Mean; St. Dev.; N Transparencies, Yes Transparencies, No Not Screened Total

1996 45.76; 16.64; 29 52.64; 17.44; 11 48.61; 20.42; 108 48.35; 19.49; 148

1997 54.83; 20.97; 29 55.09; 23.20; 11 50.94; 21.72; 108 52.01; 21.61; 148

Total Reading Battery

Mean; St. Dev.; N Transparencies, Yes

1996 42.52; 15.31; 29

Transparencies, No 46.45; 18.74; 11

Not Screened

40.30; 18.51; 108

Total

41.19; 17.92; 148

1997 50.10; 19.85; 29 55.73; 17.58; 11 46.90; 15.96; 108 48.18; 16.96; 148

JD, 12/9/97

page 9

independent variables

Total Language Battery, 1997

Lagged NCE Values

Language NCE 1996 0.651*

Language NCE 1996 0.645*

Dummy for Special Education

0.038

0.018

Dummy for Female

0.007

0.002

Dummy for LEP

-0.261*

-0.207*

Dummy for FEP

-0.039

-0.011

Dummy for Third Grade

0.055*

0.028

Dummy for GATE

0.057

0.038

Dummy for Title I

-0.014

Dummy for Transparency, ---Yes

-0.029 0.164*

Dummy for Transparency, ---No

0.088*

Total Math Battery 1997

Total Reading Battery, 1997

Math NCE 1996 0.385*

Math NCE 1996 0.383*

Reading NCE 1996 0.547*

Reading NCE 1996 0.530*

-0.049

-0.091*

0.012

-0.017

-0.074

-0.082

0.078

0.176*

0.006

0.056

0.158*

0.118*

0.063

0.037

-0.223*

-0.241*

0.037 0.034 0.003 0.244* 0.131* -0.030

0.032 0.101* 0.037 0.214* 0.113* -0.055

----

0.231*

----

0.162*

----

0.223*

----

0.153*

R2 Adjusted R2

0.789 0.784

0.815 0.809

0.371 0.354

0.448 0.429

0.419 0.403

0.456 0.437

F-Ratio for model

139.24*

130.31*

21.92*

23.91*

26.75*

24.68*

Degrees of 297 freedom for model

295

297

295

297

295

Notes: Bold* -ed valued indicate significance at the 5% level. NCE is Normal Curve Equivalent.

JD, 12/9/97

page 10

Independent Variables

Computation

Concepts and Applications

Math Subtest, 1997

Math Subtest, 1997

Lagged NCE Comp

Values

NCE 1996

0.272*

Comp NCE 1996 0.282*

C&A NCE 1996 0.295*

C&A NCE 1996 0.276*

Dummy for Special Education

-0.029

-0.057l

-0.034

-0.081

Dummy for Female

-0.135*

-0.141*

-0.006

-0.013

Dummy for LEP

0.124*

0.203*

-0.038

0.053

Dummy for FEP

0.055

0.094

-0.038

0.013

Dummy for Third Grade

0.057

0.021

0.199*

0.168*

Dummy for GATE

0.079

0.053

0.048

0.030

Dummy for Title I

-0.221*

-0.233*

-0.272*

-0.295*

Dummy for Transparency, ---Yes

0.222*

----

0.183*

Dummy for Transparency, ---No

0.122*

----

0.281*

R2 Adjusted R2

0.220 0.199

0.267 0.242

0.384 0.368

0.4471 0.453

F-Ratio for model

21.85*

10.77*

23.16*

26.26*

Degrees of 297

295

freedom for

model

297

295

Notes: Bold* -ed valued indicate significance at the 5% level. NCE is Normal Curve Equivalent.

JD, 1/15/98

page 11

Phylocom: software for the analysis of phylogenetic community structure and trait evolution, 3 pages, 0.07 Mb

doc.uments.com

About Us :: Privacy Policies :: Terms of Service :: Feedback :: Copyright :: Contact Us :: DMCA Policy

Copyright © 2018 doc.uments.com