Annual Evaluation Report 2013-2014

Tags: Learning Community, the Learning Community, FRL, the program, English Language Learners, school attendance, National Norm, Elementary Learning, implementation, child care staff, Learning Community Family Liaison Program, extended learning, Conestoga, summer school, Millard Public Schools Extended Learning, programming, students ages, Omaha Public Schools, staff members, Family Liaison, service plan, Emotional Support, elementary students, Classroom Organization, staff survey, summer camp program, Learning Community Family Liaisons, Learning Community Center of South Omaha, achievement data, Nebraska Department of Education, Preschool students, school readiness, English language, expressive communication skills, English language skills, Bracken School Readiness Assessment, Open Enrollment, Enrollment data, expressive communication, Preschool Language, BSRA, Fall Spring Students, Elementary Learning Center, educational outcomes, Evaluation Report, Buffett Early Childhood Institute, Nebraska Legislature, young students, Language Development, pre and post, Academic Readiness, Bracken, LCFL
Content: OF DOUGLAS AND SARPY COUNTIES Annual Evaluation Report 2013-2014 Elementary Centers and Programs Open Enrollment In compliance with §79-2104.02 & §79-2118 Submitted to the Nebraska Legislature January 2015
The Learning Community of Douglas and Sarpy Counties
Learning Community Coordinating Council
Subcouncil 1
Subcouncil 3
Subcouncil 5
Jeanne Anderson
Lorraine Chang
D. Paul Hartnett
Nancy Jacobson
Jack Heidel
John Synowiecki
Matthew Wickham
Martha Slosburg
Nina Wolford
Subcouncil 2 Kris Carter Roger Holthaus Marque Snow
Subcouncil 4 Amy Friedman Allen Hager Mike Pate
Subcouncil 6 Mike Avery Mike Jones Blaine McCulloh
Lorraine Chang Chair, Learning Community Coordinating Council
Mike Avery Vice Chair, Learning Community Coordinating Council
Learning Community Staff Ted Stilwill Chief Executive Officer Patti Benzel Council Operations Manager Paula Erlewine Executive Assistant
Brian Gabrial Finance Director Renee Franklin Executive Director, Elementary Learning Centers Jamalia Parker Director of Family Engagement Services
Data and Evaluation Consultants Dr. Lisa St. Clair ­ Section I Dr. Bert Jackson ­ Sections II, III
Acknowledgements: The Learning Community of Douglas and Sarpy Counties acknowledges the cooperation and assistance of the Nebraska Department of Education Data, Research and Evaluation Team: Dr. Dean Folkers, Senior Administrator; Pam Tagart, IT Applications Developer; Jill Aurand, IT Applications Developer; and Becky Hamm-Massey, IT Data/Database Analyst.
Table of Contents
Introduction
Report
Page
Section I ­ Evaluation of Elementary Learning Programs .............................. 1
Section II ­ Student Demographics ............................................................... 86
Section III ­ Open Enrollment.......................................................................... 93
Introduction The Learning Community is pleased to provide to you this Annual Evaluation Report. While the Report complies with 79-2102.02 and 79-2118, it also represents a turning point in the short evolution of the Learning Community. When the Learning Community Coordinating Council first convened in January of 2009, it set out to implement legislation that responded to the conflicts swirling in the Omaha-metro area. The discussion that started back in 2005 to 2007, about how to best meet the needs of children from poverty, continues. One difference today is a Learning Community with experience in implementing that legislative policy and focus on strategies that will have the greatest impact in meeting the academic needs of those children. The conversations about power, boundaries and finances cannot distract the Learning Community from advocating for the educational needs of children in poverty. Our work on promising solutions is reflected in the mission statement adopted by the council in 2013. It is our guidepost: Together with school districts and community organizations as partners, we demonstrate, share and implement more effective practices to measurably improve educational outcomes for children and families in poverty. The extensive learning needs of children in poverty are no less now than in 2005. Learning Community school districts have made significant, but incremental progress in narrowing the achievement gap. The differences between children whose families have means and those who do not, is increasing the "degree of difficulty" for educators more quickly than our demonstrated ability to respond. There is reason for optimism in the positive results for programs supported by the Learning Community, as documented in this report. We can add measurably to that optimism with the increasing commitment to early childhood education provided by LB 585 and now with added support from Buffett Early Childhood Institute. By collaborating with school districts and their community partners, we can make a substantial difference in long-term educational outcomes. We all want to see children overcome the challenges of poverty and succeed in school. No one wants that more than their parents and the small circle of adults who care about them. Parents and caring adults are our greatest assets. By engaging them in positive child development, we will learn from them and likely generate our best insights. We thank you for your support in this effort and look forward to discussing this work in more detail. Ted Stilwill, Learning Community CEO
Section I ­ Evaluation of Elementary Learning Programs
SECTION I. EVALUATION REPORT OF ELEMENTARY LEARNING PROGRAMS External evaluation principal investigator: Lisa St. Clair, Ed.D., Interdisciplinary Center for Program Evaluation, Munroe Meyer Institute, University of Nebraska Medical Center. Prepared 11/10/14. Background The elementary learning centers funding levy was established to launch innovative programs to impact the achievement of elementary students who face challenges in the educational environment due to poverty, limited English skills, or mobility. Evaluation Approach and Rationale Generally based upon a Utilization-Focused evaluation design (Patton, 2012), the evaluation plan employed multiple methods to describe and measure the quality of implementation, the nature of programming, and to report outcomes demonstrated by the elementary learning programs funded by the LC. These programs included the emerging Learning Community Center in North Omaha with the Kellom and Conestoga Early Childhood Partnership Program, Jump Start to Kindergarten, Extended Learning (Tutoring, After School, and Summer School programs), Literacy Coaching, the Learning Community Center of South Omaha Family Literacy Program, and the Family Liaison program. The overarching evaluation questions were: 1. Implementation: What was the nature and quality of implementation? Who accessed and participated in the program? Was there variation in implementation and if so, what factors contributed? a. What happened? b. For whom? c. What was the quality of implementation? 2. Academic focus: What were short and long term outcomes related to academic achievement? a. Did other stakeholders report improvement in student learning or engagement (parents, school day teachers)? b. Was there improvement in communication skills (literacy)? c. Was there improvement in quantitative thinking skills (numeracy)? 3. Family support focus: What did the program or school provide to families/parents that will allow greater student success in school? a. What processes did the program or school use to support the needs of families? 1
b. What processes did the program or school use to develop resources for helping to meet those needs? External Evaluation (EE) Process Funding for the external evaluation (EE) of the Elementary Learning Center programs occurs through the Learning Community's ESU core services funds which are restricted to research and evaluation. These funds were identified in statute to support research and evaluation. UNMC Munroe Meyer Institute's Interdisciplinary Center for Program Evaluation has served as the external evaluation team for the Elementary Learning Center programs since 2010. The external evaluation process is implemented collaboratively with district and agencies. The EE team meets about monthly at a committee called "The Evaluation Collaborative" for the Learning Community. This committee is well attended by district research and evaluation leadership and/or designee of the superintendent, and/or superintendents; university staff from University of Nebraska Lincoln, Omaha, and Medical Center; leadership staff from the Learning Community; and research staff from the Nebraska Department of Education. The purpose of the Evaluation Collaborative meetings is to jointly share information, planning of evaluation, and sharing results of evaluation findings on a regular basis. Another way the EE team works collaboratively with districts and agencies is to meet individually to plan their program evaluations. In Patton's Utilization Focused program evaluation model, the districts and agencies are the Primary Intended User (PIU) of the evaluation results and, therefore, are the EE team's primary customer. There are certainly common core requirements or features within the evaluations of common programs implemented across districts and agencies (one example being the Classroom Assessment and Scoring System or CLASS tool, University of Virginia). The purpose of using common measures where possible is to aid in aggregate reporting. Individual evaluation results are not shared with the Learning Community leadership or council--unless they may be reported directly to them by the school district or agency themselves. All reporting at the Learning Community leadership and council level is in aggregate. How does ICPE assure validity and quality of the evaluation reporting? The process used by ICPE is to collect data (such as direct assessment of preschool students, surveys, videos for the purpose of rating teacher/student interactions) directly in coordination with districts and agencies, to receive data from districts and agencies (student achievement and demographic data), and to compile, analyze, and report on data for two purposes. The two purposes are overall evaluation of the ELC programs of the Learning Community (reported within this report) and to share with districts and agencies. 2
ELC data are shared back with individual districts and agencies. First, they are shared to assure accuracy. Did the district or agency find the same result (e.g., effect size)? Second, they are shared once approved and finalized for the purpose of promoting utilization of results. Sometimes fostering utilization of results includes grouping multiple districts or agencies with the same type of programming for joint learning (such as occurred with the Jump Start to Kindergarten grantees). The audience for the Evaluation Report prepared by the EE team is the Education Committee of the Legislature. Once districts or agencies have approved their evaluation results (meaning agreement was reached on what those results were) and on the descriptions of their programs, they are no longer permitted to ask for edits to the contents of the final evaluation report (other than minor proofreading corrections of course). Further, the Learning Community leadership staff and council members are not permitted to request edits to the report (again, other than minor proofreading corrections). Evaluation team members are reliable in collecting and scoring assessment, observation, and rating tools. There are several processes used to assure the upmost accuracy of all data collected by ICPE staff members. Teacher completed tools: An ICPE member scores the protocol and then a second team member scores. If any differences are found, a master scorer re-scores the tool. Child measures (PLS, PPVT, Bracken): An ICPE member administers the assessment following the protocol of the examiner's manual. A second ICPE member double scores the assessment and checks for following of protocol (including re-calculating chronological age of child, rescoring, and re-interpretation of standard scores). Observational measures (ERS, CLASS, KIPS): Prior or concurrent to first observation or rating of the year, ICPE staff members re-anchor themselves with team members. In addition, CLASS has another reliability component completed online annually with the publishers of the tool (Teachstone). When the observation phase begins, an ICPE member observes and rates a classroom or representative teaching staff, or in the case of KIPS, a videotape of parent-child interactions. The observation may be in person (ERS) or on video (KIPS, CLASS). Another team member will re-score the tool to assure numerical and protocol accuracy. If questions emerge, they are addressed by the leadership team. Concurrent reliability is completed throughout the year. For example, CLASS videos submitted by programs and districts may be scored by one or more evaluation team members and in some cases, every team member for inter-rater reliability practice and assessment. After reliability processes are completed, videos are either returned to the program or district (if requested) or deleted from UNMC servers. In the next evaluation year, the Kappa statistic will be used to measure inter-rater reliability (0.60). This is an 3
improvement to the industry accepted practices of `within 1' agreement on 80% or more of the items. By following the industry standard, "reliable" ratings may appear different. Therefore, by moving to using the Kappa statistic, ratings across raters should be essentially the same. Measuring and Reporting on Program Impacts To quantify program impacts, we will report all pre and post measures relative to significance (were the results significant) and if so, what was the magnitude of the change (effect size). To understand effect size and to place it in context, Cohen (1988) suggests using d=0.20 to be small, d=0.50 to be medium, and d=0.80 to be a large effect. To describe this another way, John Hattie in Visible Learning: A Synthesis of over 800 Meta-Analyses Relating to Achievement, uses a concept called "zone of desired effects" that starts at a medium effect size, 0.40 (Hattie, 2009). Hattie suggests that a 1.0 effect size (as shown in Hattie's graph) is equal to about 2-3 years of student growth and learning. Effect sizes can be greater than 1.0; however, they are less common and are therefore not shown on the graphic. Effect sizes tend to be smaller with very young children. With younger students (infant through kindergarten), effect size is often lower because the range of measurement error is larger with very young children (Burchinal, 2008). This concern, seconded by the smaller number of early childhood assessments that measure learning domains, indicates why there might be more measurement error in the testing of young children. Therefore, for the very young, an effect size less than 0.40 may still be within the zone of desired effects. 4
Program Descriptions
Early Childhood Focus
Subsection I.1
Early Childhood Partnership Program at LCCNO
The Early Childhood Partnership program is offered through the emerging Learning Community Center of North Omaha in partnership with Omaha Public Schools and Metropolitan Community College. It is located within two buildings: Kellom Elementary and Conestoga Elementary. There are also emerging partnerships with other higher education institutions (University of Nebraska at Omaha and Creighton University).
Subsection I.2
Jump Start to Kindergarten Programs
Jump Start to Kindergarten programs offer programming to support students in the summer prior to entry into kindergarten.
Elementary Focus
Subsection I.3
Extended Learning Time Programs
I.3.1 Comprehensive Out of School Time: These programs provide out of school time programming throughout the school year. Students would be offered after school programming greater than one hour per day. This design would typically target academic and social/behavioral supports, and in some cases, family engagement services.
I.3.2 Tutoring: Tutoring ELT programs provide after school tutoring targeted to students at greatest risk for academic failure during the school year. This is typically offered in one hour sessions, one or two times per week.
I.3.3 Summer: Summer extended learning programs provide summer programming which targets academic and social/behavioral supports typically to students who have been identified as needing additional supports, and in some cases also includes recreation, health/wellness, and family engagement services.
Subsection I.4
Instructional or Literacy Coaching in Elementary Schools
Instructional or Literacy Coaching: Coaching programs provided literacy or other types of instructional coaches to teachers in elementary buildings with high levels of students qualifying for free/reduced lunch. The coaches provided support in multiple areas including individual work with teachers, professional development for teaching teams, data analysis, assessment assistance and assistance with gathering resources for use in classrooms.
5
Family Focus
Subsection I.5
Family Literacy Program at LCCSO
The Family Literacy Program is offered through the Learning Community of South Omaha (LCCSO) in partnership with One World Community Health Centers. This program provides family literacy and parenting education to families in the broader South Omaha area, with a predominant focus on serving high poverty parents who are learning English.
Subsection I.6
Family Liaison Program
The Learning Community Family Liaison Program (in partnership with Lutheran Family Services): The Family Liaison Model was established to reduce barriers to learning by providing services to students and families that address underlying issues affecting the family and child. The program's multi-pronged approach to service delivery address a variety of factors that impact the student's ability to learn.
6
Subsection I.1 Kellom & Conestoga Early Childhood Partnership Program at LCCNO
Lisa St. Clair, Ed.D., Kari Price, & Terry Stone, Ph.D.
ECPP Program Key Findings
About the Kellom & Conestoga Early Childhood Partnership Program The Kellom and Conestoga Early Childhood Partnership Program (ECPP) was implemented beginning in August of 2013 as a collaborative effort of the Omaha Public Schools, the Learning Community of Douglas & Sarpy Counties, and Metropolitan Community College. Purpose of Evaluation
Served 128 students in two schools with 119 included in the evaluation 84% of students were eligible for free/reduced lunch Students significantly improved measured receptive vocabulary (d=0.35) but did not show significant change in expressive language skills Students significantly improved in measured academic readiness for kindergarten (d=0.51)
The evaluation of the Kellom and Conestoga Early Childhood Partnership Program focused on determining the overall effectiveness of the program in providing early childhood services, parenting education and family support services, as well as more intensive family liaison services to families.
The evaluation strives to answer the following questions:
Who does the Early Childhood Partnership Program serve? Are staff and classrooms of high quality? Are students benefitting and achieving positive outcomes? Are families benefiting and achieving positive outcomes? Are pre-service students from Metropolitan Community College benefitting from participation? Are neighborhood child care programs serving children before they enter the ECPP improving their services to very young children and their families?
These questions continue to be answered by collecting data across multiple sources and utilizing mixed methods approaches.
7
Implementation: Who does ECPP Serve?
The evaluation team collected evaluation data on students served since August of 2013. Student demographics are reported in the following table.
Year
%
% Black- % White
%
Hispanic African
or
Multiple
or Latino American Caucasian or Other
13-14 16%
61%
13%
26%
Gender % Male 52%
% Verified for Special Education 30%
% FRPL 84%
Total Served To Date 128
Omaha Public Schools offered 172 school days in the 2013-14 year. Students at the ECPP schools were absent an average of 13 days. One area for continuous improvement would be to work together to partner with parents to help them understand the great importance of attending school. Staff Credentials Leadership staff: The Early Childhood Specialist responsible for ECPP holds a bachelor's degree in child development and a master's degree in Bilingual and Multicultural Education. Teaching staff: All (8 of 8) of the Preschool Lead Teachers held a valid Nebraska Teaching Certificate with an endorsement in Early Childhood Education, Preschool Disabilities, or Early Childhood Education Unified. In addition to a lead teacher in each classroom, the ECPP also has a paraprofessional working in each classroom. An early childhood paraprofessional is also assigned to each classroom and must meet the qualifications for this position within Omaha Public Schools. They held either Child Development Associate (CDA) certificate, or held or were pursuing associates' degrees or bachelors' degrees in early childhood or related fields. Coaches: Two coaches were brought on board during the 2013-14 school year to support the professional development of the teaching teams. Coaches held masters' degrees in early childhood education. Family support staff: Family support staff work in each of the two buildings to support the families of students served in ECPP.
8
Are Classrooms of High Quality?
Quality early childhood programs have been linked to immediate, positive developmental
outcomes, as well as long-term, positive academic performance (Burchinal, Vandergrift, Pianta,
& Mashburn, 2010; Burchinal, Peisner-Feinberg, Bryant, & Clifford, 2000; Ramey & Ramey, 1998).
Classroom settings themselves are associated with both positive and negative effects on young
students' motivation (Shonkoff & Phillips, 2000). Although the relationship between classroom
environment and motivation is complex and requires further study, current research suggests
that, "...students in classrooms characterized by minimal pressure to perform, ample child choice
in activities, encouragement of collaboration, and more nurturing teacher-child interactions
show more engagement when working on achievement tasks (Stipek et al.,
1995; 1998 as cited by Shonkoff & Phillips, pg. 158, 2000)."
ECERS-R
The key evaluation question for this section is: Are ECPP classrooms of high quality, as measured by industry-standard rating tools?
Early Childhood Environment Rating Scale ­ Revised
Environment Rating Scale. The quality of preschool classrooms will be measured using the Early Childhood Environment Rating Scale ­ Revised (ECERS-R). This observational tool is used to assess the quality of preschool classrooms in various domains including: Space and Furnishings; Personal Care Routines; Language and Reasoning; learning activities; Interaction; Program Structure; and Parents and Staff, as well as an overall rating of quality.
Authors: Harms, Clifford, & Cryer, 2005 Scale: 1 to 7 1 = Inadequate 3 = Minimal 5 = Good
ECERS-R Sub-scores and Overall Score, 2013-14
7 = Excellent
# of classrooms Space & Furnishings Personal Care Routines Language- Reasoning Learning Activities Interactions Program Structure Parents & Staff Overall Rating
Year & School
Overall
8 5.16 3.92 6.69 5.68 6.85 6.18 5.94 5.65
Conestoga 4 6.07 4.46 6.81 6.09 7.00 6.48 5.87 6.04
Kellom
4 4.26 3.38 6.56 5.28 6.70 5.88 6.00 5.26
Preschool classrooms were of good to excellent quality and, on average, exceeded the Nebraska Department of Education indicators of quality scores of "5" or greater on the ECERS-R. The area of Interactions, for example, was very positively rated, which is key because effective teaching begins with positive interactions. Interactions will be explored in more depth with the next evaluation tool--CLASS. An area for continuous improvement exploration would be personal care 9
routines, which focuses on health and safety (such as nutrition and hand washing). It is recommended that the program explore strategies to consistently support the areas of personal care routines across classrooms. CLASS Observation Rating. The Pre-K version of the Classroom Assessment Scoring System (CLASS) will also be completed in each preschool classroom. The Pre-K CLASS has three domains:
Emotional Support ·Positive Climate ·Teacher Sensitivity ·Regard for Student's Perspective
Classroom Organization ·Behavior Management ·Productivity ·Instructional Learning Formats
Instructional Support ·CONCEPT DEVELOPMENT ·Quality of Feedback ·Language Modeling
Dimensions include emotional, organizational, and instructional supports. Instructional Support tends to be the domain with the most opportunity for improvement as it challenges teachers to effectively extend language, model advanced language, and to promote higher-order thinking skills. Research on the CLASS supports ratings of 5 or higher within the domains of Emotional Support and Classroom Organization, and 3.25 or higher within the domain of Instructional Support, as being necessary to have impacts on student achievement (Burchinal, Vandergrift, Pianta & Mashburn, 2010). Pre-K CLASS Domain Averages 2013-14
Pre-K CLASS Classroom Assessment Scoring System Authors: Pianta, LaParo, & Hamre, 2008 Scale: 1 to 7 1-2 = Low Range 3-5 = Middle Range 6-7 = High Range
Year & School Overall Conestoga Kellom
# of classrooms 8 4 4
Emotional Support 6.09 6.48 5.70
Classroom Organization 6.03 5.96 6.10
Instructional Support 2.88 3.38 2.38
Preschool classrooms achieved the goal of 5 or greater in emotional support and classroom organization, and exceeded or approached the goal of 3.25 in Instructional Support. It is recommended that professional development focus on the strategies within the CLASS domain of Instructional Support: Concept Development, Quality of Feedback, and Language Modeling.
10
Average ratings by each subarea of the CLASS were as follows. Emotional Support The CLASS measures emotional support through the emotional qualities of teacher-student interactions, the teacher's responsiveness to student needs, and the degree to which activities and interactions are structured around the interests and motivations of the students. Research supports the theory that these aspects of teacher-student interactions are relevant to students' social and academic outcomes. Students with positive and less conflicted relationships with teachers demonstrate greater peer competencies and positive academic development (Bryk, Lee, & Holland, 1993; Crosnoe, Johnson, & Elder, 2004; Ladd et al., 1999; Roeser et al., 2000).
Item
Average Rating
Positive Climate
6.44
Positive Climate measures the emotional connection and enjoyment demonstrated between educators and students, as well as among students. This dimension also observes positive
affect among educators and/or students and positive communication. Respect (eye contact, warm voice, respectful language, and cooperation/sharing) is another indicator within
Positive Climate.
Absence of Negative Climate
5.94
The Absence of Negative Climate measures the absence of expressed negativity such as anger, hostility, or aggression exhibited by educators and/or students in the classroom. This
dimension includes punitive control (yelling, threats, harsh punishment), sarcasm or disrespect, and severe negativity (victimization, bullying, physical punishment). A higher
rating reflects a lack of or less Negative Climate.
Teacher Sensitivity
5.94
This domain evaluates educators' awareness of and responsiveness to students' academic and emotional concerns. The tool focuses on educators' awareness of students' needs,
responsiveness (acknowledging emotions, providing comfort and assistance, and providing individualized support), how educators' address problems, and students' comfort with
educators.
Regard for Student Perspectives
5.06
Regard for Student Perspectives measures the degree to which educators' interactions with students and activities place an emphasis on student centered learning activities (drawing from students' interests, motivations, and points of view). This dimension is measured by flexibility and student focus, support for autonomy and leadership (allowing students' choices, giving responsibility to students, and allowing students to lead lessons), student expression, and reasonable student freedom of movement. Strengths: Positive Climate: Staff generally demonstrated respect and positive communication with students.
11
Item
Average Rating
Teacher Sensitivity: Staff generally were aware of students' academic and emotional
concerns and made efforts to respond.
Areas to Explore:
Regard for Student Perspectives: More prompting of student autonomy and
leadership and less teacher direction and talking (such as moving to more of a learning through play, project based learning approach) would rate more positively and may associate with increased student expressive language skills (possible link to
Preschool Language Scale expressive scores).
Overall Emotional Support Rating
6.09
Classroom Organization This domain looks at a broad array of processes related to organization and management of students' behavior, time, and attention within the program. Research on management of students' behavior indicates that best practices include providing clear, consistent behavioral expectations, redirecting minor misbehavior, and using positive, proactive strategies (Emmer & Strough, 2001).
Item
Average Rating
Behavior Management
6.47
The dimension of Behavior Management evaluates how educators effectively monitor,
prevent, and redirect active misbehavior. Other behavior markers include: clear behavioral
expectations, educators' proactive behaviors (monitoring, anticipating problem behavior, and
low reactivity), effective redirection of misbehavior, and student behavior (frequent
compliance and little aggression or defiance).
Productivity
6.31
This dimension measures how educators maximize learning time, manage routines within the
classroom, ensure that transitions are brief (with learning opportunities embedded for longer
transitions), and prepare to have materials ready and accessible.
Instructional Learning Formats
5.31
The Instructional Learning Formats dimension evaluates how well the educators facilitate activities and provide interesting materials to engage students. This dimension is specifically looking for effective facilitation and expanding students' involvement through questioning and active participation in lessons and activities. It also measures the use of various modalities and materials, student interest, and clarity of learning objectives through advanced organizers (connecting prior knowledge to new lessons), summaries, and reorientation statements. Strengths: Behavior Management: Educators worked to anticipate and redirect problem behavior.
12
Productivity: Routines were managed and it was clear educators tried to maximize
learning time.
Areas to Explore:
Instructional Learning Formats: It was not consistently clear what the learning targets
were or how content might be connected to students' lives. Educators may want to
explore additional strategies for effective facilitation that expands students'
involvement in learning through questioning (again, this may link to expressive
language skills.
Overall Classroom Organization Rating
6.03
Instructional Support As measured by the CLASS, the extent to which teachers provide students with opportunities to: understand-build connections between new and previous knowledge; apply-use procedures and knowledge to help solve new problems; analyze-divide information into meaningful parts; evaluate-make conclusions based on criteria or standards; and create-put pieces of knowledge together to produce new ideas (Mayer, 2002) are measured.
Item
Average Rating
Concept Development
2.56
This dimension measures how the educators use instructional discussions and activities to promote students' higher-order thinking skills (in contrast to rote instruction). This measures how educators facilitate analysis and reasoning (why and/or how questions, problem solving, prediction, classification/comparison, evaluation), creating (brainstorming, planning, producing), integration of concepts, and connections of concepts to the real world, such as relating information to students' actual lives.
Quality of Feedback
2.72
Quality of Feedback assesses how educators extend student learning through responses to
students' ideas, comments, and work. Included in this dimension are scaffolding, feedback
loops (back-and-forth exchanges, persistence by educators, follow up questions), prompting
students' thought processes, providing additional information to expand students'
understanding, and encouragement/affirmation.
Language Modeling
3.34
Language Modeling measures the extent to which educators facilitate and encourage
students' language. Ratings include frequent conversations in the classroom, educators' use
of open-ended questions, repetition and extension (educators repeat and extend students'
responses), self and parallel talk (educators map their actions and students' actions with
language), and the use of advanced language (a variety of words, and connections to familiar
words and/or ideas).
Strengths: Language modeling: Educators did some facilitation and encouragement of students' language, including use of open-ended questions, repetition, and extension. If the program can build on this and do more of it, even more consistently, it would benefit students' Language Development.
13
Areas to Explore:
Concept development: Taking a concept from a beginning point and fully supporting the development of the concept with students was not regularly observed. Educators
may want to reflect on how to connect to students' every day experiences, their lives, as they introduce and support concept development.
Overall Instructional Support Rating
2.88
Continuous Quality Improvement. Upon completion of the ECERS- R or CLASS in each classroom, debrief consultation immediately followed with a member of the evaluation staff, the teaching team, and the Coach. Using a continuous quality improvement model, strengths as well as areas for improvement were discussed with each group. These data were also reviewed with program leadership, administrators and staff. Professional development plans can continue to be refined in accordance with the findings of the observation data. Data were also provided to the Research Office of Omaha Public Schools. Coaching Research has consistently demonstrated the importance of professional development (PD) for improving outcomes of early childhood programs. Effective PD is associated with increases in teacher knowledge, student learning, and program quality (Christ & Wang, 2013; Powell, Diamond, & Cockburn, 2013). Thus, early childhood initiatives have become increasingly focused on PD as a mechanism for increasing the effectiveness of early childhood programs and fostering children's learning and development (Powell et al., 2013). Coaching is a form of PD that takes place directly in the classroom and involves helping teachers acquire, improve, or refine specific evidence-based intervention practices or teaching behaviors, as well as offering ongoing support and individualized feedback (Hsieh et al., 2009; Wasik & Hindman, 2011). Based on the Vygotskian concept of scaffolding, coaches work one-on-one with teachers to enhance their knowledge and understanding through a process of instruction, guided practice, and reflection (Wasik & Hindman, 2011). Coaching is frequently offered as part of a multicomponent PD program that includes introductory workshops, an ongoing course, or web resources that offer information on evidence-based practices related to the content of the PD (Powell, Diamond, & Cockburn, 2013). Classroom observations are a central component of 14
coaching, a practice that allows coaches to view a teacher's implementation of the new teaching practice in his or her classroom and then provide and discuss feedback. Feedback typically includes two basic types of information that is aligned with the PD program's content: an identification of appropriately implemented practices and recommendations for practice improvements--feedback that is sometimes referred to as what "glows" and what "grows", respectively" (Powell et al., 2013, p. 387). Emerging research stipulates combining coaching with other forms of PD to effectively support teachers and improve outcomes for children. For example, Wasik and Hindman (2011) implemented the Exceptional Coaching for Early Language and Literacy (ExCELL) program in an effort to improve the vocabulary and pre-literacy skills of at-risk preschoolers. ExCELL is a comprehensive PD model for training teachers to implement evidence-based practices aimed at promoting children's literacy and language development. The PD content is organized into five modules--interactive book reading, guided conversations across the curriculum, phonological sensitivity, alphabet knowledge, and writing. In this study, Head Start teachers were randomly assigned to either an intervention group that received the ExCELL PD or to a comparison group that received the "business as usual" PD provided by Head Start. Teachers in the ExCELL group participated in a four-day summer literacy institute designed to familiarize the teachers with the goals of the project and to explain the training and coaching procedures. Thereafter, coaches provided nine months of three-to-four week training cycles in which coaches provided intensive group training for the teachers. In each group training session, coaches provided a conceptual rationale for the practices and then explained and modeled specific strategies. A few days later, the coach modeled the target behaviors in each teacher's classroom while the teacher observed the coach's modeling by using an observation checklist that highlighted key teaching behaviors. Teachers were given approximately one week to practice the targeted instruction before coaches used the same observation checklist to observe the teachers' implementation of the new practice and to provide feedback. Frequent videotaping of a teacher's implementation of practices was used to facilitate coaching and discussions. Furthermore, teachers received books, materials, and lesson plans to support the development of children's language and literacy. After one academic year, it was found that teachers in the intervention group had created higher quality classroom environments, as measured by the Early Language and Literacy Classroom Observation and Classroom Assessment Scoring System, and by videotapes of teachers' classroom book readings. Specifically, evidence from the CLASS measure indicated that intervention teachers modeled language more, provided important feedback to children on their language, and were more effective in fostering concept development in children. Converging evidence from the ELLCO revealed that teacher training significantly increased the richness of the language and pre-literacy environment in the classroom, particularly with respect to the availability and use of writing- related materials. This study also showed that children in the intervention group performed 15
significantly better than comparison-group peers on measures of receptive vocabulary and phonological sensitivity but that alphabet learning between the two groups was equivalent. In a similar study, Powell, Steed, and Diamond (2010) examined the implementation of literacy coaching with Head Start teachers participating in a PD program that included intensive workshops on evidence-based practices for promoting early literacy outcomes in four major content areas--reading, writing, letter-word Knowledge, and phonological awareness. The PD program examined in the study was part of the U.S. Department of Education's Early Childhood Educator professional development program, an initiative aimed at improving school readiness and reading outcomes for prekindergarten children living in poverty. The PD program consisted of two parts: (1) five full-day workshops across a four-month period during the fall semester, and (2) expert coaching conducted in participating classrooms approximately every third week. Analyses revealed significant differences (small to medium effect sizes) in teaching emphasis and progression across the four literacy content areas. Similar results regarding the positive effects of combining PD training (usually workshops) with practice-based coaching have been demonstrated in a variety of classroom-based interventions such as the Behavioral, Emotional, and Social Training Competent Learners Achieving School Success (BEST in CLASS) (Conroy et al., 2014), HEAD Start REDI (Bierman et al., 2008), the Chicago School Readiness Project (Raver, et al., 2008), and the Teaching Pyramid Model (Fox et al., 2011). While a number of studies have illustrated the effectiveness of combining coaching with more formal training such as coursework or workshops, there is limited research on the effects of coaching-based PD programs. In an effort to better understand the independent contribution of coaching as the sole mechanism for improving teacher practice and child outcomes, Neuman and Wright (2010) examined the impact of two forms of professional development of prekindergarten teachers' early language and literacy practice. Specifically the researchers compared the benefits of PD through coaching or coursework with a control group who received no additional PD on quality early childhood literacy practices. Center- and family-based childcare providers were randomly assigned to one of three groups: (1) PD coursework at their local community college in early literacy development; (2) PD coursework plus weekly coaching for 32 weeks (designed to put in place the practices described in the coursework); and (3) a business- as-usual control group. It was found that coursework alone did not lead to improvements in either teacher knowledge or practice. In fact, scores on both measures were indistinguishable from those of the control group. However, the childcare workers who had received coaching made statistically significant improvements in their skills and practices in both centers and home- care settings. In a recent in-depth qualitative study, Knoche, Kuhn, and Eum (2013) interviewed 21 parents, preschool teachers, and childcare providers who had engaged in coaching relationships. 16
Participants completed surveys and took part in extensive interviews in which they detailed their experiences and perspectives as recipients of coaching support. Five main themes emerged as central to the experiences of the coaches: (1) qualities of the coach, (2) resources provided by the coach, (3) qualities of the coach-coachee relationship, (4) coachee transformation, and (5) challenges to the coaching process. Each participant expressed overwhelming positive responses regarding the coaching-coachee relationship, noting that they had experienced both personal and professional growth through the partnership. They credited these positive experiences to the expertise and skill of the coaches as well as to the quality of the dyadic relationship with the coaches, stating that, in isolation, either expertise or quality of interactions would not have been sufficient for successful outcomes. These findings underscore the importance emphasizing strategies for promoting interactions across the dimensions of knowledge and relationship skills. Knoche and colleagues concluded that three implementation features were essential for successful coaching PD designs--(1) sufficient dedicated time on the part of both coach and coachee to engage in successful problem-solving sessions; (2) a reflective component; and (3) a plan for removing coaching support and helping coaches to remain self-reliant. Assuming these three strategies are in place, coaching can not only provide valuable guidance and support for parents and teachers, but also help promote the development of children and families. Therefore, as ECPP strives for continuous improvement in programming, it will be critically important to clearly identify their coaching model. And then it will be also important that the evaluation team identify effective strategies to best measure the implementation of coaching. This falls into the area of implementation science. Fixsen and colleagues (2005) suggest it is not enough to fund innovation, one must evaluate to assure that programs are successfully implemented with fidelity to an evidence-based model.
Student Outcomes School readiness is an essential concern for students entering the educational system. Preparation to perform in an educational setting is a significant benefit for students, especially those who are from diverse backgrounds, with a greater number of risk factors, and have typically poorer school performance compared to their economically advantaged counterparts (Shonkoff & Phillips, 2000).
PPVT-IV Peabody Picture Vocabulary Test IV Author: Dunn and Dunn, 2004, 2007 Scale: The average score is 100, with an average range of 85-115.
The approach used to measure student outcomes was to match fall and spring data. This means that data represented in pre and post or fall and spring data are exactly matched by student and if a student did not have a match in the fall or spring, their data were not included in the outcomes analyses.
17
Vocabulary Development The vocabulary of students is an important factor to explore when considering how students may fare as they progress through school. Students who have limited vocabularies at a very young age are likely to have more difficulty increasing their vocabulary to a level similar to those whose vocabulary is greater to start (Hart & Risley, 1995). Preschool students' receptive vocabulary development (understanding of language) was assessed using the Peabody Picture Vocabulary Test IV (PPVT-IV).
PPVT-IV Standard Scores 130
115 100.00 100 85
88.51 84.83
90.80 88.58
86.30 81.21
70 National Norm 13/14 Overall (n=112) 13/14 Conestoga 13/14 Kellom (n=57) (n=55) Fall Spring
Students significantly improved receptive vocabulary skills. Using the Peabody Picture Vocabulary Test (IV) pre and post, students significantly improved (p<.001, d=0.35) and gained an average of 3.68 standard score over the school year (M=3.68, SD=10.56). Effect size change was below the zone of desired effects (0.40 or greater). School level differences were found, with Conestoga students increasing from 88.58 to 90.80 (M=2.22, SD=10.41, p>05, not significant) and Kellom students increasing from 81.21 to 86.30 (M=5.09, SD=10.59, p=.001, d=0.48, within the 18
zone of desired effects).
Language Development
Preschool students' English (or in cases of English Language Learners Spanish) language
development skills were assessed using the Preschool Language Scales- Fifth Edition (PLS-V). This tool measures preschool students' progress with PLS-V
language by looking at both expressive communication and auditory Preschool Language language comprehension. It was administered with English language Scales- 5th Edition
students to gather ratings on the expressive skills of students and with English language learners, both expressive communication and auditory comprehension were measured. Evaluation questions included: (1) Do
Authors: Zimmerman, Steiner & Pond (2011- English, 2012-Spanish)
students make gains in English? (2) Do they make gains in Spanish?
Score:
There were too few Spanish speaking students to report on outcomes. Spanish language skills were measured at pre and post but so few students were stronger in Spanish skills than English that these outcomes will not be reported (n=8).
The mean is 100 with the average range of 85-115
English language skills were measured for the expressive communication skills.
Preschool Language Scale Expressive Communication Standard Scores ­ English 130
115 100.00 100 85
NS 85.86 86.59
NS 88.81 88.17
82.78 84.94
70
National Norm Exp English Overall
Exp English
Exp English Kellom
(n=104)
Conestoga (n=53)
(n=51)
Fall 2013 Spring 2014
Students at one of the two schools significantly improved in expressive communication skills in English. Using the Preschool Language Scale (5th) pre and post, students as a group did not significantly improve (p>.05). However, school level differences were found, with Conestoga 19
students showing a slight decline from 88.81 to 88.17 (M=-0.64, SD=10.31, p>.05, not significant) and Kellom students increasing from 82.78 to 84.94 (M=2.16, SD=6.91, p=.03, d=0.31, approaching though not within the zone of desired effects).
Student Academic Readiness for Kindergarten
The importance of concept development, particularly for students from diverse cultural and
linguistic backgrounds, has been demonstrated in numerous research articles (Neuman, 2006;
Panter and Bracken, 2009). Some researchers have found that basic concepts are a better means
of predicting both reading and mathematics than are traditional vocabulary tests such as the
PPVT-IV (Larrabee, 2007). The norm-referenced assessment selected to measure Kindergarten
student's academic school readiness is the Bracken School Readiness Assessment (BSRA). The
BSRA was used to measure the academic readiness skills of young students in the areas of colors,
letters, numbers/counting, sizes, comparisons and shapes. The mean of the BSRA is 100, with 86
to 114 falling within the average range (one standard deviation above and below the mean). It
has been used in numerous studies, including the Joint Center for Poverty Research, NICHD study of early child care and youth development, Harlem Project, and the national implementation study of Educare, to name but a few. The limitation of this assessment is that it
Bracken The Bracken School Readiness Assessment -3rd Edition (BSRA)
does not measure social/emotional readiness for school, executive functioning, and other important qualities to consider relative to "readiness for school."
Author: Bracken, 2007 Scale: The average score is 100, with an average range of 86-114
BSRAs were completed in the fall of 2013 in 8 classrooms with a total of 119 students. BSRA standard scores are displayed in the following table.
20
Bracken School Readiness Assessment (BSRA) Standard Scores 130
115 100 100 85
90.03 85.56
90.16 86.18
89.89 84.96
70 National Norm
Overall (n=112) Fall
Conestoga (n=55) Spring
Kellom (n=57)
Students significantly improved in academic readiness for kindergarten. Using the Bracken School Readiness Assessment pre and post, students significantly improved (p<.001, d=0.51) and gained an average of 4.46 standard score over the school year (M=4.46, SD=8.76). Effect size change was within the zone of desired effects (0.40 or greater). School level differences were minimal, with Conestoga students increasing from 86.18 to 90.16 (M=3.98, SD=8.17, p=.001, d=0.49) and Kellom students increasing from 84.96 to 89.89 (M=4.93, SD=9.34, p<.001, d=0.53).
The next table displays the percent of correct items identified by students across the five subscales of the BSRA (n=112 overall program, Conestoga n=55, Kellom n=57).
Subtest Colors
Percent of Mastery by Subscale (Percentage of Items Correct)
Fall
Spring
Fall
Spring
Fall
Overall Overall Conestoga Conestoga Kellom
63.30% 80.00% 66.73% 82.73% 60.00%
Letters
33.87% 53.51% 34.06% 52.85% 33.68%
Numbers and 27.83% 45.34% 27.17% 45.66% 28.46%
Counting
Sizes and
28.77% 43.14% 30.99% 45.12% 26.63%
Comparisons
Shapes
38.26% 50.67% 40.09% 48.64% 36.49%
Overall
35.77% 51.54% 37.07% 51.85% 34.51%
Spring Kellom 77.37% 54.15% 45.03% 41.23% 52.63% 51.25%
21
Utilization of Results with Schools and Programs. Teachers and coaches were debriefed on fall student outcome data in late November and on spring student outcome data in April and May. Individual student reports were prepared both fall and spring for parents. Classroom level reports were shared with teachers, coaches, and program leaders. Classroom quality data results were shared with teachers, coaches, and program leaders in the winter, shortly after quality ratings were completed. Supports for Parents Kellom and Conestoga Parent Focus Group. A focus group was held with interested parents on June 17, 2014. A very small number elected to participate (n=3). Therefore, this feedback should then be taken as baseline information that may or may not be representative of other parents involved with the program. Benefits of the Program. Parents participating in the focus group were excited about many aspects of the program. Logistically, the participants liked that the program was full day and provided all the meals and snacks. Having before and after care provided was needed by the working parents. One mother stated, "Having the meals provided was very helpful when trying to make the food last until the end of the month." Being that the program was full day it helped ease the financial burden of both daycare and food costs. Overall, they found the teachers and the staff to be caring and helpful. They reported their children were safe at school and that they teachers genuinely cared about them. One parent mentioned being able to trust them and was excited for when her younger child would be able to attend the program. Parents were adamant in that the program met and went beyond their expectations. The consistency of the program helped parents establish routines at home and built social emotional skills with their students including sharing and getting along with others. Parents discussed how the school routines had influenced their own interactions and routines at home. Parents reported that they established more routines with their children patterned after the school routines. One parent said, "It was like school was the first parent" and found it helpful to have guidance in setting improved eating and nap schedules. Parents reported noticing behaviors at home they had not previously seen such as their children being imaginative, using more advanced vocabulary and wanting to read and learn on a more frequent basis. Finally, the parents stressed the educational component of the program saying that they felt better about sending their students to this program than a daycare. One parent said she had been looking for something that would be more educational in nature and was pleased with the program. Parents reported being excited about the progress their children made over the year. Parents reported many gains made by their children including those in basic academic skills, (such 22
as knowing colors, shapes and number), life skills (knowing address and phone number) and behaviorally (engaged in learning and able to follow teacher directions). Parents recognized all of the hands-on and play opportunities their children had this year and would like to see that continue. There was some worry about how students would transition to a kindergarten classroom if the hands-on and kinesthetic activities were removed and students lost engagement. As one parent stated, "Right now, my child enjoys coming to school and I hope that continues." Opportunities for Continuous Improvement. Parents reported feeling uneasy about the home visiting component of the program. The home visiting program was seen as a "hoop to jump through" more than as a necessity. As one parent reported, "I didn't like it but you do what you gotta do to be in the program." There was a general fear of that person looking for things "wrong" in the house, particularly if there had been previous CPS involvement with the family. However, all parents reported having positive experiences with the family engagement specialist. One parent noted that she liked the option of meeting with the family engagement specialist at the school particularly as she already had other support workers coming into her home. Two other continuous improvement suggestions made by the parents were in the area of parent involvement. The suggestions were to have parent-child days and to take more fields trips as a group with other parents. Finally, parents reported liking the variety of fruits and vegetables their children were able to try but wanted healthier food overall. Focus Group Summary. Parents in this focus group were very positive about the program and the experiences it was providing for their children. They had very few suggestions to make and felt that the program was meeting their logistical needs and the educational needs for their children. Surveys with Teaching Staff End of year surveys were conducted with lead teachers from both Kellom and Conestoga. Six of the eight teachers completed the survey (75%). The purpose of the survey was to evaluate the educators' perception of the program, the effects coaching had on their teaching practices, and their perceptions of the biggest accomplishments and opportunities for improvement. For those teachers who replied to the survey on their greatest achievements they found the academic progress and growth in their students' to be very exciting. Some reported having 23
grown in their behavior management skills, building relationships with parents and learning how the inclusion model worked with their students topped their greatest achievements for the 2013- 2014 school year. Some opportunities for improvement included teachers at both schools stated they had entered into the school year not knowing specifics about the program or what the benefit of participation in this program might offer to participating families. They did however report having gained knowledge throughout the course of the year through meetings, professional development training and conversations with their coaches. When replying on the effectiveness of the coaches, the majority of the educators found them to be helpful and positive. The majority found their availability to assist in the classroom when needed was very helpful. Other teachers stated there was an uncertainty on exactly what the coaches' roles were in respect to the teachers themselves and that their presence in the classroom at times created chaos. It was suggested to have full-time paraprofessionals assigned to each classroom instead of having staff floating in and out of classrooms. They discussed how this would provide a consistent, reliable environment for the students. Not having an appropriate playground or gym time for gross motor activities was another major challenge. Some of the teachers stated it was difficult to incorporate appropriate gross motor activities without having a scheduled gym time and the playground can only serve one class at a time. Other challenges mentioned were the amount of paperwork involved for Head Start and the lunch program, not knowing the increased workload of mentoring practicum students, not getting questions answered for either themselves or their parents in a timely manner, and at times the lack of communication. Evaluation questions to be addressed in the next evaluation cycle (2014-15): Are Families Benefitting from Participation? Plan: to use Child Parent Relationship Scale (Pianta, 1992) and focus groups. Are Pre-Service Students from Metropolitan Community College Benefitting? Plan: to use Metropolitan Community College data related to matriculation, follow up surveys with graduates, and follow up surveys with graduates' employers, as well as to conduct focus groups. Are Neighborhood Child Care Programs Improving Services? Plan: to evaluate change in quality of services as measured by CLASS. Early Childhood Partnership Program Conclusions and Implications for Program Improvement 24
The Early Childhood Partnership Program started in August of 2013. There was not a tremendous amount of lead time to plan and implement the program. That being said, the program has begun with a solid foundation. External measures of program quality indicate that classrooms are being operated in a high quality manner (ERS ratings great than 5, the indicator of quality established by the Nebraska Department of Education). CLASS ratings were generally in the high quality range for two of three subscales (Emotional Support and Classroom Organization). The goal for improvement of this program is to achieve a rating of 3.25 or greater in ratings of Instructional Support, with particular focus on coaching teachers on Concept Development (taking a concept from a beginning point and fully supporting the development of the concept. 25
Sub-Section I.2: Jump Start to Kindergarten Programs
Lisa St. Clair, Ed.D., Abbey Siebler, M.A., Terry Stone, Ph.D., and Kerry Miller, M.S.
Kindergarten students from low income who attend classrooms with high quality teacher-child
interactions along with effective instructional support demonstrate higher social competence
and academic outcomes (Burchinal,
Vandergrift, Pianta, & Mashburn, 2010). Summary of Program Model
Jump Start to Kindergarten Key Findings
Jump Start programming is designed to 1140 kindergarten students were served
provide students the opportunity to become more prepared for Kindergarten and start at a more equivalent level as their peers that may have had more extensive early childhood care and educational experiences. Programming focuses on pre-academic skills, routines and social skill development.
in five districts and one community agency Served an average of 20 days in the summer 69% were eligible for free/reduced lunch Students were significantly more prepared for kindergarten by the end of
Who was served in these programs?
the program (d=0.42) CLASS ratings improved from the
Jump Start to Kindergarten programs were funded in five districts and one community agency. All subcouncils were represented
previous year High ratings of parent satisfaction were found (4 or greater on a 5 point scale)
with programs. The programs ranged from
two weeks to eight weeks, with varying hours
and days per week. All programs utilized certified teachers for part or all of their staffing.
There were a total of 1011 consented Kindergarten students served by the Jump Start to Kindergarten programs who were present for both pre and post assessment using the Bracken School Readiness Assessment.
Jump Start to Kindergarten was implemented in public schools in Bellevue, Elkhorn, Millard, Omaha, and Papillion, as well as in Educare of Omaha.
26
What was the quality of implementation for the Jump Start to Kindergarten Programs? The Classroom Assessment Scoring System (CLASS) was used to measure classroom quality in Kindergarten programs. Developed by Bob Pianta and others at the University of Virginia Center for the Advanced Study of Teaching and Learning, this external observation tool measures classroom quality across multiple domains.
Emotional Support ·Positive climate ·Teacher sensitivity ·Regard for student's perspectives
Classroom Organization ·Behavior management ·Productivity ·Instructional learning formats
Instructional Support ·Concept development ·Quality of feedback ·Language modeling ·Literacy focus
CLASS was widely implemented this program year. A total of 83 CLASS ratings were completed. These classrooms were drawn from all funded districts and one community agency that received Jump Start to Kindergarten funding. Classrooms were video-recorded, submitted, and then scored at UNMC. A CLASS report was prepared for each participating classroom and results were sent to each district and agency. Districts and agencies determined how best to share the information with the teachers. The CLASS reports included video clips and written feedback along with dimension and domain scores. CLASS ratings were collected at one point in time only (summer). The table below summarizes the average CLASS domain scores for the last four years.
CLASS Classroom Assessment Scoring System Author: Pianta, LaParo & Hamre, 2008 Scale: 1-2 = Low quality 3-5 = Moderate quality 6-7 = High quality
Jump Start to Kindergarten CLASS Domain Averages
Summer
# of classrooms
Emotional
observed
Support
2014
86
5.82
2013
32
5.55
2012
15
6.15
2011
7
6.41
Classroom Organization 6.07 5.82 6.08 5.80
Instructional Support 2.67 2.55 2.78 3.14
CLASS ratings increased on all three domains compared to the previous year, but not compared to the earliest years of implementation with pilot classrooms. With broad implementation of the CLASS it was positive to see the results improve from last year. The goal for continuous 27
improvement should be to continue to support staff professional development with Instructional Support, aiming for a rating of 3.25 or greater in order to impact student achievement (Burchinal et al, 2010). Programs may benefit from exploring professional development training with a focus on continuous improvement, particularly in the Instructional Support domain. Average ratings by each subarea of the CLASS were as follows: Emotional Support The CLASS measures emotional support through the emotional qualities of teacher-student interactions, the teacher's responsiveness to student needs, and the degree to which activities and interactions are structured around the interests and motivations of the students. Research supports the theory that these aspects of teacher-student interactions are relevant to students' social and academic outcomes. Students with positive and less conflicted relationships with teachers demonstrate greater peer competencies and positive academic development (Bryk, Lee, & Holland, 1993; Crosnoe, Johnson, & Elder, 2004; Ladd et al., 1999; Roeser et al., 2000; Silver et al., 2005).
Item
Average Rating
Positive Climate
6.25
Positive Climate measures the emotional connection and enjoyment demonstrated between
educators and students, as well as among students. This dimension also observes positive affect among educators and/or students and positive communication. Respect (eye contact,
warm voice, respectful language, and cooperation/sharing) is another indicator within Positive Climate.
Absence of Negative Climate
6.93
The Absence of Negative Climate measures the absence of expressed negativity such as
anger, hostility, or aggression exhibited by educators and/or students in the classroom. This dimension includes punitive control (yelling, threats, harsh punishment), sarcasm or
disrespect, and severe negativity (victimization, bullying, physical punishment). A higher rating reflects a lack of or less Negative Climate.
Teacher Sensitivity
5.91
This domain evaluates educators' awareness of and responsiveness to students' academic
and emotional concerns. The tool focuses on educators' awareness of students' needs, responsiveness (acknowledging emotions, providing comfort and assistance, and providing
individualized support), how educators' address problems, and students' comfort with educators.
Regard for Student Perspectives
4.18
Regard for Student Perspectives measures the degree to which educators' interactions with students and activities place an emphasis on student centered learning activities (drawing from students' interests, motivations, and points of view). This dimension is measured by flexibility and student focus, support for autonomy and leadership (allowing students'
28
Item
Average Rating
choices, giving responsibility to students, and allowing students to lead lessons), student expression, and reasonable student freedom of movement.
Strengths:
Positive Climate and Absence of a Negative Climate: Staff generally demonstrated respect and positive communication with students.
Teacher Sensitivity: Staff demonstrated awareness of students' academic and emotional concerns and made efforts to respond.
Areas to Explore:
Regard for Student Perspectives: More prompting of student autonomy and leadership and less teacher direction and talking would rate more positively.
Overall Emotional Support Rating
5.82
Classroom Organization This domain looks at a broad array of processes related to organization and management of students' behavior, time, and attention within the program. Research on management of students' behavior indicates that best practices include providing clear, consistent behavioral expectations, redirecting minor misbehavior, and using positive, proactive strategies (Emmer & Strough, 2001).
Item
Average Rating
Behavior Management
6.46
The dimension of Behavior Management evaluates how educators effectively monitor,
prevent, and redirect active misbehavior. Other behavior markers include: clear behavioral
expectations, educators' proactive behaviors (monitoring, anticipating problem behavior, and
low reactivity), effective redirection of misbehavior, and student behavior (frequent
compliance and little aggression or defiance).
Productivity
6.45
This dimension measures how educators maximize learning time, manage routines within the
classroom, ensure that transitions are brief (with learning opportunities embedded for longer
transitions), and prepare to have materials ready and accessible.
Instructional Learning Formats
5.32
The Instructional Learning Formats dimension evaluates how well the educators facilitate activities and provide interesting materials to engage students. This dimension is specifically looking for effective facilitation and expanding students' involvement through questioning and active participation in lessons and activities. It also measures the use of various modalities and materials, student interest, and clarity of learning objectives through
29
advanced organizers (connecting prior knowledge to new lessons), summaries, and
reorientation statements.
Strengths:
Behavior Management: Educators explained processes in advance, were proactive, as
well as anticipated and redirected problem behavior.
Productivity: Routines were managed and educators maximized learning time.
Areas to Explore:
Instructional Learning Formats: Learning targets were sometimes unclear.
Connection of content to students' lives and experiences were not made. Educators
may want to explore additional strategies for effective facilitation that expands
students' involvement in learning through effective questioning.
Overall Classroom Organization Rating
6.07
Instructional Support As measured by the CLASS, the extent to which teachers provide students with opportunities to: understand-build connections between new and previous knowledge; apply-use procedures and knowledge to help solve new problems; analyze-divide information into meaningful parts; evaluate-make conclusions based on criteria or standards; and create-put pieces of knowledge together to produce new ideas (Mayer, 2002) are measured.
Item
Average Rating
Concept Development
2.25
This dimension measures how the educators use instructional discussions and activities to promote students' higher-order thinking skills (in contrast to rote instruction). This measures how educators facilitate analysis and reasoning (why and/or how questions, problem solving, prediction, classification/comparison, evaluation), creating (brainstorming, planning, producing), integration of concepts, and connections of concepts to the real world, such as relating information to students' actual lives.
Quality of Feedback
2.65
Quality of Feedback assesses how educators extend student learning through responses to
students' ideas, comments, and work. Included in this dimension are scaffolding, feedback
loops (back-and-forth exchanges, persistence by educators, follow up questions), prompting
students' thought processes, providing additional information to expand students'
understanding, and encouragement/affirmation.
Language Modeling
3.10
Language Modeling measures the extent to which educators facilitate and encourage
students' language. Ratings include frequent conversations in the classroom, educators' use
of open-ended questions, repetition and extension (educators repeat and extend students'
responses), self and parallel talk (educators map their actions and students' actions with
language), and the use of advanced language (a variety of words, and connections to familiar
words and/or ideas).
30
Strengths:
Language modeling: Educators did some facilitation and encouragement of students' language, including use of open-ended questions, repetition, and extension. If the
program can build on this and do more of it, even more consistently, it would benefit students' language development.
Areas to Explore:
Concept development: Taking a concept from a beginning point and fully supporting the development of the concept with students was not regularly observed. Educators may want to reflect on how to connect to students' every day experiences, their lives,
as they introduce and support concept development.
Overall Instructional Support Rating
2.67
Student Academic Achievement The importance of concept development, particularly for students from diverse cultural and linguistic backgrounds, has been demonstrated in numerous research articles (Neuman, 2006; Panter & Bracken, 2009). Some researchers have found that basic concepts are a better means of predicting both reading and mathematics than are traditional vocabulary tests such as the PPVT-IV (Larrabee, 2007). The norm-referenced assessment selected to measure Kindergarten student's school readiness is the Bracken School Readiness Assessment (BSRA). The BSRA was used to measure the academic readiness skills of young students in the areas of colors, letters, numbers/counting, sizes, comparisons and shapes. The mean of the BSRA is 100, with 86 to 114 falling within the average range (one standard deviation above and below the mean). It has been used in numerous studies, including the Joint Center for Poverty Research, NICHD study of early child care and youth development, Harlem Project, and the national implementation study of Educare, to name but a few. The limitation of this assessment is that it does not measure social/emotional readiness for school, executive functioning, and other important qualities to consider relative to "readiness for school." BSRAs were completed pre and post with a total of 1011 consented students. BSRA pre and post standard scores are displayed in the following chart.
31
Bracken School Readiness Assessment (BSRA) Standard Scores 130
115 100 100 85
92 89
96 91
99 96
87 84
96 93
99 96
95 93
B-SS-1 B-SS-2
70 National Overall Program 1 Program 2 Program 3 Program 4 Program 5 Program 6 Norm (n=1011)
Students significantly improved overall in the Jump Start program, as well as within most of the individual districts and agencies as a whole. Effect size change varied by agency and district.
Effect Size
1.19 1.20
1.00
0.80
0.60 0.42 0.40
0.46
0.45
0.44
0.41
0.20 0.00 0.00 Overall Program 1 Program 2 Program 3 Program 4 Program 5 Program 6 (n=1011)
Significant improvement was not always found at the school or site level, as there were significant variations in change from pre to post at the individual school or site level as well as variations in the numbers of students attending each program. 32
BSRA SS Change by School, 1-30 130 115 100 85 70 55
BSRA SS Change by School, 31-58 130 115 100 85 70 55
Overall, the group of students significantly improved in their readiness for kindergarten (p<.001). Mean standard scores on the Bracken increased from 89 to 92-- moving them slightly closer to the desired mean of 100.
Bracken Percent of Mastery by Subtest Bracken Subtest Colors Letters Numbers and Counting Sizes and Comparisons Shapes Overall
Percent of Mastery Pre 91% 61% 61% 51% 61% 62%
Percent of Mastery Post 93% 67% 66% 56% 65% 67%
Percent Increase 2% 6% 5% 5% 4% 5%
33
The Jump Start to Kindergarten outcomes on the Bracken suggest that an area of strength for these students was color naming (93% mastery). An area for improvement would be Sizes/Comparisons (56% mastery). Sizes/Comparison may be a higher level skill for students as this subtest assesses their understanding of location words, comparison concepts and understanding directional concepts.
Student Achievement Summary (Bracken School Readiness Assessment). Results for the program have remained generally consistent over the first three years with a slight reduction this past summer (but yet effect sizes are still within the zone of desired effects).
Bracken School Readiness Overall Standard Scores
Year
Average Pre Average Post
Average
Standard
Standard
Bracken SRA
Scores
Scores
Standard
Scores Change
Statistical significance using T-Test analysis
Effect Size
# of students
2014 1011
88.98
91.63
2.66
2013 649
86.42
90.50
4.08
2012 800
87.97
92.08
4.11
2011 156
85.85
90.13
4.28
*Significant improvement using a paired samples t-test, one-tailed Effect size calculated using paired samples mean differences/mean standard deviation
p <.001* p <.001* p<.001* p<.001*
d=0.42 d=0.59 d=0.63 d=0.58
Utilization of Results with Schools and Programs. Teachers and schools were debriefed on the Bracken SRA results of each of their students, as well as their group of students, by a member of the evaluation team following both pre and post Bracken administration. The results were delivered to the teachers and schools one to two days after pre-assessment so that the results could be used by the teaching teams to inform and individualize instruction. Post results were also delivered to teachers and schools one to two days after Bracken administration was completed to inform them of the progress their students made. Additionally, the Learning Community hosted a meeting facilitated by the evaluator and one of the Jump Start to Kindergarten districts, along with three other districts. Program staff and research/evaluation staff from four districts participated in a collaborative data study meeting. Results from each district were presented along with program elements that were identified as areas of strength along with challenges or opportunities for improvement. Common themes that emerged from
34
the meeting were: (1) programs reported serving a greater number of students with identified disabilities, (2) comprehensive family engagement activities appeared to be associated with stronger effects, (3) programs that had been funded since 2011 will have students in third grade during the 2014-15 year and would like to share statement assessment results in a collaborative way to measure possible longitudinal results of participation in a Jump Start to Kindergarten program with some districts wanting to compare results to the group of students who were invited to participate but did not do so. Evaluation staff coded the themes that emerged from the discussion. The following are the general themes and notations that emerged from the group. One theme that occurred throughout the meeting was an interest in family engagement. Three of the four districts at the meeting were currently implementing a family engagement component, ranging from weekly family engagement days in which parents received take-home bags containing learning materials to summer home visits. It was also noted that each district used similar criteria for targeting families for its program. Specifically, each district focused on serving those with the highest need, such as those with the highest percentage of free- or reduced-lunch and students that had not had previous preschool experience. Another common theme was that each district followed curriculum guides and standards which also aligned with kindergarten standards. This was beneficial and ensured children across districts received similar learning opportunities and experiences. Each district also expressed an interest in collecting longitudinal data in order to compare the progress of students who had attended Jumpstart with those who had not attended. In order to collect this data, each district stated that they would need to agree on consistent measurements. What did parents report about the Jump Start Kindergarten Programs? Parents provided feedback on the value or usefulness of the Jump Start to Kindergarten Program. Using a collaborative process across all districts and agencies, a master parent survey was developed. Districts or agencies were then able to choose which sections they would use for their program. Parent survey data was received from each of the participating districts and agencies; however, rates of participation varied widely (1-243). Parent survey results are displayed in the following table (n=469). 35
Parent Satisfaction and Ratings of Impact How much do you agree or disagree with each statement: a. I was satisfied with the hours of the program. b. I was satisfied with the length of the program. c. I was satisfied with the program as a whole. d. The staff were excellent (caring, reliable, skilled). e. My child enjoyed attending the program. f. I was able to communicate with my child's teacher. g. I was informed about my child's progress. h. I believe that my child will be more successful in Kindergarten as a result of the program. i. I feel more prepared to be the parent of a Kindergartener as a result of the program. j. My child believes that school will be a fun place to learn. k. If my child begins to struggle in Kindergarten I would feel comfortable approaching his/her teacher or principal. Scale ranges from 1=strongly disagree to 5=strongly agree
Average 4.65 4.55 4.64 4.69 4.72 4.53 4.37 4.64 4.50 4.78 4.67
Families reported high overall satisfaction with the structure and environment of the program. They also reported high levels of impact on such items as believing their child is more ready for kindergarten as a result of the program and feeling comfortable to talk with their child's teacher if a problem emerges. The lowest level of satisfaction was (4.37) for being informed about their child's progress.
Parents were also surveyed about their perceptions of how the program impacted their child (see below).
Parent Report of Child Change
36
Check level of improvement: a. Willingness to separate from parents b. Likes to listen to stories c. Recognizes letters of the alphabet d. Knows different colors and shapes e. Plays well with other children f. Willingness to share with other children g. Interest in sharing what they have learned h. Attentiveness when read to i. Attention span for tasks j. Eagerness to attend school
My child already had these skills 67.6% 65.6% 44.1% 59.9% 59.2% 54.9% 42.1% 46.4% 35.0% 44.0%
Or choose one of the following
Did Not Improve 1.8% 0.9% 4.5% 1.5% 0.9% 2.2% 4.8% 4.2% 8.0% 2.6%
Improved 30.6% 33.5% 51.5% 38.6% 40.0% 42.9% 53.1% 49.2% 57.0% 53.4%
More than half of respondents reported child improvement in recognizing letters of the alphabet, interest in sharing what they learned, attention span for tasks, and eagerness to attend school. Some areas where the majority of students already possessed the skills were willingness to separate from parents, likes to listen to stories, knows different colors and shapes, plays well with other children, and willingness to share with other children. The lowest rated area for children already having skills was attention span for tasks (35%), which associated with the highest percentage of "did not improve" (8%), and 57% showing improvement in this area. It could be, perhaps, that one of the values of Jump Start to Kindergarten is to begin to teach children how to attend to tasks in in a school setting.
Jump Start to Kindergarten Program Conclusions and Implications for Program Improvement
Jump Start to Kindergarten programs were implemented in five districts and one community agency. A total of 1140 Kindergarten students were served an average 20 days over the summer. Students significantly improved on the Bracken School Readiness Assessment (Bracken, 2002, p<.001, d=0.42). Parents reported high levels of satisfaction with and impact by the Jump Start Kindergarten programs. CLASS ratings improved from the previous year with strengths found in emotional support and classroom organization, with opportunity for improvement in the area of instructional support. Therefore, it is recommended that programs explore professional development in the area of instructional support with emphasis on Concept Development. Given
37
the consistently positive results for the students after attending the Jump Start Kindergarten summer programs, districts and programs may want to follow students to see if the programming has lasting effects. For districts funded in the first cohort of Jump Start to Kindergarten, their first group of students will be completing 3rd grade NeSA for the first time this coming school year. 38
Subsection I.3: Extended Learning Time Programs
Lead: Jolene Johnson, Ed.D.
Introduction The Learning Community funded a number of Extended Learning Time programs that included comprehensive out-of-school time programs throughout the school year, before-school and after-school tutoring sessions with targeted academic support, and summer learning programs to students. Below is a description of the programs that served students during 2013- 2014 year.
Extended Learning Key Findings 2096 students were served 73% of students were eligible for free/reduced lunch An overall effect size was calculated and was within the zone of desired effects (d=0.59). Overall, parents found the programs to be a positive experience for their students.
One limitation to this evaluation is the variability both within programs and with how programs measure student success. The programs funded for extended learning varied in the students served, the structures implemented and in program focus. Student outcomes were measured for each program but the measures are quite different and may be difficult to make comparisons across programs.
School Year Program Descriptions
Bellevue Extended Learning. This program provided extended learning time in reading, writing and mathematics over 20 weeks during the school year for target students. Target students (grades 3rd-6th) were identified as those who were at-risk for falling behind or who were already performing below proficiency levels. Intervention was provided in nine Bellevue elementary schools with the highest levels of poverty. Students received intervention for four hours each week through the end of March 2014. Students' progress was monitored and instruction was adjusted based on their progress and individual goals.
Completely KIDS. This program focused on academic proficiency, youth development, food/nutrition, and family engagement. Completely KIDS's academic programming (mathematics, reading, writing, and science) was designed by licensed educators to align with Nebraska State Educational Standards and to supplement classroom learning in the core areas. Licensed educators will also contracted with to support individual and group learning needs while at program. Many lessons were molded to the individual learning needs of each student. The program also provided students with many opportunities to participate in educational enrichment activities, family engagement, and Coordinators worked closed with Family Liaisons
39
(one funded by the Learning Community of Douglas and Sarpy Counties, one privately funded) to identify additional supports for families. Students from Pre-Kindergarten through sixth grade were targeted for this program at three schools. The program was implemented Monday through Friday from 3:55-6:00pm for 28 weeks during the school year. Girls Inc. This program featured an out-of-school setting literacy program to promote phonemic awareness, word recognition, fluency, vocabulary acquisition and reading comprehension. It was sponsored by a community agency and the program complements the local school district's reading curriculum, utilizing the same phonics program and sequence of instruction. Certified teachers were included in the program staff to enhance the expertise, as well as to design specific interventions in response to individual needs. The overall focus of the program was to improve the percentage of students reading at grade level. This program operated Monday through Friday, for three hours per day, during the school year at two sites. Omaha Public Schools Extended Learning Time. This school year program featured Extended Learning Time (ELT) provided to select students with academic needs. It was designed to help students master content in reading and mathematics before and after school. The program design created a cohort of students with a common teacher to establish long term relationships and in-depth learning opportunities within a small group. The teacher from the ELT program and the regular classroom teacher worked together to customize instruction for each student and incorporated planned instruction time for students. An individualized goals agreement was developed for each student and had a weekly focus. Progress towards goals was reported to parents every six weeks. The program was implemented for 24 weeks, consisting of two tutoring sessions per week, for one hour per day, during the school year for students in grades 3-6. This ELT program was designed by the school district and was implemented in five schools. South Sarpy. Springfield Platteview Community Schools. During this 35 week program, students participated in a mathematics intervention program. Students received 60 minutes per week of additional instruction in mathematics before and after school. The instruction was provided in small group and individualized settings and was targeted to meet the needs of each student. Teachers and the math interventionist collaborated on a frequent basis to design and implement the best plan for each student. Students were progress monitored and the plans adjusted according to student data. Summer Program Descriptions Bellevue Summer School. This summer program featured intense instruction in the areas of reading, writing, and mathematics. The program targeted Title I students, ELL students, and other students at risk of falling behind academically. The program operated for two weeks during 40
the summer for seven hours per day, five days per week. Students entering kindergarten and 1st grade were targeted for this program. Students were served in multiple buildings. Catholic Charities Summer. This program provided academically focused summer enrichment, as well as physical and experiential activities to low income students. Goals were structured to support participants in increasing their communication skills in reading and writing along with their quantitative thinking skills in mathematics. A certified teacher structured the lessons and coached the staff to work with staff from local schools to ensure summer offerings complemented and enhanced the school curriculum. The program also provided students with the opportunity to participate in fine arts activities such as music class, swimming skills in partnership with the Red Cross, health and proper nutrition promotion activities, computer lessons, and field trips. The program was implemented for 10 weeks during the summer, 9.5 hours per day, Monday through Friday, and also allowed for early/late pick up. Students in grades K-6 were served in this ten week program. The program was implemented Monday-Friday from 6:30 am- 6:00 pm. Elkhorn Public Schools. Jump Start to Reading. This program served incoming first through fourth grade students who met certain criteria based on the AIMSweb winter benchmarking national norms. Students scoring at or below the 25th percentile received an invitation to attend the program. The three-week program was held four days week for three hours each day. The program focused on individual student reading needs and provided instruction based on one or more programs (Reading Street's My Sidewalks, Read Naturally, Guided Reading and/or Guided Writing). Students received instruction from a certified teacher. The average ratio was one teacher to six students. Girls Inc. Summer. This summer literacy program was designed to promote phonemic awareness, word recognition, fluency, vocabulary acquisition, and reading comprehensive. It was designed to complement the local school district's reading curriculum, utilizing the same phonics program and sequence of instruction. Certified teachers were included in the program staff to design specific interventions in response to individual needs and to help the program improve the percentage of students reading at grade level. Girls aged five through nine years were targeted for participation. The program operated Monday through Friday for nine hours per day throughout the summer. Kroc Center/Salvation Army Summer. The Salvation Army Ray and Joan Kroc Community Center summer day camp took place from May 26, 2014 until August 1, 2014 a total of nine weeks. Participants of the camp had the ability to register for one week of camp or all nine weeks. The camp took place Monday through Friday from 9:00 ­5:00pm with early drop off at 8:00am and late pick up at 6:00pm. 41
The staff hired for the camp included Kroc Center staff, educational and art instructors, five full time Program Aides and one part time; who were hired as seasonal employees. The children were distributed by age levels (6-7, 8-9 and 10-13). There were a few 14 year olds from the Learning Community Center of South Omaha Children were kept in those groups Monday through Thursday and came together for lunch. On Fridays the groups participated in Team Building activities, watched movies, had guest speakers, took field trips and went swimming together. On Friday evenings parents were invited to participate in the camp activities with their children and their immediate family members. This year we had a large amount of families come join their children to engage in family fun activities on Fridays. Millard Public Schools Extended Learning. This program featured summer school learning targeted to K-2 students who have academic deficiencies in reading, writing and mathematics. Students are invited from eight Millard schools with high percentages of economically disadvantaged and/or limited English proficiency students. The program was implemented for three weeks, three hours per day, in two elementary schools in the district. In addition, the program offered three Family Days that included informational, instructional, and community services in areas such as successful strategies to support student learning, health and wellness, personal finance, assessing social services, and child care. Transportation, meals, and books were provided to students, along with a bilingual liaison and licensed social worker to help families who could benefit from those services. Students entering kindergarten are also invited to attend this program as a jump-start experience for school. Ralston Public Schools: The Summer School program provided instruction in reading and mathematics to students in grades K-6. Students were selected for participation based on recent classroom data. Summer school and classroom teachers collaborated on the individual goals for each student. Instruction was provided in small group settings using a variety of strategies and different forms of technology. Specific programs used for intervention included Mathletics (Harcourt) and Leveled Literacy Intervention (Fountas and Pinnell, 2009). Salvation Army/North Corps: The "[email protected] The SAL" was an all-day long youth summer camp program targeted for students ages 6-13 years. This program provided programming, field trips, education, and recreational type activities from 9am - 5pm (M-F) for eight weeks through June and July. Youth who signed-up for the summer program were given the opportunity to be included, and they were assessed with pre- and post-tests throughout the summer months. Daily instruction was implemented though games, reading materials, puzzles, worksheets, and flash cards. This daily instruction was conducted every morning (M-F) in small groups in 20-30 minute intervals. 42
Students Served Who did these programs serve? Participation data were collected on the 2096 elementary students who attended the programs. Demographic data provided on these students indicated that 73% of the students served were eligible for free/reduced lunch. The population served by the extended learning time programs appeared to generally fall within the target of the population identified to benefit from the resources of the Learning Community--those most at risk for academic failure due to socio-economic status. School districts often targeted students for their tutoring and extended learning programs based upon school performance and need to remediate specific academic skills whereas the community based programs most often served students and families based on enrollment in the program and need for programming. Therefore, the programming provided by the programs varied considerably and should be taken into consideration when examining the results. Summer programs provided by agencies provided many activities to engage the whole child and often the family along with some academics. District programming tended to be much more focused on academics and remediating skill deficits. Parent engagement was often a component of both types of programming. Evaluation Data Collection Quality. Quality programs have been linked to immediate, positive developmental outcomes, as well as long-term positive academic performance (Beckett, Capizzano, Parsley, Ross, Schirm, & Taylor, 2009; Burchinal, Peisner-Feinberg, Bryant, & Clifford, 2000). Measurement of the quality of programs is central to program evaluation. This section reports on the CLASS observations completed by the UNMC evaluation team with extended learning programs funded through the Learning Community of Douglas and Sarpy Counties (LC). To examine program instructional quality, the evaluation team recommended use of the Classroom Assessment Scoring System (CLASS). Developed by Bob Pianta and others at the University of Virginia Center for the Advanced Study of Teaching and Learning, this external observation tool measures classroom quality across multiple domains including: emotional support, organization, and instructional delivery. According to its authors, the CLASS "is an observational tool that provides a common lens and language focused on what matters--the classroom interactions that boost student learning." It has three domains: 43
Emotional Support ·Positive climate ·Teacher sensitivity ·Regard for student's perspectives
Classroom Organization ·Behavior management ·Productivity ·Instructional learning formats
Instructional Support ·Concept development ·Quality of feedback ·Language modeling ·Literacy focus
In addition to these domains, interactions are further considered relative to dimensions. These
dimensions include aspects such as: positive climate (focuses on how teachers interact with
students to develop warm relationships that promote student's enjoyment of the classroom
community) and concept development (focuses on how teachers interact CLASS
with students to promote higher-order thinking and cognition). For these reasons, the evaluation team has identified the CLASS observation tool as a valid way to gather an externally rated measure of
Classroom Assessment Scoring System Author: Pianta, LaParo & Hamre, 2008
quality, and one with the added benefit of it having the potential to drive continuous improvement because of the specificity of the feedback from the observation.
Scale: 1-2 = Low quality 3-5 = Moderate quality
All of the after school and summer school sites participated in this piece 6-7 = High quality
of the evaluation. For the majority of the program, CLASS scores were
calculated for the program rather than per teacher. Multiple teachers were recorded, rated, and
the CLASS rating feedback report 7 given to the program included an average of their ratings. However, 6 some programs wanted feedback for
Two Year CLASS Ratings 2012-14
5.99 5.87
6.01 5.65
each teacher. In those cases, a CLASS 5 rating feedback report was completed
and provided. 4
Scores at or above 6.00 indicate high
levels of quality, scores at or above 3
2.98
2.68
3.00 are in moderate quality range,
with scores below 3.00 indicating low 2 quality. CLASS scores in national
studies have been found to be in the low to moderate quality for 1 Instructional Support (Kane et al, 2013), but effectively support
2012-13 (N=18)
2013-14 (N=35)
Emotional Support Classroom Organization Instructional Support
continuous improvement/professional development to support teacher effectiveness.
44
Across programs, average CLASS ratings 5.65 for Emotional Support, 6.01 for Class Organization and 2.68 for Instructional Support. Overall, the program average rated near the high range for Emotional Support, in the high range for Classroom Organization, and near the moderate range for Instructional Support. Recent research on pre-kindergarten and child care programs (Burchinal, Vandergrift, Pianta & Mashburn, 2010) indicated that scores of 3.25 or higher are needed within the domain of instructional support to show measurable impact on student achievement. The chart below displays the scores across all programs and teachers.
CLASS Ratings By Program/Teacher 2013-14 7
6
5
4
3
2
1 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35
Emotional Support
Classroom Organization
Instructional Support
45
Student Achievement
Effect Sizes Across Programs
Following are the effect size 2.00 changes found in each of the
d=0.59
1.56
programs.
1.50 1.33
1.32
While the overall results of the 1.00
0.90
0.60
extended learning program 0.50
0.38 0.33
0.25
indicate that it was effective in improving student achievement, 0.00
-0.10
the magnitude of the improvement varied within and
-0.50
1
2
3
4
5
6
7
8
9
between programs. All of the funded programs submitted data. However, not all of the data were
conducive to completing statistical analyses. Of the eleven programs, effect sizes could be
calculated for seven programs. The effect sizes also varied within subject areas by program.
Significant effects ranged from small effect sizes (d=-0.10) to large effect sizes (d=1.56). An overall
effect size for the program was calculated and found to be within the zone of desired effects
(d=0.59).
One note of caution with interpretation of the effect sizes must be made. The effect sizes were calculated using different types of assessments. Some assessments were standardized norm- referenced assessments, some were adaptive assessments, some were screening measures, one utilized state assessments (NeSA), while another used a district-developed, criterion-referenced assessment. The difference effect sizes found may have been influenced as much by test design and purpose of the assessment as by the program itself.
Family support focus. Parent surveys were collected for students enrolled in the extended learning programs. A total of 340 parent surveys were collected across the programs. Some programs and districts had additional questions not included on all surveys. Those questions are not reported here.
Parent Survey Results 2013-14 Survey Question I was satisfied with the hours of the program. I was satisfied with the length of the program. I was satisfied with the program as a whole. The staff were excellent (caring, reliable, skilled). My child enjoyed attending the program. I am satisfied with the level of communication I had with my child's teacher. I was informed about my child's progress. I believe that my child will be more successful in school next year as a result of the program. Scale ranges from 1=strongly disagree to 5=strongly agree
Average 4.5 4.5 4.5 4.6 4.6 4.3 4.0 4.3
46
Overall, parents rated the extended learning programs positively. The most frequent comments from parents related to their student's increase in both confidence and academic skills. For programs that had Family Days and other specific family activities, the comments were quite positive with families expressing appreciation for the opportunity to interact with their students in a fun and engaging manner. Improvements to the programs included more communication, specific information about what academic concepts their student was learning and improvements to the specific logistics of the program. Extended Learning Conclusions and Implications for Program Improvement Extended learning programs served 2,096 students across three major types of programs: tutoring programs, broader extended learning programs during the school year that served students greater than one hour daily and all/most days of the week, and summer extended learning programs. Seventy-three percent (73%) of students were eligible for free/reduced lunch. Students appeared to benefit academically from participation in extended learning programs (d=0.59). Parents were positive about the experience and the support their student had while attending the programs. One recommendation for the improvement would be to determine some type of consistent measure between the programs or at least to use a common type of assessment. It is difficult to assess which type of programming is yielding the greatest impact and to provide an understanding of the components needed to implement a high quality, impactful programming when measures are mixed (criterion referenced and standardized, norm referenced). 47
Sub-Section I.4: Literacy Coaching
Jolene Johnson, Ed.D.
Literacy Coaching was implemented with the belief that improving teachers' instruction of literacy would improve student achievement in the area of reading. Hattie's research indicated that ongoing use of formative evaluation by the teacher including data analysis and use of evidence-based models yielded an effect size in the high range (d=0.90). Given that literacy coaching provides teachers with formative information, it is possible to affect change in teacher instruction (Hattie, 2009). Other
Literacy Coaching Key Findings 6,033 students were served in two districts 84% of students were eligible for free/reduced lunch Average student score gain on NeSA- Reading was 7.87 points (d=0.23) Effect sizes change within the zone of desired effects were found from fall to spring across CLASS rating domains (external measures of teacher quality, ds ranging from 0.54 to 0.67)
research into literacy coaching found that
intensive coaching activities such as modeling
and conferencing are significant predictors of student reading achievement (Elish-Piper & L'Allier,
2011).
Summary of Program Models
Two districts were funded to implement literacy coaching: Omaha and Westside. The coaching models/frameworks used by the districts varied but there were common elements in the models including: feedback, modeling, working with teachers collaboratively and having minimal time spent in direct instruction of students.
Who was served in this program?
Across both districts, a total of 6033 students were served through literacy coaching in the building in which they were located. Of these students, 84% were eligible for free/reduced lunch. A total of 570 teachers were impacted by literacy coaching but not all teachers participated in the CLASS observations or worked with a Learning Community funded literacy coach.
What was the quality of the program?
Quality of the program was evaluated using several different methods. The CLASS observation tool was used as a pre-post measure for a large sample of teachers. Focus groups were conducted
48
with literacy coaches, teachers and administrators. Finally, an online survey was administered to teachers to provide their input on the program. CLASS Ratings Teaching and learning interactions significantly improved in ratings of emotional support, classroom organization, instructional support and student engagement, with effect size change found within the zone of desired effects. New to the evaluation process for the 2013-14 school year was the use of the CLASS rating tool. Districts varied in their selection processes with one selecting all teachers within buildings and the other identifying volunteer teachers. Classroom teachers were videotaped during the first semester, provided written feedback along with CLASS ratings and then were videotaped once more toward the end of second semester. Each district was provided the opportunity to debrief with the evaluation team on the results of the observations. One district chose to debrief with all teachers in a large group setting while the second district chose to debrief with teachers in small groups at individual buildings. The following chart shows the fall to spring ratings on the CLASS tool across all the teachers participating in the evaluation process. To examine overall changes, a One Way ANOVA was used and was significant across all domains with medium to large effect sizes: Emotional Support (F=12.341, p<.01, 2=0.09), Classroom Organization (F=13.927, p<.01, 2=0.10), Instructional Support (F=14.320, p<.01, 2=0.10) and Student Engagement (F=8.229, p<.01, 2=0.144).
7
6
5.65
5.14
5
4
3
2
1 Emotional Support
CLASS Ratings 2013-14 6.58 6.14 3.6 2.99
Classroom Organization
Instructional Support
Fall Spring
5.57 4.77 Student Engagement
Cohen's (1988) guidelines for effect size with a One Way ANOVA, І. According to him: · Small: 0.01 · Medium: 0.059 · Large: 0.138
49
A second set of analyses was used to test for significant differences using a matched sample of teachers (N=53). A paired t-test was used. Analyses included whole group, district A, and district B. Pre to post change varied in small ways when comparing overall to Districts A and B, as displayed in the following two graphs.
Overall Pre to Post CLASS Ratings (matched sample)
7
6.14
6
5.17
5
4.77
6.55 5.72 5.44
4
3.55
2.95 3
2
1 Overall Pre
Overall Post
Emotional Support Classroom Organization Instructional Support Student Engagement
District A and B Pre to Post CLASS Ratings (matched sample)
7 6.26
6
5.12
5
4.79
6.61 5.67 5.15
5.97 5.24 4.72
6.47 6.08 5.79
4
3.66
3.39
2.94
2.96
3
2
1 District A Pre
District A Post
District B Pre
District B Post
Emotional Support Classroom Organization Instructional Support Student Engagement
50
Overall effect size changes are displayed in the following chart.
Overall CLASS Rating Effect Size Changes
1.60
1.52
1.40
1.20
1.00
0.80
0.67 0.65
0.60
0.58 0.54
0.40
0.20
0.79
0.59
0.64
0.28
0.83 0.60 0.49
0.00 Overall
District A
District B
Emotional Support Classroom Organization Instructional Support Student Engagement
Overall effect size changes were within the zone of desired effects. District A showed effects within the zone except in Student Engagement. District B showed effects across all CLASS subscales. Some caution must be used
Teacher Survey
A total of 570 teachers completed the online survey developed collaboratively between the districts and the evaluation team. The survey explored satisfaction with the coach and the program and the perceived helpfulness of each coaching activity. Each district administered the survey online to teachers and submitted the data to the external evaluation team. Items in the chart below were rated from 1 to 5 with 1=Strongly Disagree and 5=Strongly Agree. For one district, open-ended items asking about benefits and challenges of literacy coaching were included in the survey. For the other district, the open-ended items were completed in focus groups conducted by members of the evaluation team.
Survey Item My literacy coach/facilitator and I have a positive working relationship. My literacy coach/facilitator listens to me. My literacy coach/facilitator is available when I need him/her.
District A 4.70 4.73 4.17
District B 4.27 4.13 3.99
51
Survey Item
District A
When I have a problem, my literacy coach/facilitator is
4.57
helpful in developing a plan to address it.
My literacy coach/facilitator communicates with me clearly.
4.57
Building level support was positive as it related to the
4.53
literacy coaching/facilitator program.
Considering everything, I am satisfied with the literacy
4.27
coaching program.
1=Strongly Disagree to 5=Strongly Agree
District B 3.99 4.19 3.90 3.79
While the ratings were in the positive direction for every question, there is a difference in the ratings between the two districts. Teachers from District A tended to rate the items on the survey more positively than teachers in District B. Teachers were also asked to rate the how useful certain coaching activities were to them in their teaching practice. Multiple activities were listed and teachers who had participated in the activity rated how useful it was to them in their literacy teaching practices. A great deal of variability was again found between the teachers in District A and those in District B as shown in the chart below. None of the activities received an overall rating above a 4.0 in District B, but five activities (Other, Professional Development, Small Group/Differentiated Instruction, Modeling Lessons/Strategies and Coaching/Feedback) all scored about a 4.0 a high level of usefulness.
Ratings of Usefulness of Coaching Activities
Other (instructional decisions, curriculum integration) Professional Development Data Analysis Classroom/Student Observation Lesson Planning Small Group/Differentiated Instruction Modeling Lessons/Strategies Coaching/Feedback 1 District B District A
3.49 4.11 3.71 4.15 3.58 3.72 3.37 3.75 3.26 3.88 3.48 4.37 3.62 4.46 3.39 4.12 1.5 2 2.5 3 3.5 4 4.5 5 1=Not Useful at All to 5=Very Useful
52
Focus Groups for District A In the spring of 2014, multiple focus groups were conducted with staff members. Teachers, literacy coaches and building administrators met with members of the evaluation team to discuss the literacy coaching program funded by the Learning Community. Overall, teachers, literacy coaches and administrators viewed the Literacy Coaching program as an asset in their buildings. Literacy coaches in the buildings were viewed as experts in the field of reading, approachable, respectful to teachers, and able to follow through with teachers. These literacy coach qualities helped to engage teachers into dialogue about reading and reading practices. Benefits of Literacy Coaching Teachers reported seeing literacy coaches as another professional with whom to share ideas, ask for help with lesson planning, and gain access to other resources they may not have time to find. Teachers across all grade levels remarked time and again about needing someone to pull resources and materials together and the literacy coaches were able to do that for them. The teachers expressed satisfaction with the modeling and co-teaching activities provided by the coaches. For newer and first year teachers having a literacy coach was critical to them understanding and implementing all the components of the reading curriculum. "Being a first year teacher in the district, it helped tremendously. She helped me out so it made me feel like I could be successful". And for the more veteran teachers having the literacy coach showing them how to use and implement a key curriculum piece was reported to be a big help. Administrators echoed the sentiments stating that the coaches were a significant resource for beginning teachers in being able to provide almost daily feedback and support while at the same time were able to re- energize some veteran staff with new approaches and feedback. Improvements in both students' interest in reading and in reading test scores were seen in the buildings by all parties, but particularly mentioned by the administrators. Test scores have increased each year since implementing literacy coaching and students continue to exit special services programs (such as reading tutoring, RtI). It's not just the numbers that are improving but as one administrator stated, "Our students are enjoying reading; they're loving reading." Timeliness and specificity of the feedback provided by the coaches were seen as a positive by the classroom teachers and were two components the literacy coaches worked to continuously refine and improve. Coaches reported seeing progress in many teachers they worked with especially on understanding and implementing best practices in reading instruction. Progress was seen particularly in the areas of guided reading and in the levels of student questioning. The level of 53
engagement for teachers working with the coaches was high. For the teams that worked with the coaches, a difference was seen in how teachers were planning reading instruction. Strategies that Worked Modeling and Co-Teaching: Literacy coaches and teachers agreed that modeling and co-teaching were the most helpful in providing feedback and pushing the teacher to the next level. Teachers appreciated being able to observe coaches and ask real-time questions. Coaches felt that the opportunity to provide instant feedback and suggestions was beneficial to the coaching process. Feedback Forms: Having a form allowed staff to get used to receiving feedback and also for the coaches to be cued into asking reflective questions. The form allowed for more objectivity, gave coaches the opportunity to provide specific feedback and ask reflective questions about what was observed in the classroom. Differentiation for Staff: Teachers expressed satisfaction in being able to work with the literacy coaches in a variety of ways. Literacy coaches talked about the necessity of having the freedom to work with teachers in ways that worked for them. One example would be the team lesson planning versus individualized coaching as that allowed the literacy coach to build relationships and rapport with a particular group of teachers who were a bit resistant to the idea of one on one coaching. Administration Involvement: Literacy coaches reported feeling completely supported by the administration in their building. Being able to sit down on a weekly basis with the principal and teacher leader to discuss things happening in the building was beneficial to both coaches and the administration. Having the principals discuss the importance of the role of the literacy coach made it a bit easier to define and fill the role of a coach. Collaboration between the Coaches: The coaches and the administrators both discussed how key it had been for the literacy coaches to be able to collaborate and work together throughout the year. The coaches felt like they could call the other one whenever they needed to talk about an idea or to problem solve. The administrators strongly believed that the collaborative effort was making the program stronger. Challenges of Literacy Coaching Time: Time was a challenge for both teachers and literacy coaches. Teachers discussed that how once NeSA testing started, the time with coaches decreased. Literacy coaches echoed that sentiment. Having to administer or help administer assessments reduced time spent meeting with teachers and thus, the last quarter of school was difficult. Coordinating schedules could be 54
a challenge although teachers did feel like the literacy coaches tried to be as flexible as possible and the literacy coaches felt like teachers respected them and their time. Coaches expressed some concern about not being able to equally reach all teachers. "I don't want them to feel like I ignored them but not everyone gets equal amounts." This concern was echoed in the teacher focus groups, but the teachers understood that not everyone needed the same amount of coaching or support. Role(s) of the Literacy Coach: Coaches often had multiple roles. Teachers stated some confusion about the actual role of the literacy coach. Other teachers questioned the amount of time the coach spent doing other duties while others wondered what the exact role of the coach was. Was it administrative? Was it as a co-professional? Coaches had a similar challenge related to knowing when to call in administrators. They needed to have the trust with teachers and an open dialogue. Yet, when teachers were not responsive or not following curriculum, coaches wanted a clearer idea of when to let the teacher evaluator/school administrator aware of the issues in the classroom. One question asked was, "How do you keep from being evaluative and maintaining relationships with teachers?" CLASS observations: Being a new evaluation component, the CLASS observation tool was met with skepticism by the teachers. In the focus groups, the teachers talked about feeling judged and were unsure about the usefulness of both the tool and the feedback. Also, some would have preferred the taping to occur at different times during the year if it needed to occur at all. District B Themes on literacy coaching from District B come from the open-ended survey items with the teachers and focus group answers from the coaches. Benefits of Literacy Coaching Teachers reported the benefit of being able to see modeling of lessons and new strategies. Several commented on being able to take away new insights after observing the coach model a lesson with the same students the teacher worked with daily. A second benefit repeated over and over was having a professional in the building to collaborate with on lessons, ideas and strategies. Qualities that helped teachers value the literacy coach were when the coach was accessible, approachable, positive and flexible. One teacher remarked "It was nice being to bring a question or idea to the coach and not worry about being evaluated or judged." 55
Feedback provided by the coaches was found to be a benefit when it was specific and insightful to the teacher's lesson and provided in a supportive, direct manner. Teachers commented on wanting to improve and feeling frustrated when they only received shallow, "fluffy" feedback. Several teachers commented on the usefulness of having literacy coaches help them with small group instruction. They reported understanding Guided Reading and small group instruction better and being able to implement it more effectively after working with the coach. Coaches reported that for teachers who are open to changing they have seen benefits to the coaching model. They felt that writing was improving and there was an increased focus on Guided Reading within the classrooms. Consistent with the teachers, coaches felt that peer observations were beneficial to practice and allowed teachers to reflect on their own teaching practices. Challenges to Literacy Coaching Unclear Role: Teachers and coaches alike expressed confusion and frustration about the role of the coach. Teachers felt like the coaches were pulled in too many directions and often had to help out administration which left them without coaching time. Coaches themselves expressed a sense of frustration about not being sure how they could empower teachers and how their role needed to be different and separate from that of the principal. Lack of Buy-In to the Coaching Model: Coaches reported that about 50% of their teachers were interested in the coaching model and really wanted to improve. Several teachers commented on wanting to have the literacy coaches work with struggling students instead of teachers. Lack of working with struggling students was brought up multiple times as a negative about the coaching model. Time and Scheduling: Time and schedules was brought up as a challenge. Time for coaching was perceived to be affected by administering assessments (NeSA) and also by multiple meetings. Teachers and coaches both reported not having enough time to schedule and work together. Building Level Leadership: The support provided by building level leadership varied but was something that coaches struggled with. One coach reported "I feel like I'm alone on an island. I'm not questioned, but I don't necessarily get support." The coaches in the focus group professed strong belief in needing building and perhaps district level support in helping with the coaching model. 56
Student Achievement
While the theory behind instructional coaching implies that teacher change will happen first, student outcomes will still be analyzed as a long-term outcome of the model. Because of the variable experiences with the coaching and the length of time coaching has been implemented in each district, results are reported both by district and in aggregate form. Scale scores on the Nebraska State Assessment- Reading (NeSA-R) were provided to the evaluation team for the 2012-13 and 2013-14 school years. The NeSA-R was chosen as the measure to use given that it is same assessment across both districts. Both districts also give primary assessments but they are not included in this assessment. A paired samples t-test was conducted to compare the scores between years for each district.
District A District B Overall *p<.01
NeSA-R Mean Scale Score Change (2012-13 to 2013-14) +37.38 +2.81 +7.87
Literacy Coaching Implications and Recommendations:
Cohen's d 0.84* 0.12* 0.23*
Literacy coaching associates with improvements in teacher practices and in turn, student achievement. The changes on CLASS ratings would suggest the power and effectiveness of feedback on instructional practices. Information from teacher surveys and focus groups indicate that teachers want to improve their instruction and value the relationship with a coach when it is collaborative and professional in nature. Teachers appreciated immediate feedback on their practices and the opportunity to observe a fellow professional modeling a lesson. However, when the relationship with the coach was not one of collaboration and there was a perceived inequity of roles, teachers were less receptive to the feedback and to working with the coach. The personal characteristics (approachability and professionalism) are as important as content expertise in building a trusting, coaching relationship.
A recommendation for the program would be to continue meeting with all the districts involved with instructional coaching to discuss models, common practices and common evaluation measures. Next year's evaluation (2014-15) will include coaching model descriptions in addition to CLASS ratings, student achievement, and other evaluation data.
57
Section I.5 LCCSO:
Family Literacy Program Lead: Jolene Johnson, Ed.D. This was a year of expansion and change for the Learning Community Center of South Omaha (LCCSO). The program moved from the previous location to a newly renovated building which allowed for them to expand and serve more families. The program grew from serving approximately 60 participants at a time to serving between 160-180 participants
Family Literacy Program Key Findings 243 families and 374 students have been served by the program 94% of the families were eligible for free/reduced lunch 94% were "very satisfied to satisfied" with program services Parents significantly improved their English language skills (d=0.69) Parents demonstrated significant improvement in their parent-child interactions (d=0.34)
at one time. As with any program that
sees substantial growth and change,
LCCSO had to address unfamiliar issues and develop new solutions. The expansion of the program
required that more families be provided with transportation, child care and other resources in
order to access the English and parenting classes provided by the center. Additional staff were
added and folded into the existing structure. It also required that current staff members perform
multiple roles within the center in order to meet the needs of the participants.
Summary of Program Model LCCSO was formed in 2012 as a collaborative effort of The Learning Community of Douglas and Sarpy County, OneWorld Community Health Centers, and Boys Town. LCCSO began providing family literacy services to parents and their children in its temporary location across the street from the Public Library in South Omaha, and moved into its permanent center in South Omaha in the fall of 2013. Parents participating in the program met at the center to attend classes and access services. While parents participated in educational activities, on-site child care was provided for their children eight years old and under.
58
To help children from low-income families succeed in school, LCCSO collaborated with member school districts and community partners to activate long term strategies to support parents in their efforts to promote their children's education by teaching them the skills they need. LCCSO participants received a wide range of interrelated services, including, but not limited to: Parenting Education Navigator Services Adult Education Parent and child outcomes were measured using a variety of assessments in order to evaluate the effectiveness of the various components of the program. The following sections will address what is being measured and present initial and follow-up results, beginning with parents/adults and followed by their children. Parenting Education LCCSO collaborates with various organizations to deliver diverse workshops (KidSquad, Visiting Nurses Association, etc.). A further example of this is the program's alliance with Boys Town which integrated Common Sense Parenting® (CSP) into LCCSO group workshops. CSP was a practical, skill-based multiple-week parenting program which involved classroom instruction, videotape modeling, roleplaying, feedback and review. Professional parent trainers provided instruction, consultation and support to LCCSO participants, addressing issues of communication, discipline, decision making, relationships, self-control and school success. Parents were taught proactive skills and techniques to help create healthy family relationships that fostered safety and well-being. Additionally, family activities were planned and implemented by the LCCSO staff and included field trips to UNO, graduation celebrations and parent-child time during school days the children had off. Navigator Services LCCSO employed navigators that served as personal parent advocates, helping parents gain better understanding of the public school system, community resources and adult education programs. Navigators built strong relationships with participants to ensure individualized education and support. Home Visitations: Navigators visited participants' homes to communicate with parents, conduct informal needs assessments, connect parents with resources, model supportive 59
learning activities, coach parenting skills, and attend to specific needs. Navigators completed home visitations as necessary, but on average, these were completed approximately once a month. Each participant worked with their navigator to design a Family Literacy Plan (FLP) and set personal and familial goals. Adult Education/Literacy English as a Second Language (ESL): Adult participants attended English language classes two days a week during the academic year and throughout the summer. English classes were leveled based on `BEST Plus' scores and teacher input in order to provide a more consistent learning experience. Best Plus is the measurement tool used to assess English learning progress (generally, at the beginning of participation and again after every 60 hours of learning time). Who did LCCSO Serve? Of the 243 families served in 2013-14, 132 were new participants to the program. The program served approximately 374 children across fourteen schools. In addition, two childcare rooms at the center provided a safe environment for infants, toddlers and preschool children so their parents could attend classes. For the families with students in school, 94% qualified for free/reduced lunch status. What was the Quality of Services Implemented? Multiple tools were used to measure growth, assess perceptions of the participants and demonstrate program quality. The evaluation is both summative and developmental in nature. The tools selected for the evaluation provided outcome information as well as informed the implementers about what is working and what needs improvement. Focus Groups Multiple focus groups were conducted in August 2014 to allow participants who had been with the program for six months or longer the opportunity to voice their experiences and thoughts. Questions were broad in nature and asked about the participants overall experience with the program, satisfaction levels with multiple facets of the program (navigators, parenting classes, resources, English classes) and ideas for improvements to the program. Clicker questions and open-ended questions were used to get the highest 60
level of participation from all members of the focus groups. Parent and Student Academic Achievement Evaluation results encompassed four sections: improved language, improved relationships, improved sense of confidence and student achievement. Most sections were supported with both qualitative and quantitative data. Parent and student academic achievement results are described first, with improved relationships and sense of confidence in using English in the Parenting Impacts section. Parents significantly improved their English language skills. The graph below shows the continued growth made by participants in the program. Participants showed significant gains in their English language skills for the second year in a row. Not only were the scores statistically significant using multiple paired samples t-tests but the improvement over time has major implications for functional, vocational and social communication. The average mean for participants entering the program was at a level 2 (M=412) while the mean for participants who have had roughly two years of class was at a level 8 (M=626). To put the scores in context, a score of 400 equates to a person having a few phrases with limited functionality while a score over 600 is equivalent to a person being able to participate effectively in social and work conversations. The ability to communicate beyond simple phrases has allowed the participants to interact to an increased degree with their student's teacher and in the workplace. One student commented, "I think the possibility of getting a job is a little better because we can now express ourselves in two languages." Effect size change on BEST Plus equated to 0.69 overall, well within the zone of desired effects.
Best Plus Scores Students Made Continued Gains 900
700
500
300
412
100
1 (n=225)
460 2 (n=151)
485 3 (n=86)
d=0.69
510
562
4 (n=45)
5 (n=24)
626 6 (n=12)
61
Parenting Impacts Parents reported improved parent-child relationships. This qualitative finding was bolstered by externally rated parent/child interactions showing significant improvement and effect sizes approaching the zone of desired effects. Parenting classes were offered one time a week and provided instruction on several topics including Common Sense Parenting, Cooking Matters, academic information, personal finance and how to manage difficult child behavior. Parents were videotaped as they entered the program and again one year later. The Keys to Interactive Parenting Scale (KIPS, Comfort et al, 2006) was used to assess and measure parent-child interactions across three categories: Building Relationships, Promoting Learning and Supporting Confidence. Parents were video-recorded once a year to assess parenting skills and behaviors with their children who were under 6 years of age. Navigators recorded the parent and had it analyzed by the evaluation team. Once the Navigator received the report back, he or she provided feedback and coaching to the parent on parenting strategies and techniques.
KIPS Scores 2013-14 5.00
4.00
3.77
3.35
3.00
d=0.34 4.13
2.00
1.00
1 n=193
2 n=30
3
n=11
Pre and post KIPS scores were analyzed using paired samples t-tests. The KIPS data for 2013-14 yielded an effect size approaching the zone of desired effect (d=0.34) and was similar to the effect size for 2012-13 (d=0.41).
Focus Group Report
Multiple focus groups were conducted to allow participants who had been with the program for six months or longer the opportunity to voices their experiences and thoughts. There were 69 participants. Questions were broad in nature and asked about the participants overall experience with the program, satisfaction levels with multiple facets of the program (navigators, parenting classes, resources, English classes) and ideas for improvements to the program. Digital response questions and open-ended questions were used to get the highest level of participation from all members of the focus groups.
62
Levels of Parent Satisfaction Very Satisfied Satisfied Unsatisfied
Navigators EnglishTeachers English Classes
49% 49% 55%
45% 48% 45%
6% 3% 0%
Responses showed that over 90% of the participants were either very satisfied or satisfied with all aspects of the program. When asked about academics, parents felt more comfortable reading with their student than in helping with mathematics. Parents indicated feeling comfortable approaching school staff and attending school functions with 97% reporting that they attended parent-teacher conferences. Themes. The following themes emerged across groups and participants. 1. Overall satisfaction with the program was high. Parents reported feeling satisfied with the English classes, parenting classes, childcare and the services of the navigators. 2. Relationships were a key component to program success. Participants often referred to the relationships they had developed with the teachers, navigators and other staff members. The level of comfort in the relationships was particularly evident with the Boys Town instructor. 3. Participants reported a loss of fear and increase in confidence in using English and interacting with the community. Another area in which parents gained confidence was in helping students with reading and homework. 4. Parenting classes are a huge benefit. When asked how helpful the parenting classes were, 97% of the respondents reported the classes were somewhat helpful to very helpful. Not only did the participants find the classes helpful but they asked that the classes be three hours in length (like the English classes).The classes mentioned specifically as being the most helpful were the parenting classes conducted by a Boys Town staff member. The parents appreciated learning about how to interact with their children, deliver consequences and have better conversations. Classes on cooking and nutrition were popular with participants requesting additional classes on nutrition. Other areas that 63
multiple parents requested more instruction was how to deal with older children and bullying. 5. More time in the program was a common suggestion for improvement. Two years may not be enough time in the program for some of the participants. Participants wanted more time in the program to continue to develop their skills. Some participants expressed concern about teacher turnover and how they felt that set them back a few months. Participants also expressed a desire to go further in their studies and learn more about academics. Staff Survey and Feedback. A staff survey was administered to all staff members (administration, navigators, child care staff and teachers) of the center. Eleven surveys were completed. A link to an online survey was provided and participants were given three weeks to complete the brief survey. A survey was chosen over a focus group to provide more anonymity and for staff convenience. An online survey allowed for staff to complete the items when time permitted rather than having to attend one more meeting. Staff members were asked to respond to questions about the strengths of the program, challenges, suggestions for improvement and the biggest successes. The themes that emerged from the survey were consistent with parent feedback and other data collected by the evaluation team. 1. The biggest strength that emerged from the responses was the amount of teamwork and collaboration the staff felt they had in implementing and running the program. As the program has grown, several staffers commented on how everyone pitched in to complete the roles necessary in building the program. 2. Staff indicated that providing an all-program for families was essential to its success. Families were able to access the program because of the transportation, child care, and navigation services that are available to the families. The focus on the entire family was considered to be a strength by multiple staff members. 3. A challenge reported consistently across staff members was the expansion of the center. While viewed positively, staff members discussed the need for more communication with leadership, more training, and perhaps more staff as they felt "spread thin" this year as they handled multiple roles and responsibilities. 4. One improvement suggested by staff members was to increase the amount of plan time for teachers, navigators and child care staff. Student Outcomes Student data were requested and obtained from Omaha Public Schools for the students whose families had participated in the LCCSO programming. Data obtained included demographic 64
information, attendance and achievement data for students across the fourteen schools. Data could only be given if the NSSRS ID (state ID) number was provided to the navigators and evaluation team by the parents. The sample of students for this data was robust at nearly 100 students (N=97). Attendance An area of focus for family-school engagement is attendance. Students need to be in school in order to fully benefit from the curriculum and instruction. According to a recent Nebraska Education sub-committee status report (2013), students who are absent fewer than 10 days in a school year score roughly ten points higher on the NeSA-Reading test and twelve points higher on the NeSA-Mathematics test. A relationship between academic achievement and school attendance would indicate that fewer absences should yield higher student achievement. Most Students Missed 5 or Fewer Days of School 65%
5 or fewer days
25% 6-10 days
7% 11-15 days
3% More than 15 days
The majority (90%) of students missed 10 or fewer days of school. Achievement Data Many of the students are yet too young to have taken the Nebraska State Assessment (NeSA). However, information on the K-2 district developed reading assessment was provided. From fall to spring, students showed significant gains on the assessment in reading with the effect size being above the zone of desired effect (d= 1.53). It should be noted that this is a criterion referenced assessment that is not age anchored or normed and growth is expected. Rather the effect size is shows raw data improvement. Students will continue to be followed and their assessment scores collected to show effects, if any, on their statewide assessments and on other assessments to be collected by the district.
65
LCCSO Program Implications and Recommendations The Learning Community Center of South Omaha continued to impact both the language skills and parent engagement of the participants attending the program. The overall quality of the program was considered satisfactory by nearly all of the participants. Parents reported increased levels of confidence and requested more education--both for themselves and about the children. Student outcomes were beginning to show some impact of the programming, particularly in regard to high levels of school attendance and significant growth on the K-2 district assessment (although because this was a criterion referenced assessments, results must be considered with some caution). One recommendation would be to continue following the students as they continue through school to measure the lasting effects of programming. Parents continued to seek more information and instruction around academic concepts (particularly math) and parenting practices. Further, as their children get older, they expressed interest in learning more about technology issues and bullying. They saw the Learning Community Center of South Omaha as a trusted resource for providing supports and information. A second recommendation would be to continue partnering with the school district to provide additional academic and homework supports for the participants asking for that guidance in helping their elementary students. Leadership from the program should reflect on how and where this activity may best fit into the program, with some consideration given to the higher level language classes. Additionally, staff leadership could reflect on how they might provide staff with additional plan time and timely clear communication. Staff focus group results suggested this could help the continuation of quality programming for the families involved with the program and for staff retention. 66
Subsection I.6 Learning Community Family Liaison Program
Jolene Johnson, Ed.D.
The Learning Community Family Liaison Program (in partnership with Lutheran Family Services) was established to reduce barriers to learning by providing services to address the underlying issues affecting the family and child that impact the child's ability to learn. This intensive program is designed to support the needs of students who have multiple, complex challenges that are associated with needs outside of the school environment but are impacting school and academic success. These stressors affecting both family and child may be wide ranging and inclusive
Family Liaison Program Key Findings Served 305 students in 14 schools 96% of students were eligible for free/reduced lunch Parent stress ratings significantly decreased from intake to discharge (d=1.16) K-2 reading scores improved significantly (d=1.31) 58% of students were absent from school 10 or fewer days
of financial, physical, psychological, logistical or
other factors. Service provision occurred primarily via the Family Liaison (LCFL) was housed in the school and provided targeted services to individual students and their families.
Summary of Program Model
The program placed Family Liaisons (LCFLs) in 14 elementary schools within the Omaha Public School District. Schools that had an LCFL were located in achievement subcouncils two (2) and five (5). The program employed 13 staff including one (1) program director and twelve Learning Community Family Liaisons.
The Learning Community Family Liaison Program's (LCFLP) intent was to build on existing school efforts to provide supports and problem-solve the needs of students. The LCFL was responsible for brokering services to meet the student and family's needs.
The LCFL provided case management, intensive intervention in two ways--targeted student supports and universal student supports.
The Learning Community Family Liaison Program model has evolved to include four different strategies for service provision. The LCFL's primary intent was to support at-risk students who struggled academically and experienced significant challenges in the home, often due to stress
67
within the family. This primary objective has remained consistent since the program's inception. The model was changed this year to eliminate the LCFL's role in school-wide events to enhance more targeted service planning to meet individualized goals. In order to support this effort, the addition of service domains was implemented as a way to better categorize the services being brokered.
LCFL Program Model
Input
Activities
Output
Outcome
Impact
Who/what's involved
What occurs
First effects
Second effects
Over time effects
Persons involved
FLs provide intensive
FL partners with
Parental stress is Parents are
include: School staff, Family Liaisons, family, professionals, community resources, students, Student Assistance Teams Location: At school, at the family's home and in the community (accessing resources as needed)
student and family support for 90 days via: Targeted service planning to meet individualized goals (Domains include: Family, Living Situation, social support, Health, Safety, Legal, Educational- Vocational. Family/student assessments to identify academic/family
family and other
reduced and/or
empowered, develop
stakeholders to
positively
knowledge and/or
create tailored
impacted
ability to manage
service plan for
Student and
stress
youth/family using
stakeholders
Student's academic
SMART goals
implement
success indicators
Family/child are
service plan
increase and/or are
assessed across
which addresses
positively impacted
academic and
need
behavioral/mental 90% of goals
health domains
(per student)
Student's academic
will be met at
needs are targeted
the end of the
90 day period
Time: 90 Days
need Team meetings to
monitor progress
and revise service
plan as needed
Target: Individual
students and families who
provide consent
LCFLP Service Domains. The LCFLP services domains are the most common areas of need for students and families served by the Learning Community Family Liaison. Identifying the service domains through a series of assessments and organizational intake processes are integral in identifying the appropriate interventions, supports and S.M.A.R.T. goals essential for developing an effective individual student and family case plan.
Family. In human context, a family is a group of people commonly classified as: a nuclear family (husband, wife, and children); single parent (mother or father and children); extended family (grandparent, aunt or uncle and children) or non-related family (associated through other
68
arrangements including foster parenting.) In most societies the family is the principal institution for healthy development and socialization of children. Living Situation. The living situation encompasses a description of the circumstances in which the family unit resides. It may include the type of housing, the link between housing and family, or the lack of housing (homelessness). Social Support. Social support is the perception and actuality that one is cared for, has assistance available from other people, and is part of a supportive social network. These supportive resources can be emotional (e.g., nurturance), tangible (e.g., financial assistance), informational (e.g., advice), or companionship (e.g., sense of belonging). Support can come from many sources, such as family, friends, organizations, etc. Health-Physical. Are the student's physical health needs being addressed? Has the student accessed the School Based Health Center? Does the student have access to Medicaid, SCHIP, or private insurance? Are there unmet physical health needs within the family? Mental (Behavioral & Emotional). Does the student have any behavioral or emotional needs that are not being addressed? Does the student have any unresolved or unmet needs that are impacting student or adult relationships in the school setting? Are any problem behaviors blocking a family member's chances of having a good life? Do any other family members living in the family unit or living outside of the family unit have any unmet mental health (behavioral or emotional) needs? Are there unresolved issues that impede normal interaction within the family or community? Safety. Safety is the state of being "safe" the condition of being protected against physical, social, spiritual, financial, political, emotional, occupational, psychological, educational or other types or consequences of failure, harm or any other event which could be considered non-desirable. Legal. Legal refers to the system of rules and guidelines which are enforced through social institutions to govern behavior of an individual, family, organization or community. 69
Educational/Vocational. Educational/vocational encompasses the educational needs of the child, children in the family as well as the parents and/or guardians. The vocational needs are pertinent to older youth, but also to the family unit. Individual LCFLs described challenges and successes the LCFLP referral process. Team members' reported experiences vary over time, by school, and school district. Bi-monthly reflection sessions were held with the full LCFL team to explore their experiences implementing the program. Data collected demonstrate that each LCFL experience differs. School community members including principals, guidance counselors, Student Assistance Team (SAT) team members and teachers impacted the LCFLs' ability, or opportunity, to implement the program. Reflection sessions explored the topic of how referrals were received. LCFLs described a wide range in this process that varied by school. Referrals came predominantly from school guidance counselors and principals but LCFLs also reported that they came after or during SAT meetings, through teachers, other school staff, parents, and through pursuing them on their own. This year the LCFL program worked with OPS to integrate the LCFLs into SAT meetings. In reflection sessions, LCFLs reported variation in having access to SAT meetings by school. One of the barriers that they reported was not always being alerted when a SAT meeting was scheduled, possibly due to not being on the OPS e-mail system. Some of the LCFLs reported greater success with getting SAT meeting notifications as the school year progressed. The LCFLs reported that being part of SAT meetings was valuable in their work in that it gave them opportunities to give parents their information and resources. They also identified SAT teams as an asset in relationship building with school staff and families, and as another way to get background information on children and families. Who did the LCFL Program Serve? The Learning Community Family Liaison program served approximately 305 students and their families across 14 schools in the Omaha Public Schools. Students served ranged from kindergarten through sixth grade. Of students served, 66% were male, 34% were female The majority of students were identified as Hispanic (39%) or African American (34%) Most students received free/reduced lunch (96%) Special education services were provided to 27% of the students Over one-third of students (35%) identified a home language other than English For the families reporting an income level, 68% reported earning less than $20,000 annually 70
Students had the following types of service domain goals assigned throughout their time in the program: Cultural/Spiritual, Educational/Vocational, Family, Living Situation, Medical, Mental Health, Safety and Social, and Other. Goals and Goal Attainment Goals were developed for each of students enrolled in the LCFL program. Goals set by the families and the liaisons spanned multiple categories with over half (51%) of the goals being set in the educational/vocational category. Goals for family made up another 16% of the total and goals for mental health were third most frequent (12%). Goal status was updated as families left the program. The table below shows the number of goals in each category and their ending status.
Goal Category Educational-Vocational Family Living Situation Mental Health Social Recreational Safety Medical Cultural/Spiritual Other Total
No Status 33 14 2 7 7 1 1 1 66
Regressing 3 0 1 1 5
Maintaining 17 5 3 6 1 1 33
Improving 46 13 4 8 7 1 1 2 82
Achieved 91 29 20 21 18 1 5 1 1 187
Not Achieved 51 15 8 16 9 1 2 102
Students and families were successful with 63% of the goals set for them (either achieved, maintained or improved) while 23% of the goals were in the not achieved to regressing categories. The remaining 14% of the goals were classified as having not status which could happen for a variety of reasons.
71
Success Stories Kellom Elementary: A family at Kellom was referred as the student was transitioning from another school. At the time of referral, the student was showing extreme behavioral issues, including defiance and aggression towards her mother. As part of the student's mental health goal, the FL program referred her to Alegent-Creighton therapy services. Together with the Alegent-Creighton therapist, the FL helped the family reconnect with their family doctor to resume medical and mental health prescriptions for both the student and her mother. After getting their medical and mental health needs in order, the mother is now able to focus on employment and her student's educational needs more. The FL is also continuing to work with the student on appropriate conduct and social skills. Jackson Elementary: A 4th grade student at Jackson Elementary School was referred to the Family Liaison Program because he was not following directions in class. According to the student's teachers, he was constantly running in class and needed additional assistance with his math. A Family Liaison met with family and the father has been helping the student with his math homework every evening. The father also enrolled the student in Tae Kwando Classes. As a result of the student's behavior improvement, everything at home is much better and the family's stress level has decreased. Service Plans There was variation in the quality of service planning by individual LCFL. Average scores from a review of 21 randomly selected service plans suggest that the quality of service planning conducted by LCFLs varied throughout the school year with the highest score being 25.67 and the lowest 30.63 out of a possible 28 points. In January 2012, a Service Plan Rubric was developed by the Family Liaison team with support from UNMCs evaluation team as a strategy for improving the quality of service plans written by LCFLs.1 The rubric addressed three areas: Responsiveness; SMART goals and Professionalism. The areas were determined by FSL management to ensure that the service plans aligned with program goals. Responsiveness required that the service plan responds to all of the unique needs of that student and family present at intake and that later emerge. SMART goals required that all goal areas be specific, measureable, achievable, realistic, and timely. Professionalism required that documentation accurately describe the work being done with the target child and family. The rubric assigned a point value to each of the fourteen indicators within the three overarching categories. A score of low quality received a zero, of medium quality received a one, and of high quality received a two. There were 28 points possible. The LCFL team voted to set a score of 23 1 Service plans are critical to the work of the FL. They include individualized student/family goals, service provision rationale and strategies for monitoring and adjusting their work with the family. 72
as their indicator of quality. In other words, the team agreed that a score of 23 indicated that the plan had addressed the student/family's need, was written in such a way that the goals were appropriate and achievable and accurately described the nature of their work with their clients. The evaluation team reviewed a randomly selected sample of 21 services plans at three different points across the year. Before the rubric was established, the average score was 17.8 points, in 2011-12the average score after the rubric was established was 20.3 points and in 2012-13 the average score was 20.47 points. For the 2013-14 school year, the average score across the 21 randomly selected plans was 23.60 points which is above the score set for quality (23 points). Progress was seen from last year to this year which was demonstrated across multiple plans. While there continued to be some variation across FSLs, 67% of the plans reviewed met the criterion score of 23. Student Achievement Teacher ratings of student ability in the areas of mathematics, reading, and writing increased overall for students from intake to discharge. The differences were only significant in the areas of mathematics and reading. Teachers were asked to rate students' ability in the areas of mathematics, reading and writing at intake. These ratings ranged from 1 to 3 with a rating of 1 indicating a student's ability to be `below expectations,' 2 indicating a student's ability `meets expectations' or 3 indicating a student's ability `exceeds expectations.' The pre/post ability ratings by teachers (n=175) are depicted in the following chart. Teacher Ratings of Students' Pre-Post Academic Achievement
For
proper
comparison, only
the ratings of
students
with
complete pre and
post data were
analyzed (n=175).
From intake to
discharge, more
students were
rated as meeting or
exceeding
70
60
50
40
30
20
10
0 Intake Math Discharge Math
Intake Reading
Discharge Reading
Intake Writing
Below Meets Exceeds
Discharge Writing
73
expectations in the subject of mathematics. The mean score difference was determined to be statistically significant using a one-tailed, paired samples t-test, t(175)=2.979, p=.003). Cohen's d was calculated and was in the low range, d=0.23. In the area of reading, significantly more students were rated as meeting or exceeding expectations at discharge. The difference in teacher ratings of reading was determined to be statistically significant using a one-tailed, paired samples t-test t(175)=2.565, p=.011). Cohen's d was calculated yielding an effect size in the low range, d=0.19. Finally, a one-tailed, paired samples t-test on teachers' rating of writing was found to not be significant (t(178)=1.6561, p=.10). Therefore, no effect size was calculated. In addition to teacher ratings, achievement scores were obtained from the school district. The achievement scores included the K-2 reading assessment and the NeSA scores from 2012-13 and 2013-14. A paired-samples, t-test was conducted using the pre to post data on the K-2 district developed reading assessment (t(102)=13.269, p=.000) and found to be significant. Cohen's d was calculated and was found above the zone of desired effect (d=1.31). Scale scores on the NeSA-R and NeSA-M declined from 2012-2013 to 2013-14 although the declines were not significant. It should be noted that with this program, families and students are not served for a full year, but rather for a period of 90 days. The scores may reflect more of the family situation and crisis level rather than the actual impact or success of the program over one year. Parenting Impacts Parent stress ratings from intake to discharge significantly decreased. Ratings changed from 6.66 at intake to 3.89 at discharge. At intake, parents rated their perceptions of stress on a scale of 1 to 10 with 1 indicating a low level of stress and 10 indicating a high level of stress. For proper comparison, only the ratings of parents with complete pre and post data were analyzed (n=65). Parents reported an initial mean stress rating of 6.29. The decrease of parent stress ratings from 6.29 to 3.17 was statistically significant using a one-tailed, paired-samples t-test, t=9.09, p<.001) which indicates that this change was not due to chance. Cohen's d was calculated and found to be above the zone of desired effects (d=1.16). The effect size for 2013-14 follows the pattern of significant decrease in parent stress found in the two previous years. 74
10.00 9.00 8.00 7.00 6.00 5.00 4.00 3.00 2.00 1.00
d=0.50 6.10 3.97 2011-12
Parent Stress Declined Three Year Trend
d=1.67 6.66 3.89
d=1.16 6.29 3.17
Intake Discharge
2012-13
2013-14
Overall, parents who completed a post survey "agreed to strongly agreed" that the program had impacted them. Of the four survey questions on client satisfaction, 100% of the participants were satisfied with the services provided when applicable to their situation. The four questions addressed looked at services provided, treatment by the liaison, timeliness of services and tendency to refer others. Surveys used a 5-point Likert scale to rate parent responses to "As a result of the Family Liaison Program..." There were 14 parent responses to this survey. Parent Rating of Impact of LCFL Program
Item
Rating Interpretation
I feel more confident in my ability to
4.00 Parent rating is positive, exactly
support my child academically.
between agree and strongly agree.
I have a better understanding of my
4.00 Parent rating is positive, leaning
children's academic needs.
towards agree.
I believe I have a better understanding of 3.92 Parent rating is positive, leaning
how to deal with stress.
towards agree.
I have a better understanding of the
4.00 Parent rating is positive, exactly
attendance requirement at my child's
between agree and strongly agree.
school.
Disagree Strongly=1 ; Disagree =2 ; Neutral =3 ; Agree =4 ; Strongly Agree =5
The evaluation team conducted a parent focus group during the summer of 2014. The focus group was held in conjunction with a family picnic and celebration at a local park. Families participating in the focus group mentioned several positives about the program. One aspect that all families agreed was positive was being connected to needed resources. Resources discussed included housing assistance and mental health referrals. One parent appreciated being connected especially after being turned down by other programs.
75
Parents expressed appreciation for their individual liaisons. Families felt they could call their liaisons whenever the need arose and reported that their liaisons were positive people. One improvement that families suggested was for the program's duration to go beyond the ninety day time limit. Families expressed a need to stay connected with a person or at least to have follow-up contacts after the ninety days were over. LCFL Program Conclusions and Implications The LCFL program was widely viewed as an asset within schools and to families for multiple reasons. The program continued to expand services with more families, despite an evolving model. Parental stress significantly decreased in a three year trend and teacher ratings of student reading and mathematics achievement levels increased. The quality of service plans improved almost three points from the previous year and the average score was above the criterion score set for quality. The families and students served by this program viewed it positively with the most common recommendation being extended time in the program. One thing to consider is the long-term effects of this short-term program. Are families able to sustain the lower levels of stress without receiving continued supports? For next year's evaluation, more information will be gathered on specific family stressors and areas of functioning in addition to breaking the services down by which Learning Community Center they are serving. 76
Overall Conclusions
The Learning Community funded a variety of programs to serve its mission of overcoming barriers to student achievement. The evaluation used diverse methods, combining quantitative and qualitative approaches, to describe and measure the quality of implementation, the nature of programming, and to report outcomes demonstrated by the elementary learning programs funded by the Learning Community including early childhood focused, elementary focused, and family focused programs. The LC served 10,076 students in the past program year. Overall, the evaluation results of the funded programs were positive and suggested that the Learning Community's efforts were accomplishing two overarching tasks: (1) programs appear to be using evaluation data for improvement and (2) examination of family or student data suggested they were showing improvement. Effect size improvements for participants in most programs were within the zone of desired effects.
The following table summarizes findings for the 2013-14 year by program. It includes the type of program, the number of student served, percentage of those students eligible for free or reduced price lunch in school, the type of measurement source used for calculation of effect size, and the effect size found. Results are most likely attributable to collective impact--the result of multiple efforts from the schools, parents, community partners, and the program described. A limitation of this evaluation is that not all measures are created equal. Meaning, effect size changes are reported for a variety of measures, some of which are standardized, norm referenced assessments and others were district created, criterion referenced assessments. Effect size improvements are naturally much larger on criterion referenced pre and post measures where growth is expected, as opposed to norm referenced, age-anchored tools where growth is more likely attributable to interventions.
Summary of Findings for 2013-141
Program
Number of Students Served
Early Childhood
128
Partnership
Program
Jump to Start Kindergarten
1,140
Extended Learning
2,096
Literacy Coaching
6,033
% FRPL 84% 69% 73% 84%
Measurement Bracken School Readiness Assessment Bracken School Readiness Assessment AimsWeb, MAP, DIBELS, NeSA NeSA-Reading CLASS Ratings
Effect Size 0.51 0.42 0.59 0.23 0.54-0.67
77
Program Family Literacy (LCCSO) Family Liaison Program Overall 1Collective impact
Number of Students Served 374
% FRPL 94%
305
96%
10,076
81%
Measurement BEST-Adult English Parent-Child Interactions K-2 Reading Parent Stress Declined K-2 Reading K-2 Mathematics
Effect Size 0.69 0.34 1.53 1.16 1.31 NS NS to 1.53
The table that follows compares effect sizes in the primary area of focus for the program found across programs over multiple years, where such results exist and where such a primary focus exists. If more than one primary focus, a range of effect sizes may be listed. If NS is noted, the results were not significant. A dash ( -- ) indicates no data were available to calculate a group result. NA indicates program did not exist in that program year. Note: For the 2011-12 year, extended learning had a range of effect sizes found but there was such variability in the data provided and examined, it was deemed most appropriate to report a group effect size beginning in 2012-13.
Comparison of Effect Size Impacts Across Years1
Program
2010-11
Effect Size
Early Childhood Partnership
NA
Prog
Jump Start to Kindergarten
0.58
Extended Learning
--
Literacy Coaching
--
Family Literacy: LCCSO
--
Family Liaison Program
--
Overall
--
1Collective impact
2011-12 Effect Size NA 0.63 -- NS -- Stress 1.39 Rdg 0.48 Wrtg 0.50 Math 0.47 NS to 1.39
2012-13 Effect Size NA
2013-14 Effect Size 0.51
0.59 0.30 0.56 0.41-1.06 Stress 1.67 Rdg 0.35 Wrtg 0.22 Math NS NS to 1.67
0.42 0.59 0.54-0.67 0.34-0.69 K2 Rdg 1.53 Stress 1.16 K2 Rdg 1.31 NS to 1.53
78
Summary Overall, the programs evaluated in this report served the students that the Learning Community targeted and provided quality programming. A total of 10,076 students were served this program year (81% eligible for free/reduced price meals). When available, outcomes related to academic achievement were measured and in general, showed that students benefitted from the additional resources, with strongest effect sizes found in external ratings of teaching/learning interactions (CLASS, 0.54-0.67), school readiness (0.42-0.51), and student achievement (NS to 1.53). Recommendations that follow pertain opportunities for improvement. Recommendations 1. The process and concept of `knowledge transfer' or sharing of best practices across buildings, districts, community agencies, and systems, has begun to be evaluated as part of the external program evaluation of the Learning Community and should be more comprehensively implemented over the next two to three years. 2. It is recommended that districts operating Jump Start to Kindergarten and other early childhood programs such as the Early Childhood Partnership Program at Kellom and Conestoga explore professional development in the area of instructional support with emphasis on Concept Development. 3. For extended learning programs, the evaluation team recommends determining some type of consistent measure or common type of assessment to measure student progress. 4. It is recommended that districts operating literacy or instructional coaching models continue meeting to discuss models, common practices, and common evaluation measures. 5. It is recommended that, where possible, longitudinal follow up data be gathered on students who participated in Learning Community funded programming, to determine whether long term benefits of participation are found. 6. It is recommended that the Learning Community Center of South Omaha and Omaha Public Schools continue to partner together to determine the most effective ways to support parents in helping their children with academic work and homework. 79
References Amendum, S. J., Vernon-Feagans, L., & Ginsberg, M. C. (2011). The effectiveness of a technologically facilitated classroom-based early reading intervention. The Elementary School Journal, 112(1), 107-131. doi:10.1086/660684 Beckett, M., Borman, G., Capizzano, J., Parsley, D., Ross, S., Schirm, A., & Taylor, J. (2009). Structuring out-of-school time to improve academic achievement: A practice guide (NCEE #2009-012). Washington, DC: National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education. Retrieved from http://ies.ed.gov/ncee/wwc/publications/practiceguides. Benson, P. L. (2006). All kids are our kids: What communities must do to raise caring and responsible children and adolescents . Jossey-Bass. Bierman, K. L., Domitrovich, C. E., Nix, R. L., Gest, S. D., Welsh, J. A., Greenberg, M. T., & ... Gill, S. (2008). Promoting academic and social-emotional school readiness: The Head Start REDI program. Child Development, 79(6), 1802-1817. doi:10.1111/j.1467-8624.2008.01227.x Bracken, B. (2002). Bracken school readiness assessment. San Antonio, TX: Harcourt Assessment Inc. Bryk, A., Lee, V., & Holland, P. (1993). Catholic schools and the common good. Cambridge, MA: Harvard University Press. Burchinal, M. R. (2008). How measurement error affects the interpretation and understanding of effect sizes. Child Development Perspectives, 2(3), 178-180. Burchinal, M. R., Peisner-Feinberg, E., Bryant, D. M., & Clifford, R. (2000). Children's social and cognitive development and child care quality: Testing for differential associations related to poverty, gender, or ethnicity. Applied Developmental Science, 4, 149­165. Burchinal, M., Vandergrift, N., Pianta, R., & Mashburn, A. (2010). Threshold analysis of association between child care quality and child outcomes for low-income children in pre-kindergarten programs. Early Childhood Research Quarterly, 25(2), 166­176. Buysse, V., Castro, D. C., & Peisner-Feinberg, E. (2010). Effects of a professional development program on classroom practices and outcomes for Latino dual language learners. Early Childhood Research Quarterly, 25(2), 194-206. doi:10.1016/j.ecresq.2009.10.001 Center for Applied Linguistics (2005). BEST Plus Test Administrator Guide. Washington, DC. Christ, T., & Wang, X. (2013). Exploring a community of practice model for professional development to address challenges to classroom practices in early childhood. Journal Of Early Childhood Teacher Education, 34(4), 350-373. doi:10.1080/10901027.2013.845630 Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed). Hillsdale, NJ: Erlbaum. Comfort, M., Gordon, P. R., & Unger, D. G. (2006). The Keys to Interactive Parenting Scale: A Window into Many Facets of Parenting. Zero to Three, 26(5), 37-44. Crosnoe, R., Johnson, M. K., & Elder, G. H. (2004). Intergenerational bonding in school: The behavioral and contextual correlates of student-teacher relationships. Sociology of Education, 77(1), 60-81. Dunn, L. M. & Dunn, L. M. (2007). Peabody picture vocabulary test, fourth edition. Minneapolis, MN: Pearson Assessments. 80
Dunst, C. J., Trivette, C. M., & Raab, M. (2013). An implementation science framework for conceptualizing and operationalizing fidelity in early childhood intervention studies. Journal Of Early Intervention, 35(2), 85-101. doi:10.1177/1053815113502235 Elish-Piper & L'Allier (2011). Examining the relationship between literacy coaching and student reading gains in grades k-3. The Elementary School Journal, 112 (1), 83-106. Emmer, E. T., & Stough, L. M. (2001). Classroom management: A critical part of Educational Psychology, with implications for teacher education. Educational Psychologist, 36(2), 103-112. Fantuzzo, J. W., Gadsden, V. L., & McDermott, P. A. (2011). An integrated curriculum to improve mathematics, language, and literacy for Head Start children. American Educational Research Journal, 48(3), 763-793. doi:10.3102/0002831210385446 Fixsen, D. L., Naoom, S. F., Blase, K. A., & Friedman, R. M. (2005). Implementation research: A synthesis of the literature. Fountas, I. C., & Pinnell, G. S. (2009). Fountas & Pinnell Leveled Literacy Intervention: Lesson Guide, Volume 1 Orange System: Lessons 1-30. Heinemann. Fox, L., Hemmeter, M., Snyder, P., Binder, D., & Clarke, S. (2011). Coaching early childhood special educators to implement a comprehensive model for promoting young children's social competence. Topics In Early Childhood Special Education, 31(3), 178-192. doi:10.1177/0271121411404440 Gersten, R., Dimino, J., Jayanthi, M., Kim, J. S., & Santoro, L. (2010). Teacher Study Group: Impact of the professional development model on reading instruction and student outcomes in first grade classrooms. American Educational Research Journal, 47(3), 694-739. doi:10.3102/0002831209361208 Hamre, B., Goffin, S. & Kraft-Sayre, M. (2009). Measuring and improving classroom interactions in early childhood settings. http://www.teachstone.org/wp- content/uploads/2010/06/CLASSImplementationGuide.pdf Harms, T., Clifford, R. M., & Cryer, D. (2005). Early childhood environment rating scale (Rev. ed.). New York: Teachers College Press. Hart, B., & Risley, T. (1995). meaningful differences in the everyday experience of young American students. Baltimore: Paul H. Brookes Publishing Co. Hattie, J. (2009). Visible Learning: A synthesis of over 800 meta-analyses relating to achievement. New York, NY: Routledge. Hemmeter, M., Snyder, P., Kinder, K., & Artman, K. (2011). Impact of performance feedback delivered via electronic mail on preschool teachers' use of descriptive praise. Early Childhood Research Quarterly, 26(1), 96-109. doi:10.1016/j.ecresq.2010.05.004 Hsieh, W., Hemmeter, M., McCollum, J. A., & Ostrosky, M. M. (2009). Using coaching to increase preschool teachers' use of emergent literacy teaching strategies. Early Childhood Research Quarterly, 24(3), 229-247. doi:10.1016/j.ecresq.2009.03.007 Knoche, L. L., Kuhn, M., & Eum, J. (2013). "More time. More showing. More helping. That's how it sticks": The perspectives of early childhood coachees. Infants & Young Children, 26(4), 349-365. doi:10.1097/IYC.0b013e3182a21935 81
Ladd, G. W., Birch, S. H. and Buhs, E. S. (1999), Children's Social and Scholastic Lives in Kindergarten: Related Spheres of Influence?. Child Development, 70: 1373­1400. doi: 10.1111/1467- 8624.00101. Landry, S. H., Swank, P. R., Anthony, J. L., & Assel, M. A. (2011). An experimental study evaluating professional development activities within a state funded pre-kindergarten program. Reading And Writing, 24(8), 971-1010. doi:10.1007/s11145-010-9243-1 Larrabee, A. (2007). Predicting academic achievement through kindergarten screening: An evaluation of development and school readiness measures (Doctoral dissertation). Retrieved from: Dissertation Abstracts International section A: Humanities and social sciences. (AAI3228216). Mashburn, A. J., Downer, J. T., Hamre, B. K., Justice, L. M., & Pianta, R. C. (2010). Consultation for teachers and children's language and literacy development during pre-kindergarten. Applied Developmental Science, 14(4), 179-196. doi:10.1080/10888691.2010.516187 Mayer, R. E., & Moreno, R. (2003). Nine ways to reduce cognitive load in multimedia learning. Educational psychologist, 38(1), 43-52. Nebraska Department of Education, 2011-2012 State of the Schools Report: A Report on Nebraska Public Schools. Retrieved from: http://reportcard.education.ne.gov/Main/Home.aspx. Neuman, S. (2006). N is for nonsensical. Educational Leadership, 64(2), 28-31. Neuman, S. B., & Wright, T. S. (2010). Promoting language and literacy development for early childhood educators. The Elementary School Journal, 111(1), 63-86. doi:10.1086/653470 Panter, & Bracken, B. (2009). Validity of the Bracken school readiness assessment for predicting first grade readiness. Psychology in the schools, 46(5), 397-409. Patton, M. Q. (2012). Essentials of Utilization-Focused Evaluation. Thousand Oaks, CA: Sage Publications. Pianta, R. (1992). Child Parent Relationship Scale. Charlottesville, VA: University of Virginia, Center for Advanced Studies on Teaching and Learning. Pianta, R. C., Mashburn, A. J., Downer, J. T., Hamre, B. K., & Justice, L. (2008). Effects of web-mediated professional development resources on teacher-child interactions in pre-kindergarten classrooms. Early Childhood Research Quarterly, 23(4), 431-451. doi:10.1016/j.ecresq.2008.02.001 Pianta, R., LaParo, K., & Hamre, B. (2008). Classroom assessment scoring system (CLASS). Baltimore, MD: Brookes Publishing. Powell, D. R., Diamond, K. E., Burchinal, M. R., & Koehler, M. J. (2010). Effects of an early literacy professional development intervention on head start teachers and children. Journal Of Educational Psychology, 102(2), 299-312. doi:10.1037/a0017763 Powell, D. R., Steed, E. A., & Diamond, K. E. (2010). Dimensions of literacy coaching with Head Start teachers. Topics In Early Childhood Special Education, 30(3), 148-161. doi:10.1177/0271121409341197 Powell, D.R., Diamond, K.E., & Cockburn, M.K. (2013). Promising approaches to professional development for early childhood educators. In Saracho, O.N., Spodek, B. (Eds.), Handbook of Research on the Education of Young Children (pp. 385-392). New York, NY, US: Routledge Publishers. 82
Ramey, C. T., & Ramey, S. L. (1998). Early intervention and early experience. American Psychologist, 53, 109-120. Raver, C., Jones, S. M., Li-Grining, C. P., Metzger, M., Champion, K. M., & Sardin, L. (2008). Improving preschool classroom processes: Preliminary findings from a randomized trial implemented in Head Start settings. Early Childhood Research Quarterly, 23(1), 10-26. doi:10.1016/j.ecresq.2007.09.001 Roeser, R. W., Eccles, J. S., & Sameroff, A. J. (2000). School as a context of early adolescents' academic and social-emotional development: A summary of research findings. The Elementary School Journal, 443-471. Shonkoff, J. P., & Phillips, D. A. (2000). From neurons to neighborhoods: The science of early childhood development. National Academy Press. Stipek, D. J. (1998). Motivation to learn: From theory to practice (2nd ed.). Boston: Allyn and Bacon. ED 369 773. Vescio, V., Ross, D., & Adams, A. (2008). A review of research on the impact of professional learning communities on teaching practice and student learning. Teaching And Teacher Education, 24(1), 80-91. doi:10.1016/j.tate.2007.01.004 Wasik, B. A., & Hindman, A. H. (2011). Improving vocabulary and pre-literacy skills of at-risk preschoolers through teacher professional development. Journal Of Educational Psychology, 103(2), 455-469. doi:10.1037/a0023067 Zimmerman, I., Steiner, V., & Pond, R. (2011-English, 2012-Spanish). Preschool Language Scale, 5th Edition. San Antonio, TX: The Psychological Corporation. 83
Report prepared by: St. Clair, L.; Johnson, J.; & Stone, T. (2014). Section IV. Evaluation report of programs funded by elementary learning centers 2013-14. Omaha, NE: Learning Community of Douglas and Sarpy Counties, http://learningcommunityds.org. Director of this Learning Community Evaluation Lisa St. Clair, Ed.D. Assistant Professor, MMI & Pediatrics Munroe-Meyer Institute University of Nebraska Medical Center 985450 Nebraska Medical Center Omaha, NE 68198-5450 [email protected] (402) 559-3023 (office) and (402) 677-2684 (cell)
Project Director-Jolene Johnson, Ed.D. Database Manager-Amber Rath
Evaluation Team
Sarah Baird
Karime Rios
Nicole Buchholz
Holly Reeves
Linda Comfort
Adriana Rey
Colleen Gibilisco
Regina Robbins
Becky Harris
Colleen Schmit
Jennifer Harmon
Megan Shepherd
Kerry Miller
Abbey Siebler
Amanda Mills
Terry Stone
Kristina Norwood
Cynthia Villanueva
Kari Price
Becky Zessin
Special thanks to the assistance of research/evaluation staff and administration of district and agency partners, as well as to the staff of the Learning Community. Thanks also to our student data entry team from University of Nebraska at Omaha, College of Arts and Science, Psychology Department (Jake Cunningham, Samantha St. Clair, and Sean St. Clair).
Funding for this external program evaluation was provided through the Learning Community of Douglas and Sarpy Counties. http://learningcommunityds.org
84
Collaborate Evaluate
Improve
85
Section II ­ Student Demographics
Section II and III prepared by external evaluator, Dr. Bert Jackson, in cooperation with Brian Gabrial, Learning Community Finance Director. Section II ­ Student Demographics This section of the report provides general enrollment information, as well as data associated with student eligibility for free or reduced price lunch (FRL) and ELL (English Language Learner) services for the 2013-2014 school year. Comparative data from previous years are also presented. The Nebraska Department of Education (NDE) provided the data included in this section. Enrollment data are submitted to NDE by each school district and reflect counts as of the last Friday of September 2013. The NDE refers to these data as the Fall Membership1 Demographic Information by Subcouncil Nebraska Statute establishes six Achievement Subcouncils within the two-county area of the Learning Community, dividing the population among the Subcouncils as equally as feasible. In 2011, the Subcouncil boundaries were changed because population shifts had affected proportional representation on the Learning Community Coordinating Council. Therefore, comparisons among the Subcouncils across years can only be made for the past three school years (2011-2012 and 2012-2013) since Subcouncils were composed of different schools in previous years. Table II.1 (page 88) presents demographic data for each Subcouncil for the 2013-2014 school year, including the total number of enrolled students, percent eligible for free or reduced lunch (FRL), and percent of English Language Learners (ELL). 1 The Fall Membership counts are used rather than end-of-year counts for consistency across years. For that reason, the numbers in this report may differ from those appearing in the NDE State of the Schools Report. 86
Table II.1: 2013-2014 Total Enrollment, Free and Reduced Lunch, and ELL by Subcouncil
K-6 7-12 Subcouncil Total K-6 7-12 Subcouncil Total K-6 7-12 Subcouncil Total K-6 7-12 Subcouncil Total K-6 7-12 Subcouncil Total K-6 7-12 Subcouncil Total K-6 7-12 Subcouncil Total
SC 1 1 1 2 2 2 3 3 3 4 4 4 5 5 5 6 6 6 All LC All LC All LC
Total Enrol l ment 8,363 6,823 15,186 8,743 7,031 15,774 9,584 5,866 15,450 12,182 10,671 22,853 12,379 10,210 22,589 12,858 9,989 22,847 64,109 50,590 114,699
Number FRL 3,526 3,624 7,150 8,021 4,786 12,807 5,368 2,845 8,213 2,489 1,811 4,300 8,407 6,303 14,710 2,261 1,569 3,830 30,072 20,938 51,010
Percent FRL 42.2% 53.1% 47.1% 91.7% 68.1% 81.2% 56.0% 48.5% 53.2% 20.4% 17.0% 18.8% 67.9% 61.7% 65.1% 17.6% 15.7% 16.8% 46.9% 41.4% 44.5%
Number ELL 352 185 537 1,556 280 1,836 1,372 201 1,573 315 41 356 3,181 489 3,670 170 23 193 6,946 1,219 8,165
Percent ELL 4.2% 2.7% 3.5% 17.8% 4.0% 11.6% 14.3% 3.4% 10.2% 2.6% 0.4% 1.6% 25.7% 4.8% 16.2% 1.3% 0.2% 0.8% 10.8% 2.4% 7.1%
State
292,940
131,849
45.0%
17,695
6.0%
Student enrollment in the six Subcouncils ranges from 15,186 in Subcouncil 1 to 22,853 in Subcouncil 4.
The percentage of students who qualify for FRL varies greatly among the Subcouncils, from approximately 17% and 19% in Subcouncils 6 and 4, respectively, to 81% in Subcouncil 2. Subcouncils 1, 3, and 5 also have higher percentages of FRL than the Learning Community total of 44.5%.
At 16.2%, Subcouncil 5 has the highest percentage of English Language Learners. Subcouncils 2 and 3, with 11.6% and 10.2%, also have a higher percentage than that of the in Learning Community as a whole, which is 7.1%.
87
Demographic Comparisons Across Years Table II.2 compares enrollments for the past three years, and Figures II.1 and II.2 (page 4) compare FRL and ELL numbers in 2013-2014 with 2012-2013.
Table II.2: 2011-2012, 2012-2013, and 2013-2014 Enrollment by Subcouncil
2011-2012 Enrollment
2012-2013 Enrollment
2013-2014 1 Year Percent 2 Year Percent
Enrollment
Change
Change
Subcouncil 1
14,676
14,988
15,186
1.32%
3.48%
Subcouncil 2
16,223
15,917
15,774
-0.90%
-2.77%
Subcouncil 3
14,809
15,013
15,450
2.91%
4.33%
Subcouncil 4
22,408
22,676
22,853
0.78%
1.99%
Subcouncil 5
22,050
22,254
22,589
1.51%
2.44%
Subcouncil 6
20,728
21,650
22,847
5.53%
10.22%
Total
110,894
112,498
114,699
1.96%
3.43%
Enrollment in the Learning Community increased by approximately 2% over the previous
year (approximately 2,200 students). Between 2011-2012 and 2013-2014 the increase
was 3.43%
The enrollment in all Subcouncils except Subcouncil 2 increased. Subcouncil 2, which covers the northeastern part of Omaha Public Schools, has declined by 2.77% over the three-year period.
The increase in Subcouncil 6 (10.22%) is considerably greater than any other Subcouncil. Subcouncil 6 is comprised of the districts in the southwest portion of the Learning Community: Papillion-La Vista, Elkhorn, Gretna, Douglas County West and Springfield Platteview.
Figure II.1: 2012-2013 and 2013-2014 Free and Reduced Lunch by Subcouncil
100.0%
80.0%
80.9% 81.2%
60.0% 40.0%
44.2% 45.0%
44.0% 44.5%
46.9% 47.1%
52.2% 53.2%
62.9% 65.1%
20.0%
18.3% 18.8%
16.6% 16.8%
0.0%
State
LC
SC 1
SC 2
2012-13 Percent Free and Reduced Price Lunch
SC 3
SC 4
SC 5
SC 6
2013-14 Percent Free and Reduced Price Lunch
88
The percentage of Learning Community students who qualify for free or reduced priced lunch increased by one-half percent (.5%), while in the State (including the Learning Community), the increase was .8%.
Economic diversity does not show any indication of movement toward geographic equalization. o In all six Subcouncils, the percentage of students who qualify for FRL increased slightly. o The increase in Subcouncils 1, 2, 4, and 6 was less than one percent. o Subcouncil 5, which has the second highest percentage of FRL (65.1%), had the greatest increase, 2.2%.
Figure II.2: 2012-2013 and 2013-2014 ELL by Subcouncil 25.0%
20.0%
15.0%
10.0% 5.0%
6.0% 6.0%
0.0%
State
6.7% 7.1%
11.6% 10.0%
9.8% 10.2%
15.4% 16.2%
3.2% 3.5%
1.5% 1.6%
LC
SC 1
SC 2
2012-13 Percent English Language Learners
SC 3
SC 4
SC 5
2013-14 Percent English Language Learners
1.1% 0.8% SC 6
The percentage of ELL increased slightly in the Learning Community and remained constant in the State. Subcouncil 2 experienced an increase of 1.6%, while the change in all other Subcouncils was less than one percent. Free and Reduced Lunch Concentration Figure II.3 (page 91) provides additional information about the concentration of poverty within the Learning Community. The graph shows the number of schools that have FRL percentages within ranges of 10%. The first bar in each set represents the average number of schools in each interval in the previous four years and the second bar shows the number in the 2013-2014 school year.2
2 Over the four-year period, the number of schools increased. A total of 200 schools are included in the four-year average. In 2013-2014 the Learning Community included 205 schools. 89
Figure II.3: Number of Learning Community Schools in FRL Intervals of 10% Comparing 20132014 with the Previous Four-Year Average
50
45
40
35
30
29 26
25
20
15
10
5
0 0-10
37 33 10-20
25 18 20-30
16 13 30-40
19 16 40-50
20 18 50-60
12 13 60-70
15 15 70-80
17 17 80-90
27 19 90-100
Prior 4 Year Average 2013-14 School Year
A primary goal of Open Enrollment is to improve the economic diversity of Learning Community schools. Progress toward this goal would be illustrated by an increase in the number of schools in the middle ranges of the graph and a decline in the number on each end; however, that trend is not occurring. Generally, the number of low poverty schools is decreasing; the number of high poverty schools is increasing; and the number of schools in the middle ranges has remained fairly constant. The exception is the number of schools in the 20% to 30% range. In that range the number has increased, indicating that schools previously in the lowest two ranges (0 to 20%) have likely moved into the 20% to 30% range. In 2013-2014, approximately half (50.2%) of the schools in the Learning Community could be described as economically segregated. Fifty-nine (59) schools have FRL percentages of 20% or less and 44 have 80% or more. The four-year average in these high and low ranges is 51% (102 of the 200 schools). There are more high poverty and fewer low poverty schools now than in the past. Comparing the four-year average with 2013-2014, seven fewer schools had FRL percentages of 20% or less, and eight more schools fell in the 80% and above range. The proportion of schools in the middle ranges (30 to 70 percent) is slightly lower in 20132014 than in the four-year average. The previous four-year average number of schools within that range is 65 (32.5% of the 200 schools). In 2013-2014, 62 schools (30.2% of 205 schools) fell in the 30% to 70% range. The greatest increases are in the 20% to 30% and the 90% to 100% ranges. Figures II.4 and II.5 (page 92) provide a comparison of Learning Community schools with other Nebraska schools. Figure II.4 shows the percentage of schools in Nebraska (excluding Learning Community schools) in each of the 10% ranges of FRL and Figure II.5 shows the percentages in the Learning Community.
90
Figure II.4: 2013-2014 Percentage of Nebraska Schools in FRL Intervals of 10% (Excluding Learning Community)
25.0% 20.0% 15.0%
12.7%
22.7%
21.4%
16.1%
10.0% 5.0% 0.0%
2.8% 0-10
6.0% 10-20
20-30
30-40
40-50
50-60
8.6% 60-70
4.6% 70-80
2.7% 80-90
1.9% 90-100
Figure II.4 illustrates that most Nebraska schools fall in the middle ranges of free and reduced lunch concentrations, and few schools fall in the very low and very high ranges.
More than a third (37.5%) of all Nebraska schools outside the Learning Community fall in the middle two ranges (40% to 60% FRL), and more than two-thirds of the schools (68.8%) have FRL percentages between 30% and 70%. These percentages are very similar to the previous year (not shown). Only 4.6% of the Nebraska schools outside the Learning Community have FRL percentages of more than 80%, and only 8.8% of the schools have FRL percentages of 20% or less, again very similar to the previous year.
Figure II.5 shows the distribution of schools within the Learning Community. The contrast in the two graphs is dramatic. In the Learning Community, a far greater proportion of schools fall in the very high and very low ranges, while fewer schools are in the middle ranges.
Figure II.5: 2013-2014 Percentage of Learning Community Schools in FRL Intervals of 10% 25.0%
20.0%
16.1%
15.0% 10.0%
12.7%
12.2%
13.2%
9.8%
7.8% 6.3%
6.3%
7.3%
8.3%
5.0%
0.0% 0-10
10-20
20-30
30-40
40-50
50-60
60-70
70-80
80-90
90-100
Only 17.6% of the Learning Community schools fall in the 40% to 60% range, approximately 20% less than in the rest of the State.
91
Expanding the range results in similar discrepancies between the Learning Community and the State. In the Learning Community only 30.2% of the schools are in the 30% to 70% range while in the rest of the State more than twice that number (68.8%) are within the 30% to 70% range. In the Learning Community 28.8% of the schools have 20% or fewer students who qualify for FRL, while in the rest of the State only 8.8% fall in this range. Similarly, in 21.5% of the Learning Community schools, more than 80% of the students qualify for FRL, while in the rest of the state only 4.6% of the schools fall within that high poverty range. These data demonstrate the dramatic difference in the economic diversity of Learning Community schools and other schools in Nebraska. The majority of schools in Nebraska are relatively diverse economically, while the majority of schools in the Learning Community are segregated economically into schools with relatively low and relatively high concentrations of poverty. Students outside the Learning Community are more likely to be enrolled in an economically diverse school, while students in the Learning Community are more likely to be enrolled in an economically segregated school. These comparisons were almost identical to those made in the 2013 Evaluation Report. It does not appear that there is much progress toward greater diversity in Learning Community schools. There has been little change in the number of schools in the middle ranges and at the extremes. The majority of schools in the Learning Community continue to be economically segregated. 92
Section III ­ Open Enrollment
Section III ­ Open Enrollment This section of the report describes the status of Open Enrollment. The Nebraska Department of Education (NDE) provided enrollment data, and Learning Community school districts provided information about the number of Open Enrollment applications and their approval. Before presenting the Open Enrollment data, it is important to have a common understanding of application procedures and the difference between Open Enrollment and Option Enrollment. Application Process Each year applications are available in November and must be submitted to the requested districts by March 15th. Applications may be submitted to multiple districts and may list as many as three schools of choice in each district. The applications include self-reported eligibility for free or reduced price lunch (FRL) based on federal guidelines provided with the application. School districts approve or deny an application based on available capacity and following the priority sequence outlined in the Learning Community Diversity Plan:3 1) First priority goes to students who have a sibling who currently attends, and will also be attending, the requested school the year the Open Enrollment applicant first attends. 2) Second preference goes to students who contribute to the socioeconomic diversity of the school. In schools with a percentage of students qualifying for FRL that is greater than the total of all schools in the Learning Community (approximately 44.5% in 2013-2014), the priority goes to students who do not qualify for FRL, and in schools that have a lower percentage of FRL-eligible students than the Learning Community total, the priority goes to students who do qualify for FRL. 3) After approving all applicants in the first and second priority categories, all other applications become eligible. At each level of priority, if there is not capacity to accept all applications in that category, a lottery is conducted. Districts must notify applicants of approval or denial by April 5th, and applicants must notify the districts of their acceptance by April 25th. Although families may apply to multiple school districts, they may accept Open Enrollment in only one district. As required by Nebraska Statute, the number of applications received and approved is submitted to the Learning Community by member school districts in September of each year. 3 Available capacity at each grade, in each school, is determined through a systematic process jointly developed by school district and Learning Community Coordinating Council representatives. Each year school districts submit documentation of capacity to the Learning Community's Chief Executive Officer. 93
Open and Option Enrollment Beginning with the 2010-2011 school year, school districts' reports to the Nebraska Department of Education (NDE) included identifying students as open enrolled or option enrolled. Open Enrollment refers to students who transfer to another school or school district through the Learning Community's Open Enrollment process, which went into effect in the 2010-2011 school year. Option Enrollment designates students who transferred between school districts prior to the 2010-2011 school year through a process that was implemented Statewide in 1993. Students who reside outside the Learning Community two-county area, and transfer to a Learning Community school, continue to be classified as Option Enrollment. An important difference between Option and Open Enrollment is the application of the priority sequence described above. Under Option Enrollment districts were not required to give priority to students who could potentially improve the diversity of a school. Learning Community schools may currently have both Open Enrollment and Option Enrollment students. All students who transferred among Learning Community districts, beginning with the 2010-2011 school year, are classified as Open Enrollment students. Those who transferred prior to the 2010-2011 school year are, for the most part, still classified as Option Enrollment students, although districts report that some students who previously were classified as Option Enrollment have changed their status to Open Enrollment by going through the Open Enrollment process. One other variation is noteworthy. Some districts use the Open Enrollment process for some students who request transfers to another school within their resident district, while others do not. The Status of Open Enrollment Table III.1 (page 10) shows the number of new Open Enrollment students and the percent qualifying for FRL in each of the four years of Open Enrollment. These numbers reflect each year's enrollment as reported in the Nebraska Department of Education Fall Membership. The total represents the number of students who have accessed Open Enrollment and who, at one point in time, were enrolled as Open Enrollment students. It does not represent the total number enrolled each year. 94
Table III.1 Number of Students Open Enrolled for the First Time in 2010-2011 through 20132014 and Percent FRL
YEAR 2010-2011 2011-2012 2012-2013 2013-2014 Total
NUMBER NEW OPEN ENROLLMENT STUDENTS IN FALL MEMBERSHIP 2,563 2,463 2,315 2,168 9,509
PERCENT NEW OPEN ENROLLMENT STUDENTS WHO QUALIFY FOR FRL 41.98% 44.62% 42.33% 43.91%
LEARNING COMMUNITY PERCENT FRL 41.86% 43.48% 43.96% 44.47%
The number of students who open enroll has remained fairly constant, with just a slight decline each year.
Each year, the percentage of new Open Enrollment students who qualify for FRL has been similar to that of the Learning Community as a whole; although, in 2012-2013 and 20132014, the percentage was slightly lower than the Learning Community, while in the first two years it was slightly more than that of the Learning Community.
Table III.2 shows the total number of Open Enrollment students in each year of the program. The total each year includes the new students reported in Table III.1 and the number of Open Enrollment students from previous years who continued as Open Enrollment students.
Table III.2: Total Number of Open-Enrolled Students and FRL Percentages for 2010-2011 through
2013-2014
YEAR
TOTAL NUMBER OPEN ENROLLMENT STUDENTS IN FALL MEMBERSHIP
PERCENT OF TOTAL OPEN ENROLLMENT STUDENTS WHO QUALIFY FOR FRL
LEARNING COMMUNITY PERCENT FRL
2010-2011
2,563
41.98%
41.86%
2011-2012
4,334
42.52%
43.48%
2012-2013
5,769
40.65%
43.96%
2013-2014
6,535
41.68%
44.47%
The total number of current Open Enrollment students (6,535) is 2,974 less than the total number of new Open Enrollment students across the four years of the program (9,509). These 2,974 students were, at one time, open-enrolled and in 2013-2014 are no longer classified as Open Enrollment students. In 2011, 2012, and 2013, a total of 796 Open Enrollment students were seniors. In addition to their graduation, a number of factors might account for the drop-off. Moving out of the Learning Community
95
Moving into the Open Enrollment district, therefore becoming a resident student Moving to a different school district within the Learning Community and choosing to attend a school in that district Returning to their resident school and district Each year, as shown in Table III.1 (page 10), the percentage of newly enrolled FRL Open Enrollment students has been similar to that of the Learning Community. However, in the past two years, the total percentage of currently enrolled Open Enrollment students is somewhat less than that of the Learning Community total: 40.65% in 2012-2013 and 41.68% in 2013-2014, approximately 3% less than the total in the Learning Community. This means a higher percentage of FRL Open Enrollment students than Non-FRL students have been among those who were once classified as Open Enrollment and are no longer. This may be related to the fact that families with lower incomes tend to change residences more frequently than higher income families. Many of the explanations for a student's change in classification from Open Enrollment to resident (described above) involve moving to a new residence. Table III.3 (page12) shows the number of Open Enrollment students in each grade, in all four years of the program and the degree of change (increases or decreases) from year to year. The numbers in the 2013-2014 column are cumulative. They include students who enrolled for the first time in the 2013-2014 school year, as well as those who enrolled in the three previous school years and continued to be open enrolled in the 2013-2014 school year. The number at a particular grade reflects students who newly enrolled at that grade level and those who were one grade below that grade in 2012-2013. For example, the 2013-2014 third grade enrollment of 614 includes 2010-2011 kindergartners, 2011-2012 first graders, and 2012-2013 second graders who continued as third grade Open Enrollment students in 2013-2014 and any third grade students who were newly enrolled in 2013-2014. This cohort of students is highlighted in yellow on Table III.3. 96
Table III.3: Number of Open Enrollment Students by Grade and Percent Change by Year
GRADE LEVEL KG 1 2 3 4 5 6 7 8 9 10 11 12 Total
2010-11 OPEN ENROLLMENT STUDENTS IN FALL MEMBERSHIP 512 165 182 150 150 124 118 219 105 387 152 167 132 2,563
2011-12 OPEN ENROLLMENT STUDENTS IN FALL MEMBERSHIP 605 576 260 283 250 234 258 273 286 385 386 287 251 4,334
2012-13 OPEN ENROLLMENT STUDENTS IN FALL MEMBERSHIP 583 645 639 313 374 324 311 371 349 482 485 480 413 5,769
2013-14 OPEN ENROLLMENT STUDENTS IN FALL MEMBERSHIP 572 587 661 614 353 368 379 410 420 562 530 538 541 6,535
PERCENT
PERCENT
PERCENT
CHANGE FROM CHANGE FROM CHANGE FROM
2010-11 TO 2011- 2011-12 TO 2012- 2012-13 TO 2013-
12
13
14
18.16%
-3.64%
-1.89%
249.09%
11.98%
-8.99%
42.86%
145.77%
3.44%
88.67%
10.60%
96.17%
66.67%
49.60%
-5.61%
88.71%
38.46%
13.58%
118.64%
20.54%
21.86%
24.66%
35.90%
10.51%
172.38%
22.03%
20.34%
-0.52%
25.19%
16.60%
153.95%
25.65%
9.28%
71.86%
67.25%
12.08%
90.15%
64.54%
30.99%
69.10%
33.11%
13.28%
In general the number of Open Enrollment students in a cohort increases as it moves through the grades. For example, the 2010-2011 the first grade cohort increased by 95 students (from 165 to 260) in second grade, by 53 in third grade, and by 40 in fourth grade. This increase occurs in each cohort of students with few exceptions. Following the increases in the cohort of 2010-2011 grade nine students is of particular interest. In 2010 there were 387 9th grade Open Enrollment students. In 2011 this cohort (10th grade) remained, essentially, the same. However, in the next two years (11th and 12th grade), the enrollment in this cohort increased by 94 and 61 students, respectively. Therefore, at least 94 students enrolled as Open Enrollment students, for the first time, in their junior year of high school and 61 students, in their senior year. It is important to understand, however, that some of these students had undoubtedly attended the same school previously as residents and became Open Enrollment students as a means of staying in that school after moving to another school district. The last three columns of the table show the percentage of change at each grade level from year to year. The percentage of increase at each grade level was understandably greatest between the first and second year of the program (approximately 69%). The growth continued in 20122013 and 2013-2014 but to a lesser degree (approximately 33% and 13%, respectively). At some point, as increasingly larger numbers of Open Enrollment students' progress through the grades, the number will become more consistent from year to year. That is, at some point, Open Enrollment will reach its maximum capacity and remain at approximately the same number from year to year. If the current trend continues this stability will likely occur within the next two to three years.
97
Open Enrollment and Diversity As previously described, Open Enrollment potentially contributes to a school's economic diversity in two ways: 1) Students who qualify for FRL enroll in schools with relatively lower percentages of FRL students. 2) Students who do not qualify for FRL enroll in schools with relatively higher percentages of FRL students.
Table III.4 shows the number of FRL-eligible Open Enrollment students who are enrolled in schools that are below the percentage of the total Learning Community (44.5%) and the number of students who do not qualify for FRL enrolled in schools that have FRL percentages above that of the total Learning Community. It is important to understand, that we cannot say the general diversity of the schools has actually changed to the degree the table might imply. Open-enrolled students' resident school is not known. The FRL-eligible student who transfers to a school with a relatively low percentage of FRL students, but whose resident school also has a relatively low concentration of FRL, has not positively affected diversity. The school she or he left is potentially less diverse because of the transfer. The same is true of the Non-FRL student who enrolls in a school with a large proportion of FRL. If that student's resident school is also a high FRL school, diversity has likely not been improved. Although they may positively affect the diversity of the school in which they open- enroll, their transfer potentially has a negative effect on the diversity of the school they left.
Table III.4: FRL Open Enrollment Students in Schools with Lower Concentrations of FRL than the Learning Community Total and Non-FRL Open Enrollment Students Enrolled in Higher FRL Schools
YEAR 2010-2011 2011-2012 2012-2013 2013-2014
Total Open Enrollment 2,563 4,334 5,769 6,535
Number FRL in Schools with FRL Percentage < LC Total 647 908 1,500 1,659
Percent FRL in Schools with FRL Percentage < LC Total 25.24% 20.95% 26.00% 25.39%
Number Non- FRL in Percent Non- FRL in
Schools with FRL Schools with FRL
Percentage > LC Percentage > LC
Total
Total
233
9.09%
267
6.16%
548
9.50%
630
9.64%
Approximately 35% of the Open Enrollment students are enrolled in schools that follow the intention of the Learning Community Diversity Plan. More than a quarter (25.39%) of the Open Enrollment students who qualify for FRL are enrolled in schools with relatively lower percentages of FRL, and 9.64% of the students who do not qualify for FRL are enrolled schools with relatively higher percentages of FRL. Whether they are contributing to diversity, however, is not known. To determine the effect on school diversity would require knowing the FRL percentage of their resident school, as well as the FRL percentage in the school in which they open-enrolled.
District Participation in Open Enrollment This section provides Open Enrollment information for each of the 11 member school districts, including the number of applications received and approved and the number of students designated as Open Enrollment students.
98
As required by Nebraska Statute, application information was submitted to the Learning Community by each school district. Enrollment data were supplied by NDE and reflect Fall Enrollment Membership (counts on the last Friday of September). Table III.5 shows the number of Open Enrollment applications received and approved and the number enrolled in the 2013-2014 school year. It is important to be aware of differences in the reporting dates for the applicationrelated information to the Learning Community and enrollment information to NDE for Fall Membership. School districts are required, by statute, to report their application and approval data to the Learning Community by September 1 of each year. For consistency, and to accommodate the September 1 deadline, districts use their counts the third Friday in August, approximately one week after the start of the school year. Districts report fall enrollment data to NDE, as of the last Friday in September, approximately six weeks after the September I report to the Learning Community. This six-week time lapse may account for differences between the number of applications approved and the number enrolled.
Some districts, in certain situations, use the Open Enrollment process for transfers from one school to another within the district, while other districts do not.4 This distinction is made in the tables that follow.
Table III.5: New Applications Received and Approved and Number Enrolled for the 2013-2014
School Year
SCHOOL DISTRICT
APPLICATIONS RECEIVED AND APPROVED FOR 2013-14
NON-RESIDENT RESIDENT
TOTAL
APPLICANTS APPLICANTS APPLICANTS
TOTAL APPROVED
PERCENT APPROVED
2013-14 NEW OPEN ENROLLMENT STUDENTS
ENROLLED NON-RESIDENT STUDENTS
ENROLLED RESIDENT STUDENTS
TOTAL ENROLLED
OPS
345
221
566
458
80.9%
190
3
193
Elkhorn
109
12
121
23
19.0%
11
1
12
DC West
68
0
68
68
100.0%
70
1
71
Millard
828
46
874
728
83.3%
527
99
626
Ralston
351
0
351
346
98.6%
276
0
276
Bennington
24
0
24
3
12.5%
1
3
4
Westside
651
0
651
505
77.6%
403
3
406
Bellevue
313
0
313
273
87.2%
216
1
217
Pap-LV
522
20
542
447
82.5%
310
5
315
Gretna
19
0
19
10
52.6%
8
0
8
Springfield
62
0
62
62
100.0%
40
0
40
Total
3,292
299
3,591
2,923
81.4%
2,052
116
2,168
The differences in the percentage of accepted applications across districts are caused by differences in the capacity to accept students from other districts at the grade level and in the school requested. Some districts are growing rapidly, and schools may already be crowded, while other districts have greater capacity to add students. Overall, 81.4% of the applications were approved. This percentage is similar to previous years.
4 Districts may give school transfer priority to resident students who request the transfer before February 15. 99
The two smallest school districts, DC West and Springfield Platteview, both approved all applications.
The most rapidly growing districts, Elkhorn, Bennington, and Gretna, understandably had the lowest approval rates.
The number of approved applications (2,923) is 755 more than the number enrolled. This is, in part, due to the fact that families can apply to multiple school districts; 2,923 represents the number of applications approved, not the number of students approved. Multiple school districts may have approved the same student's application. The difference between the number of applications and the number of students who actually enrolled can be attributed to a number of other factors as well, such as moving between the time of the approval and the start of the school year or deciding to stay in their resident school.
Table III.6 shows the number of Open Enrollment students who are enrolled in a school, which is not within their home districts' boundaries. It excludes those who transferred to a school within their resident district through Open Enrollment. It also shows the proportion of non-resident Open Enrollment students in each district's total enrollment. These data are also from the NDE Fall Membership.
Table III.6: Percent of Non-Resident Open Enrollment Students in School Districts' Total Enrollment in the 2013-2014 School Year
SCHOOL DISTRICT
2012-2013 NON- RESIDENT OPEN ENROLLMENT STUDENTS IN FALL MEMBERSHIP
2013-2014 NON- RESIDENT OPEN ENROLLMENT STUDENTS IN FALL MEMBERSHIP
2012-2013 PERCENT 2013-2014 PERCENT
OF NON-RESIDENT OF NON-RESIDENT
OPEN ENROLLED OPEN ENROLLED
STUDENTS IN TOTAL STUDENTS IN TOTAL
ENROLLMENT
ENROLLMENT
OPS Elkhorn DC West Millard Ralston Bennington Westside Bellevue Pap-LV Gretna Springfield Total
551 36 50 1,589 375 14 866 1,000 620 24 81 5,206
585 45 105 1,844 580 7 1,177 1,020 814 29 113 6,319
1.15% 0.57% 7.41% 7.01% 13.04% 0.90% 14.47% 10.34% 5.97% 0.72% 8.40% 4.63%
1.21% 0.66% 14.56% 8.07% 19.69% 0.40% 19.31% 10.52% 7.57% 0.82% 11.11% 5.51%
After three years of the Open Enrollment program, 6,319 students are classified as Open Enrollment and are enrolled in a school outside their resident district. However, the proportion of total enrollment that number represents (5.51%) is relatively small.
Millard has the largest number of non-resident, open-enrolled students, representing 8.07% of its total enrollment.
100
Ralston and Westside have the largest proportion of non-resident Open Enrollment students, with 19.69% and 19.31%, respectively.
Open Enrollment ­ Option Enrollment Comparisons As described at the beginning of Section III, Open Enrollment has been in existence in the Learning Community since 2010-2011. Prior to 2010, the State Option Enrollment system was used by all Nebraska school districts, including Learning Community districts, for the transfer of students across district boundaries. This year, for the first time, the annual report provides data comparing the two programs. The Nebraska Department of Education provided Option Enrollment information for Learning Community districts for the three school years prior to the implementation of Open Enrollment (2007-08, 2008-09 and 2009-10).
Table III.7 shows the number and percentage of Option Enrollment and Open Enrollment students by year for kindergarten, first, second and third grade. Only these grades are reported because the 2013-14 third grade cohort entered kindergarten in the fall of 2010, the first year of Open Enrollment. In 2010-2011, and in the next four years, all students who transferred from one Learning Community district to another did so under the Open Enrollment program, rather than Option Enrollment. In grades four through twelve, the new transfers are classified as Open Enrollment, but those grades also contain students who transferred among Learning Community districts prior to 2010, under Option Enrollment, and most of those students continue to be classified as such.
Table III.7: Number and Percent of Option Enrollment and Open Enrollment Students by Year
GRADE LEVEL
2007-08 OPTION ENROLLMENT
2008-09 OPTION ENROLLMENT
2009-10 OPTION ENROLLMENT
2010-11 OPTION AND OPEN ENROLLMENT
2011-12 OPTION AND OPEN ENROLLMENT
2012-13 OPTION AND OPEN ENROLLMENT
2013-14 OPTION AND OPEN ENROLLMENT
KG
445
441
494
528
622
605
595
1
416
476
490
611
595
661
609
2
471
468
523
590
636
663
676
3
451
494
542
607
649
625
635
Total
1,783
1,879
2,049
2,336
2,502
2,554
2,515
Percent of Total
Enrollment at
5.35%
5.50%
5.82%
6.50%
6.83%
7.00%
6.79%
Included Grades
In the column headed "2010-11 Option and Open Enrollment Students in Fall Membership" the only kindergarten students who are classified as Option Enrollment are those who transferred from a District that is not part of the Learning Community. The same is true of kindergarten and first grade in 2011-12; kindergarten, first and second grade in 2012-13 and kindergarten through grade three in 2013-14. The bottom row in the table shows the percentage of the total enrollment in these four grades that are classified as Option or Open Enrollment. For example 1,783 kindergarten through third grade 2007-08 Option Enrollment students represent 5.35% of the total Learning Community enrollment in those four grades that year. In the first year of Open Enrollment (2010-11), the 2,336 Option and Open Enrollment students represent 6.5% of the total Learning Community enrollment in those grades.
101
It is also important to know that all students in the Option Enrollment columns (the first three columns in Table III.7) are students who transferred to a school that is not in their resident district. As explained earlier, because some districts have used Open Enrollment for transfers among schools within their districts, the numbers in the last four columns (Option and Open Enrollment) include some students who are attending a school within their resident district's boundaries. This is not the case in the Option Enrollment columns. Not shown in the table, but important to understand, is the fact that only 16 of the 528 kindergarten students in the 2010-11 column are Option Enrollment students, meaning that 16 students transferred to a Learning Community school district from a district that was not within the Learning Community. In each of the other grades in the 2010-11 column, approximately 70% to 75% of the students are Option enrollment, students who transferred prior to the implementation of Open Enrollment. In the 2013-14 column, when all students in kindergarten through third grade who transferred among Learning Community districts are classified as Open Enrolled, and only those who transferred from districts outside the Learning Community are classified as Option, approximately 97% of the 2,515 transfer students are Open Enrollment students. The implementation of Open Enrollment in 2010-11 does not appear to have increased the incidence of student transfer across district boundaries. The proportion of the total enrollment represented by Option and Open Enrollment students ranges from 5.35% in 2007-08, when all were Option Enrollment, to 6.67% in 2013-2014 when most were Open Enrollment. Each year until 2013-14, the proportion Option Enrollment, and the combined proportion of Option and Open enrollment, increased slightly (less than 1%). In 2013-14 when almost all the students were classified as Open Enrollment the percentage went down slightly. Although the table shows a slightly larger proportion of student transfers after the implementation of Open Enrollment, the increase between 2009-10 and 2010-11 is only slightly larger than the increase in previous year. This difference could be attributed to the fact that Open Enrollment numbers include some students attending a school within their resident district. Table III.8 (page 18) compares the percentage of FRL-eligible Option and Open Enrollment students over the past seven years. In the first three years, there were only Option Enrollment students, while in 2010-2011 through 2013-2014 the Learning Community districts had both Option and Open Enrollment students. 102
Table III.8: Number and Percent of FRL-Eligible Option and Open Enrollment Students
Compared to the Total Learning Community Percentage of FRL
YEAR 2007-2008 2008-2009 2009-2010 2010-2011 2011-2012 2012-2013 2013-2014
Total Number of Option Enrollment 6,788 7,051 7,552 6,007 4,755 3,717 3,001
Number of Option Enrollment Qualifying for FRL 1,434 1,562 1,899 1,500 1,152 799 643
Percent of Option Enrollment Qualifying for FRL 21.13% 22.15% 25.15% 24.97% 24.23% 21.50% 21.43%
Total Number of Open Enrollment
Number of Open Enrollment Qualifying for FRL
Percent of Open Enrollment Qualifying for FRL
Learning Community FRL Percentage
N/A N/A N/A 2,563 4,334 5,769 6,535
N/A N/A N/A 1,076 1,843 2,345 2,724
N/A N/A N/A 41.98% 42.52% 40.65% 41.68%
36.46% 36.76% 40.08% 41.86% 43.48% 43.96% 44.47%
The data in the Table III.8 illustrate a rather dramatic difference in the percentages of Option Enrollment and Open Enrollment students who qualify for FRL.
In 2007-08 through 2009-10, when only the Option Enrollment program existed, the percentage of FRL-eligible Option Enrollment students ranged from 21.13% to 25.15%, approximately 15% lower than the total percentage in the Learning Community in those years.
As described earlier in this section, the percentage of FRL-eligible Open Enrollment students is similar to the Learning Community as a whole with differences each year ranging from less than 1% to approximately 3%.
Since the implementation of Open Enrollment, Option Enrollment students who qualify for FRL has remained relatively low. The lower percentage among Option Enrollment students in more recent years could be somewhat affected by the fact that, proportionately, more high students are included in those numbers and a lower percentage of high school students, than elementary and middle school, apply for FRL. However, this fact alone would likely account for only a small proportion of the difference.
In summary, it appears that the proportion of students who open-enroll is similar to the proportion that option-enrolled in the past, but there is a greater proportion of students who qualify for FRL among the Open Enrollment students than among Option Enrollment students. Further, the percentage of Open Enrollment students who qualify for FRL is similar to the percentage of the Learning Community districts as a whole, while the percentage of Option Enrollment students who qualify for FRL is considerably less than the Learning Community total, both in the past and currently.
Student Performance and Open Enrollment In prior years, this report to the Education Committee included a section in which we provided an analysis of the impact of the implementation of the Open Enrollment policy on student performance on the Nebraska State Assessments. After three years of analysis and reporting of these data, we are discontinuing the report section on student performance because it is clear
103
that no valid conclusions can be reached from the analysis, and it may be misleading to continue to report data, thereby leaving the impression that some findings or conclusions will be possible. Even though it is our observation that Learning Community school districts have faithfully implemented the Open Enrollment policy, valid conclusions are not possible. Neither a casual, nor even a correlational, relationship between the implementation of the Open Enrollment policy and student performance on Nebraska Assessments can be shown for the following reasons: An inherent assumption of the Open Enrollment policy might be that students in high poverty schools would benefit from moving to lower poverty schools, perhaps benefitting from the higher expectations or other supposed advantages of a low poverty school. There is no evidence of any significant movement of students from high poverty schools to low poverty schools or the reverse. Therefore it is impossible to conclude that such movement resulted in significant impact to overall student performance. There are too many intervening variable that cannot be controlled to offer any conclusions as to the academic benefits of Open Enrollment. The largest of these variables is the fact that those parents who seek open enrollment constitute a "voluntary sample" of parents who make the choice to undertake the Open Enrollment process. Perhaps if there were waiting lists with significant numbers of similarly, highly motivated parents, we could compare the results of the two groups to determine if the performance of the open enrolled students was better than that of the students who remained on a waiting list, but this is not the case. Therefore, we conclude that further analyses of these data would be meaningless and, possibly, misleading. The existent data provided in the State of the Schools Reports on the NDE website provide sufficient information and analyses about the performance of Learning Community students. 104

File: annual-evaluation-report-2013-2014.pdf
Title: Microsoft Word - COMPLETE_Sections I_II_III_2013-2014_FINAL
Author: perlewine
Published: Tue Dec 9 13:04:54 2014
Pages: 111
File size: 1.12 Mb


INTRODUCING CHILDREN TO, 3 pages, 0.13 Mb

Folk music, 7 pages, 0.14 Mb

Heart to Heart♥, 42 pages, 1.03 Mb
Copyright © 2018 doc.uments.com