The Role of Volunteers in the Assessment Process

Tags: learner, ASSESSMENT, ROLE, VOLUNTEERS, Ontario, volunteer tutor, program staff, APPENDIX, training plan, the volunteer, student progress, self-assessment, Adventures, Initial Assessment, Adult Literacy, volunteer, reading and writing, Ontario Literacy Coalition, Adult Basic Education Support, Ministry of Training, Colleges and Universities, Adult Learning Resource Centre, staff members, Adult Learning, National Centre for Vocational Education Research, volunteer tutors, Paula Berardinelli, volunteer staff, New York City Adult Literacy Initiative, adult learner, Laubach Literacy Ontario, communication skills, self-confidence, ongoing assessment, Community Literacy, Renfrew County Community Upgrading Program, Vancouver Community College, Ontario's Community Literacy Agencies, Community Literacy of Ontario, Kingston Literacy, Literacy Link Eastern Ontario, assessment information, Tutor Training
Content: The Role of Volunteers in the Assessment Process Vicki Trottier Community Literacy of Ontario December 2001
ACKNOWLEDGEMENTS
Researched and Written by: Vicki Trottier
Project Management Support: Jette Cosburn Co-Executive Director, CLO
Reference Group:
Audrey Anderson Joyce Bigelow Jean Doull Barbara Petepiece Joan Robinson Jeanne Tucker Johanna White Dan Woods
Editing:
Peggy Bridgland
Desktop Publishing:
The Right Type Kingston, Ontario
Published by:
Community Literacy of Ontario 508 ­ 80 Bradford St. Barrie ON L4N 6S7 705-733-2312 (voice) 705-733-6197 (fax) [email protected] http://www.nald.ca/volman.htm December 2001
Special thanks to the program staff and volunteers who participated in focus groups throughout the province. Thanks also to those who responded to questionnaires and shared their knowledge and best practices.
This project was made possible with funding from the Ontario Ministry of Training, Colleges and Universities and the National Literacy Secretariat (HRDC). CLO thanks both organizations for their ongoing support of community-based literacy initiatives.
Table of Contents Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 Some Terms and Definitions . . . . . . . . . . . . . . . . . . . 9 Why use Volunteers?. . . . . . . . . . . . . . . . . . . . . . . . . 13 Tutor Training . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 The Volunteer's Role in the Assessment Process . . 20 Focus on Ongoing Assessment . . . . . . . . . . . . . . . . 28 Self-Assessment . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38 Resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44 Appendix 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47 Appendix 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51 Appendix 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55 Appendix 4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65 Appendix 5 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73 Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105
5 Introduction Learning assessment may be the cornerstone to a quality literacy program. Effective assessment practices will determine how well literacy programs are helping people improve their literacy skills.1 IN 1997, THE Ministry of Education and Training2 began the process of program reform within funded Literacy and Basic Skills (LBS) agencies across the province of Ontario. This program reform was designed as a multi-year strategy with four components: · Learning Outcomes · assessment · articulation · accreditation3 Since its beginning in 1994, Community Literacy of Ontario (CLO) has supported its member programs in their work with volunteers. We believe that volunteers have an important and valuable role to play in the assessment process. In 2000, CLO was funded to carry out a project to research and report on the volunteer's role in assessment. The project was designed to: · research the current range of volunteer involvement in the assessment process; · research effective models for volunteer involvement in assessment provincially, nationally and internationally; · gather and share information on effective strategies and best practices;
6
THE ROLE OF VOLUNTEERS IN THE ASSESSMENT PROCESS
... validated the premise that volunteers do indeed play a significant role in assessment.
· describe the strengths and weaknesses of different models of volunteer involvement; · research how agencies support volunteer tutor involvement in the assessment process; · develop a resource manual of models, best practices, resources and strategies; and · develop a series of bulletins designed specifically to inform volunteer tutors about learning outcomes, demonstrations and assessment. Extensive research revealed little formal documentation, however focus group research and personal interviews provided a significant amount of anecdotal evidence that validated the premise that volunteers do indeed play a significant role in assessment. Research for the manual and tutor bulletins was also conducted with focus groups of both volunteers and staff in Toronto, Sarnia, North Bay, Timmins, Kirkland Lake and Simcoe County. A wide selection of print material was consulted as were a variety of websites from across the world (see bibliography). This manual is the result of the work done throughout the project. We hope it will provide community-based agencies with some tools they can use and also some insights into how other programs in Ontario, Canada and beyond involve their volunteers in literacy assessment. To help make the manual a useful resource, we have included a variety of sample forms in Appendix 4 and Appendix 5. Programs may photocopy and use these forms, although we do ask that you give credit to the program source where indicated. These forms are also included in the companion CD-ROM and may be edited and revised to suit individual agencies. In addition to this manual, CLO also produced a series of four tutor bulletins that were distributed to agencies throughout 2001. The bulletins have been reproduced in Appendix 3 of the manual and are also available on the CD-ROM. As with
INTRODUCTION
7
the forms, programs are free to reproduce and distribute the bulletins as they see fit. Assessment is more than just a good thing to do. It is a program responsibility, as outlined in the Core Quality Standards: "A quality literacy program evaluates learners' progress on an ongoing basis. The process involves the learners and contributes to their development."4 Ontario literacy programs are not alone in this view. The John Howard Society in British Columbia includes a similar statement in their documentation: "A quality adult literacy program uses a variety of flexible, learner-centred assessment procedures when learners enter the program, while they are being tutored and when they leave the program."5 The topic of assessment is discussed, described and debated in a myriad of resources. It is not within the scope of this document to go into detail about what assessment is or its purpose. This manual has been written to provide an overview of the role of the volunteer in the assessment process.
Volunteers are the heart and soul of community-based literacy programs. Most programs were founded through the efforts of volunteers and have continued to exist thanks to volunteer support. Volunteers are often the unsung heroes of these programs. Like anything else, however, community-based literacy programs in Ontario have gone through changes, and the role of the volunteer has also changed. As program reform has become a way of life for these programs, staff has also experienced a changed role. These changing roles can be summed up as follows: staff is responsible for the big picture -- overall programming and accountability. Volunteers are responsible for the day-to-day things that go on in a program -- tutoring the learner and supporting the staff. Assessment is not a stand-alone or isolated activity. It takes place within the context of learning. It isn't done to or by any one person. It involves everyone -- staff, volunteer and
Volunteers are the heart and soul of communitybased literacy programs. Most programs were founded through the efforts of volunteers and have continued to exist thanks to volunteer support. Volunteers are often the unsung heroes of these programs.
8
THE ROLE OF VOLUNTEERS IN THE ASSESSMENT PROCESS
Assessment is the common thread -- there are many ways to do it but it is always done.
learner. This manual will look at each of the roles, with the focus on the volunteer tutor. Ontario is not alone in relying on volunteers in the assessment process. An Internet search using Google (http://www.google.com) resulted in 3,380 matches for the phrase "Volunteer Literacy Tutor Job Description." A review of job descriptions from across the world showed that virtually all literacy programs ask their volunteers to be involved in assessment to some degree. There is no one, typical literacy program in the province of Ontario. When reading this manual, you may see scenarios that are similar or completely different from what you are used to. Assessment is the common thread -- there are many ways to do it but it is always done.
Learning involves knowledge and understanding. So assessment involves answering the question: "What do the learners already know and understand, and what do they need to know and understand?" Learning involves skills and strategies. So assessment involves answering the question: "What can the learners already do, and what do they need to be able to do?" Learning involves attitudes and values. So assessment involves answering the question: "What do the learners think and feel and believe about what they are learning, and about themselves as learners?"6
9 Some Terms and Definitions
Assessment The gathering and analysis of information about the abilities, needs, interests, learning styles, and achievements of learners.7 Literacy practitioners make judgments which assist them in setting and revising learning goals, informing learners of their progress and identifying the successful completion of goals as set out in the training plan. Assessment is a joint activity, conducted by literacy practitioners in co-operation with learners.8
Common Assessment The use by Literacy and Basic Skills (LBS) agencies of comparable assessment tools and approaches, based upon the common language of learning outcomes. It does not refer to the use of a standardized, common assessment tool.
This approach provides a method for comparing assessment results and establishing their portability among LBS agencies. Common assessment also facilitates communication about the achievements of learners with other stakeholders of the LBS program, such as training programs and employers.9 Demonstrations A formal method of ongoing and exit assessment. Demonstrations provide the learner with the opportunity to apply several skills to a real life task. Demonstration activities must be grounded in the needs, goals and priorities of learners.10 A demonstration is different from a learning activity because it includes an assessment component and it requires a learner to be able to transfer learning to another task. Demonstrations are usually developed by staff members who have experience and training in literacy assessment. In some programs, however, volunteer tutors help develop demonstrations because they are aware of the skills the learner
This approach provides a method for comparing assessment results and establishing their portability among LBS agencies.
10
THE ROLE OF VOLUNTEERS IN THE ASSESSMENT PROCESS
Exit assessments are not always possible because some learners leave without notice.
is working on. The learner should also participate in developing the demonstration. For example, she11 can choose how she would like to demonstrate her new learning. For more information about demonstrations, refer to On the Level, written by Lindsay Kennedy and published by CLO. Exit Assessment An assessment that is done to determine if a learner has achieved the short-term goal as set out on his training plan or when he leaves the program. In the first situation, the learner may continue in the program but work towards a new shortterm goal. Exit assessments are not always possible because some learners leave without notice. However, when they are possible, they allow the tutor and the learner to examine whether they: · have achieved certain competencies or qualifications; · have met the stated goal; and · were satisfied with the learning experience.12 Goal-Directed Assessment An initial assessment process that documents what potential learners in literacy programs have already achieved and still need to achieve in order to reach specific goals. It gives learners a clear picture of their learning needs as they relate to employment, training or personal objectives. It also gives learners, assessors and instructors a realistic strategy to meet these objectives.13 Initial Assessment A way to assess and then to evaluate, as carefully and comprehensively as possible, a learner's skills, knowledge, aptitudes and interests in relation to his individual goals. Initial assessment enables the practitioner to identify: · what skills, knowledge, aptitudes and abilities learners have or believe they have;
¤¤ ¤
SOME TERMS AND DEFINITIONS
11
· what learners want and need to know or do to function effectively at work, at home and in the community; · what goals learners have; · what learners require to reach their goals; and · what learners expect to gain from a program.14
Learner-Centred Assessment Assessment that is done with students, not to students. In contrast to traditional, standardized tests, learner-centred assessment engages students in an active process of finding out what they already know and what they would like to learn. Results are used to plan instruction that centers on student goals as well as to affirm students' pre-existing knowledge and steps made towards attaining goals. Assessment that is process-oriented. Assessment should not be a one-time, one-way test geared towards determining a grade level. It should be an ongoing process that seeks to find what is known now, what is being learned, and what needs to be learned in the future. Assessment that is empowering to both students and instructors. The results of assessment are not meant to be written on a form and then forgotten. In order to empower the participants, assessment should be useful, practical, and related to the life and goals of the learner. It should help the tutor know where to start and where to go. It should assist the learner in clarifying her goals and understanding her own strengths and weaknesses.15
Assessment that is done with students, not to students.
Learning Outcomes A language for measuring and documenting milestones in learners' achievements; not a curriculum.16
Ongoing Assessment Assessment that occurs at any point during the learner's program. It can be formal (demonstrations and/or tests) or informal (personal observation or notes in a file).
12
THE ROLE OF VOLUNTEERS IN THE ASSESSMENT PROCESS
Ongoing assessment allows both the tutor and the learner to examine: · how they are progressing towards an agreed upon goal; · how they might improve; and · how well the learning program fits the learner's learning needs.17 Portfolio A file that contains a representative selection of a student's work. It is meant to show progress over a period of time so that both learner and teacher can see what has been achieved.18 The development of a portfolio also provides an opportunity for learners to better understand their strengths. There are a number of resources available on the topic of portfolios. Short-Term Goal What the learner will accomplish in the amount of time he is willing to commit to the program.
13 Why use Volunteers?
Crucial to a successful program is the volunteer tutor's multifaceted role as assessor, instructor, advocate and motivator.19
THE VALUE OF USING VOLUNTEERS has long been recognized by community-based literacy programs. However, most of the evidence is anecdotal; not much has been done in terms of hard research, facts and figures. One of the few examples comes from an American volunteer literacy program called Time to Read. In 1988, they reported that "the key to effective assessment of learner progress [is the] close interaction between tutors and learners."20 Assessment is a measure of program accountability. Literacy practitioners must be properly trained to conduct formal assessments. Quality training, combined with on-the-job experience, gives practitioners the skills they need to conduct thorough, accurate assessments. The amount of time and training required to gain this level of expertise is significant and beyond the scope of what literacy programs can reasonably expect from volunteers. In fact, an Australian study discussed this issue of volunteers carrying out increasingly complex tasks and the differences between paid and unpaid practitioners. The authors of the study came to the conclusion that the role of volunteers in literacy programs has changed significantly and that programs should give careful consideration to the type of tasks and the level of responsibility they assign volunteers.21 If we acknowledge that there is a difference between the role of staff and volunteers in literacy programs and that the role of volunteers has changed and is continuing to evolve, why do we even recruit volunteers? After all, volunteer recruitment and management are time-consuming tasks.
... they reported that "the key to effective assessment of learner progress [is the] close interaction between tutors and learners."
14
THE ROLE OF VOLUNTEERS IN THE ASSESSMENT PROCESS
Volunteers are more than just unpaid labour. They provide an invaluable service in terms of establishing an ongoing relationship of trust with learners.
The primary reason that community-based literacy agencies have chosen to include volunteers in their programming is because volunteers can (and do) offer a different level of service to literacy learners. There is also a misconception that volunteers are simply unpaid labour, but monetary issues should not be the primary focus in a volunteer-based program. Instead, agencies need to be able to answer the question, "If we had more than enough funding to hire all the staff we need, would we still use volunteers? And what jobs would those volunteers do?" The answer to the first question should be a resounding "Yes!, we would indeed continue to use volunteers!" Volunteers are more than just unpaid labour. They provide an invaluable service in terms of establishing an ongoing relationship of trust with learners. Because they meet one-on-one (or sometimes in small groups) with learners, they get to know people in a way that simply isn't possible for program staff. By working so closely with learners, volunteers can provide program staff with a wealth of informal assessment information that staff can in turn use in their formal assessment tracking and reporting. The level of detail that a volunteer can provide about the learner's skills, successes and struggles is integral to the overall assessment picture. In addition, volunteers bring a high level of commitment and enthusiasm to literacy instruction, and they also bring a wealth of skills and background experience from their own jobs and lives that complement and enhance those of the program staff. Volunteer tutors can (and do) complement the work of staff through individualized instruction or assisting with small group work, thus freeing up program staff to devote themselves to issues of accountability: demonstration of learning, tracking progress, formal assessments, etc. When it comes to assessment, the ongoing feedback that a volunteer can provide is worth more than its weight in gold. This ongoing feedback is a key component in the team effort that goes into conducting thorough and accurate assessments. The chapter entitled "The Volunteer's Role in the Assessment
15 WHY USE VOLUNTEERS?
Process" provides a more in-depth look at the scope of the volunteer tutor's job.
Of course, volunteers can't be expected to simply sign up and start tutoring. They must be provided with initial and ongoing training (see the chapter "Tutor Training"). They must be given clear guidelines about their role and also about the staff 's role. They must also know what they can expect in terms of support, personal safety and other considerations. Appendix 4 includes some sample job descriptions from a variety of literacy agencies. Once the prospective volunteer has been recruited, screened, trained and assigned a learner, he becomes an integral and valuable part of the literacy agency -- far more than simply unpaid labour. Working together with the learner to deliver the best possible programming, the unpaid and paid practitioners form a winning combination.
They must be given clear guidelines about their role and also about the staff's role. They must also know what they can expect in terms of support, personal safety and other considerations.
¤ ¤¤ ¤ 16 Tutor Training
Tutors are generally asked to observe and report on how the learner goes about learning a new skill or how he applies that new skill during a tutoring session.
THERE IS CURRENTLY no "standard" tutor training package in Ontario's community-based literacy agencies.22 Tutor training may be offered on a one-on-one basis as the need arises or the agency may offer group sessions at regular intervals. Some programs hold annual training updates while others host more informal tutor get-togethers as a form of ongoing training. Virtually all agencies include some training on assessment, usually during the initial training. Assessment training is not always broken out as a separate unit or module in tutor training. Discussion might center around providing feedback to the learner and program staff, or there might have been some suggestions about reviewing previous lessons. Tutors are generally asked to observe and report on how the learner goes about learning a new skill or how he applies that new skill during a tutoring session. This is all ongoing assessment and is one of the tutor's key roles. Here are some samples from job descriptions and policies from programs in Ontario and beyond: People, Words & Change is a literacy program in Ottawa, Ontario. Its website23 lists a number of volunteer tutor qualifications including a "willingness to keep records and give feedback on the learner's progress." The Handbook for Literacy Tutors suggests keeping records/notes and reporting to staff about activities and progress.24 The Renfrew County Community Upgrading Program expects its tutors to "keep an ongoing record of the work done and progress made by the student."25 The Literacy Council of South Temiskaming in Northern Ontario asks its tutors to "keep written records of the
student's progress and report regularly to co-ordinator with evaluation and recommendations."26 The Midland Area Reading Council includes the following responsibilities in its Tutor Job Description: "Set goals with the student and frequently evaluate progress towards them" and "Complete semi-annual written progress reports."27 Kingston Literacy requires its tutors to "complete weekly tutor reports" and "discuss the student's progress with the Tutor Coordinator."28 This program has developed an assessment kit to provide a "user-friendly, manageable method to track learning."29 Many other programs across the province have incorporated or adapted the ongoing assessment forms and methods in the kit for their tutors to use. The Literacy Council of Norristown in Pennsylvania uses its online tutor report form to find out about "significant achievements."30 Setting the Compass, a resource produced The Association of Literacy Coordinators of Alberta, includes the following responsibilities in a Volunteer Adult Literacy Tutor Job Description: "evaluating lessons and providing literacy coordinator with accurate documentation (including student progress)."31 A variety of American literacy programs such as The Literacy Project in Massachusetts, Project Read in California, the Literacy Council of Reading-Berks in Pennsylvania and the Kent County Literacy Council in Oregon all include a reference to reporting student progress in their tutor job descriptions.32 In 2000, Community Literacy of Ontario produced a report called Skills for the Future as part of the Practitioner Training Strategy project. The report included a skills list for literacy tutors, and observation skills were included as a high priority.33 Some sample volunteer tutor job descriptions are included in Appendix 4. Although each job description is unique, there
¤¤ 17 TUTOR TRAINING
18
THE ROLE OF VOLUNTEERS IN THE ASSESSMENT PROCESS
An Internet search of the term "literacy tutor" or "literacy volunteer" using any common search site (Google, AltaVista, WebCrawler, Copernic, etc.) will result in literally thousands of hits.
are many similarities amongst them. For example, a sense of humour is often included as a desired qualification for volunteers. Respect for the learner and a need to maintain confidentiality are also common elements. They all include some form of reporting back to program staff about the learner's progress. An Internet search of the term "literacy tutor" or "literacy volunteer" using any common search site (Google, AltaVista, WebCrawler, Copernic, etc.) will result in literally thousands of hits. Many literacy programs in both Canada and the United States have websites, often with a special section aimed at the volunteer tutor. If you have the time, it is worth surfing these sites. They often contain some "gems" such as suggestions for lesson planning or other tutoring tips. Here are a few sites that contain useful information: · Adult Education Resources on the Web at http://www.research.umbc.edu/~ira/AEresweb.html · Queensland Council for Adult Literacy (scroll down the page to link to Tutor Tips, discussion forums and archived tip sheets) at http://cwpp.slq.qld.gov.au/qcal · Panhandle Adult Literacy Centre, Florida (click on the volunteer resources link) at http://www.tfn.net/palc/ about.htm · Carleton University's "Tips at Your Fingertips" at http://www.cyberus.ca/~llessard/Sample.html · Lake County Literacy Program at http://literacy.kent.edu/~illinois/illteach1.htm (these are downloadable pdf files) · North Central Regional Educational Laboratory shares a variety of literacy assessment techniques at http://www.ncrel.org/sdrs/areas/issues/content/ cntareas/reading/li7lk29.htm · or just type "Tips for Adult Literacy Tutors" into a search engine, and you will be able to explore almost 5,000 links!
19 TUTOR TRAINING
When a literacy program is providing training for volunteer tutors, staff has to make a choice about how much information to include. There are many, many resources available in the literacy field, and it is impossible to include them all in initial training. Usually, volunteers are introduced to the basics and given an overview of the literacy field along with some instructional techniques. It is also important for tutors to understand what the training plan is and how to use it when planning tutoring sessions. They also need to be familiar with the five levels of literacy learning used in Ontario. As tutors become more experienced and more comfortable in their role, they might be interested in knowing more about learning outcomes, learning levels or literacy in general. The "Resources" chapter at the end of the manual lists some good starting points for volunteers who would like to learn more.
Literacy program staff should share the learner's training plan with the volunteer so that everyone is aware of the results of the initial assessment, the learner's goals and the overall plan to help the learner reach those goals. As the tutor and learner work together and get to know each other, they may discover that the entry-level skills reported on the training plan don't accurately reflect the learner's skills. When this happens, the volunteer tutor should contact a staff member to discuss her observations. It's important for everyone concerned -- learner, volunteer and staff member -- to understand that initial assessments are not perfect. They are simply a snapshot of the skills and abilities the learner demonstrated at one particular point in time. Often the learner is nervous during the interview or hasn't done this type of work for a while. Remember that the tutor and learner will develop a special relationship as they work together. Between them, they will more clearly identify skills and abilities. When tutors and learners share this knowledge with staff members, it encourages teamwork and allows programs to provide a positive and rewarding learning experience.
Literacy program staff should share the learner's training plan with the volunteer so that everyone is aware of the results of the initial assessment, the learner's goals and the overall plan to help the learner reach those goals.
20 The Volunteer's Role in the Assessment Process Who are the volunteers and what is their role? [Volunteers are people] who, of their own free will, choose to undertake specified work within an organization without payment, for the benefit of the community, themselves and the organization. This definition recognizes that volunteers participate: of their own free will without financial reward by undertaking clearly established and ongoing tasks in the delivery of agencies' programmed services in order to satisfy various personal and community needs. Volunteers do not substitute for paid employees: the role of the volunteer complements that of paid staff members, both in the management of programs and by enhancing the quality of service provision. The roles and responsibilities of paid staff and of volunteer staff must be clearly defined. This avoids confusion and facilitates understanding and support within the volunteer and paid staff team.34
THE VOLUNTEER'S ROLE IN THE ASSESSMENT PROCESS
21
THE ASSESSMENT PROCESS is just that -- a process. There are steps to follow and different roles for people to take on. Complete and thorough literacy assessment is truly a team effort. It involves staff, volunteers and learners, and each person has an important role to play in the process. In a nutshell, staff are responsible for conducting formal assessment (initial, demonstrations, exit), and volunteers play a vital role in gathering informal (ongoing) assessment information. Staff look at the big picture (training plan) and volunteers focus on the day-to-day activities (lesson plans). Learners are involved and consulted throughout the process. Sometimes, a volunteer tutor might think he is "just" a volunteer. But volunteers are an essential component in the assessment process! Because they meet with learners on a regular basis, they have the opportunity to get to know them better than a staff member can. As they work together, a relationship of mutual trust develops, and the volunteer will be able to identify areas of strength and weakness. The tutor can then pass on information about the learner's skills and abilities that will help program staff document progress.
Volunteers can't do it all, however -- nor should they. Staff, volunteers and learners must work together to ensure that learners are working towards their stated goals and that their needs are being met. It really is about teamwork and like any good team, everyone has a job to do and no one job or individual is more important than the other. It is everyone's skills, experiences and abilities put together that creates that "winning combination". It is important for everyone to clearly understand the role each person plays and how volunteers and staff can and should support each other. Because assessment is ultimately a program responsibility, it is the job of the staff to ensure that formal assessments are thorough and accurate and that the results are recorded in the learner's file. This type of assessment activity is a highly skilled one and requires significant training and experience.35 The volunteer's role is to provide support and regular updates to the staff. Staff members should conduct initial and exit
Volunteers can't do it all, however -- nor should they.
22
THE ROLE OF VOLUNTEERS IN THE ASSESSMENT PROCESS
When a tutor is helping a learner with a new skill, she will notice that some skills are easier for him to grasp than others.
assessments and should also administer demonstration activities. Volunteer tutors should help learners work towards demonstrating their learning and keep staff advised of ongoing progress. Instruction and ongoing assessment go hand in hand. When a tutor is helping a learner with a new skill, she will notice that some skills are easier for him to grasp than others. The tutor will discover clues to the learner's learning style and preferences (e.g., does he repeat things out loud, does he count on his fingers). It will soon become apparent whether he likes to work in a quiet room or in a room with other learners. The tutor should use this information to help develop lesson plans. For example, it will help determine whether more or less time should be spent on reviewing the previous session's work or if instructions are better given verbally or in writing. All of this is assessment information and the gathering and reporting of this information on an ongoing basis is the key component of the volunteer's role in the assessment process. The job of the volunteer is to report examples of successes and difficulties to a staff member. Most programs require their volunteers to submit regular progress reports to ensure that this gets done. Some sample reports are included in Appendix 5. But volunteers don't need to wait until it's time to submit a report to provide feedback to the staff -- when the tutor notices that the learner is using a new skill or has made a significant accomplishment, she should give a staff member a phone call or send a quick e-mail. Staff appreciate getting this information because it helps them more accurately track progress. The learner and the volunteer tutor will be working together to meet the short-term goal described in the training plan. The plan will also list some ways that the learner can demonstrate his new skills. The volunteer should let a staff member know when the learner is ready to perform a demonstration. Sometimes, the tutor can help design the activity that is part of the demonstration, but the ultimate responsibility for administering and assessing demonstrations of skill rests with program staff.
THE VOLUNTEER'S ROLE IN THE ASSESSMENT PROCESS
23
Because volunteers (and learners!) play key roles in the assessment process, they should have access to information about and input into any documentation used for the purposes of assessment. The learner needs this information because she needs to know how she is doing and what she still needs to work on. The tutor needs the information to help him design learning activities and to know what skills gaps still exist.
Staff should clearly define lines of communication. Many literacy programs in Ontario identify one staff member as the main volunteer contact. Tutors should feel comfortable speaking to that person about any concerns they might have. The designated staff member should also have some training in volunteer management. The value of well-thought out policies and procedures, job descriptions and reporting mechanisms cannot be overstated.36 Volunteers also need to be clear about just what they are reporting on and in what level of detail. Program staff need to identify whether they want reports submitted after every session, after every four sessions or less frequently. (When establishing the reporting periods, staff should take into consideration that they are responsible for tracking learner progress and more frequent reports can help identify areas of concern before problems arise.) Staff should also require that volunteers report back not so much on the isolated skills the learner is using, but rather on HOW the learner is using her new skills. Anecdotes about a family member noticing an improvement or an account of something the learner did that she couldn't do before provide valuable information to the staff and help them determine when a learner is ready to perform a demonstration. Appendix 5 includes some sample report forms that are used in Ontario community-based literacy agencies. Everyone should also remember that communication is not a one-way street! While it is important for the volunteer and learner to submit regular progress reports, it is equally important for program staff to check in every so often just to see how things are going. In many programs, staff members
Staff should clearly define lines of communication. Many literacy programs in Ontario identify one staff member as the main volunteer contact.
24
THE ROLE OF VOLUNTEERS IN THE ASSESSMENT PROCESS
will arrange to visit a tutoring session every few months. This allows that person not only to observe first-hand how the match is working but also to establish a more personal relationship with both the volunteer and the learner. Both staff and volunteers need to be clear on just who is doing what in the assessment process. Typically, a staff member conducts the initial assessment and, in consultation with the learner, develops the training plan. The training plan will include the learner's short-term goal and expectations for exit demonstrations. It might also include suggested resources to help the tutor decide what materials to choose when developing lesson plans. The staff member then decides on a good tutor match for the learner and arranges for the pair to meet. Sometimes, the staff member will attend the first few tutoring sessions.
The volunteer becomes the main link between the learner and program staff. This doesn't mean, however, that the staff member no longer has an interest in the learner!
At this point, the staff 's active role in the learner's training diminishes. The volunteer becomes the main link between the learner and program staff. This doesn't mean, however, that the staff member no longer has an interest in the learner! Although the staff person will not be taking an active role in the day-to-day literacy training, he must be kept informed about progress and be advised immediately of any concerns. Sometimes volunteers think that they are bothering or disturbing staff but rest assured -- staff really do want to know! When a volunteer is first matched with a learner, she should review the training plan with the learner and be sure that they are both familiar with this important document. It contains the blueprint they will use to plan their time together. It is important that the volunteer has an understanding of what the training plan is used for and also a working knowledge of the five literacy levels used in Ontario. Without this knowledge, it will be virtually impossible to develop lesson plans that work towards achieving the goals set out in the training plan.37 As part of ongoing instruction, the volunteer and learner will review and practice skills and make changes to the lesson plans as appropriate. Perhaps they will need to add more time to review some areas or to practice skills. They will also work
THE VOLUNTEER'S ROLE IN THE ASSESSMENT PROCESS
25
together to complete progress reports to submit to program staff. At some point in time, the learner and the tutor will feel confident that the learner is ready to demonstrate his skills. They will let the staff member know, and he will arrange a demonstration. Although the volunteer will help the learner prepare for the demonstration, it should be administered and formally assessed by a staff member.
Based on the results of the demonstration, the staff member will work with the volunteer and the learner to determine the next steps. This could mean that the learner needs to move on to more difficult work or that he has completed his goal and is ready to move on to the next step. Sometimes the volunteer tutor might feel that she is filling out form after form that no one ever reads. Rest assured that the ongoing assessment information tutors provide is valuable feedback that program staff use to monitor and track progress and to plan next steps. The tutor can also use the information to help in lesson planning and to help the learner reflect on her progress (see the chapter "Self-Assessment"). Unfortunately, the frequent filling out of progress report forms can quickly become a rote exercise. To avoid this, the tutor should encourage the learner to reflect carefully on not only what he has learned but on how he is using his new skills. Reflection is an integral part of internalizing learning, which in turn is the true measure of learning.
Based on the results of the demonstration, the staff member will work with the volunteer and the learner to determine the next steps.
Instructors should encourage students to reflect on their learning and explore how and why work shows progress or the need for additional practice. Reflection encourages learners ...demanding that they evaluate their work, think of possible solutions to problems and become responsible for progress toward their goals.38
26
THE ROLE OF VOLUNTEERS IN THE ASSESSMENT PROCESS
But when does the volunteer do all of this assessment work? Many tutors find that the time they spend with their learner goes by too quickly as it is. Because assessment is part of instruction, it should be included in regular meeting time.39 For example, completing the reports together is good practice for filling in forms and offers the opportunity to use new spelling words and other skills. Taking the time to reflect on learning and identifying personal strengths and weaknesses are also part of the self-management/self-direction domain. The next chapter will provide a more in-depth look at ways to incorporate ongoing assessment into learning activities and tutoring sessions.
THE VOLUNTEER'S ROLE IN THE ASSESSMENT PROCESS
27
Flow of the Assessment Process
The following description demonstrates how the assessment process can flow smoothly and also highlights the role of the staff and the volunteer:
When new volunteers enroll with the program, they are worried about knowing "where to start". They are relieved that the Program Coordinator will do an initial assessment and then provide them with information!
Student
After being matched, the tutor and learner will do regular "check-ups" and complete progress reports. They discuss the
Volunteer Tutor Where Am I?
Program Coordinator
reports with the Coordinator and together they review the training plan and decide on what material to use next. When
it's time for the learner to
move on, everyone meets
How to Start?
Initial Assesment
once again to discuss how things went and how learning and progress
compared to what was set
Match Tutor/Student
Make Training Plan
out in the training plan. The learners like to see this "proof" of learning.
Everyone at the Literacy
Council seems happy with this
Check-Ups/Reports
process. By conducting the initial assessment, the Coordinator gets to know the learner. That helps her make a
good match with a volunteer. The volunteer
then establishes a strong working relationship with
the learner and can provide valuable ongoing assessment
information. Both volunteers and learners like having
the training plan to refer to and like to be able to identify indicators of progress.
When it's time for the learner to move on, everyone gets together to take a look at how closely the learner's progress and learning matched what was set out in the training plan. They find this gives a nice sense of closure.40
28 Focus on Ongoing Assessment Learners need and want to be able to see how successful they are. ... Working from his training plan and lesson plans keeps your focus on his goals and makes it easier to do assessments. Assessment should be part of learning activities. Use it to see what teaching methods are successful and to modify those that are not as effective.41 Ongoing assessment has to do with [learning] activity that occurs continuously. [It] has less to do with written reports and far more to do with the interactive, dynamic roles of both teachers and learners. It has to do with responding to learners' questions every day and with actively noting the kinds of questions learners ask, the ways in which learners respond to print and oral communications, the kinds of mistakes they make, the ways in which they go about correcting their own mistakes, and the ways in which [others] might correct them. This kind of ongoing observation and assessment is inseparable from good teaching practice.42
FOCUS ON ONGOING ASSESSMENT
29
INSTRUCTION AND INFORMAL, ongoing assessment are part of the learning cycle. As Isserlis stated, they are continuous, inseparable activities. They also form the basis of the volunteer's role in the assessment process (see the chapter "Role of the Volunteer"). This process is constant and cyclical -- both the learner and tutor will reflect on the work they are doing and then make changes to the lesson plan based on their observations.43
This combination of teaching and observing starts before the very first tutoring session. Program staff will need to share the results of the initial assessment with the volunteer tutor so that he will have an idea of where to start. After all, the tutor will need to be aware of the learner's objectives before he can assess anything! In ongoing assessment, the focus will be on monitoring and reporting on: · Knowledge and understanding; · Skills and strategies; and · Attitudes and values. The training plan can help the volunteer and the learner identify specifically what they will be working on in these three areas. Ideally, they should have easy access at all times to a copy of the training plan. Whatever resources and topics the staff and/or tutor choose to include in the first lesson, they will be focused on the learner's expressed needs (as identified during the initial assessment phase and then recorded on the training plan). The actual instruction that takes place in that first lesson will reveal further information that should be used to develop and/or refine plans for future lessons.44 Ongoing assessment involves more than just the observation of learning, however. It is also helpful if the volunteer tutor is aware of other factors that might affect the learner's progress. For example, if the tutor notices that the learner is having difficulty reading small print, a referral for an eye examination might be appropriate. Or, if it necessary to continually repeat
Program staff will need to share the results of the initial assessment with the volunteer tutor so that he will have an idea of where to start.
30
THE ROLE OF VOLUNTEERS IN THE ASSESSMENT PROCESS
The learner might also mention difficulties with other, non-literacy issues. For example, she might mention a problem she is having with a childcare provider or with another agency.
instructions or to speak in a louder voice, perhaps the learner could benefit from a hearing assessment. The learner might also mention difficulties with other, nonliteracy issues. For example, she might mention a problem she is having with a childcare provider or with another agency. Although this type of significant issue can have an impact on learning, it is not the role of a volunteer tutor to solve these issues for the learner. However, the tutor can provide support as the learner works through these issues; it is also possible to incorporate the situation into a lesson; i.e. writing a letter, completing forms, calculating budgets. Each individual tutor will need to determine his own comfort level in dealing with this type of situation. When in doubt, or if it becomes apparent that the learner needs more support than the volunteer can give, a staff member should be advised. Bringing Learning and Assessment Together The Ottawa-Carleton Coalition for Literacy's Tutor Handbook provides the following suggestions to help volunteer tutors easily incorporate assessment activities into all aspects of lesson planning and tutoring.45 Things to Do Before the Lesson · Focus and reflect on ideas for working with the learner, based on his goals. · Break goals into manageable chunks; build success into every lesson. · Decide what the learner will achieve in each lesson (outcomes). · Decide what activities will be presented. · Plan a variety of activities to include reading and writing in each session. · Build on skills the learner already has. · Decide how to present activities and think about the time needed. · Collect or prepare the materials you will need.
FOCUS ON ONGOING ASSESSMENT
31
Things To Do During the Lesson · Allow time for him to "warm up." · Review concepts that were new or difficult from the last lesson. · Explain what will happen in this lesson and why. · Show how the "parts" relate to the "whole." · Relate new materials to past experience. · Don't introduce too many new things at once -- focus. · Let him set the pace. · Allow enough time for practice. · Have lots of opportunities for questions. · Watch for signs of fatigue or discouragement. · Allow time for breaks, especially for people with physical disabilities. · Be flexible enough to put prepared lessons aside if the learner wants to work on a particular problem. · Encourage the learner to try things alone (e.g. shopping trip, library). · Share successes. · Ask for feedback. · Remember, a little praise goes a long way. · Make it fun! · Have fun yourself! Things To Do After the Lesson · Keep a record of goals and lesson plans. · Make notes on what went well in the lesson and note any difficulties. · Review what to do next time. · Encourage the learner to keep records too, such as a journal of "reflections" about the lessons -- he can look back and see the progress he has made. · Ask yourself: Did we achieve the outcomes that we were working on?
32
THE ROLE OF VOLUNTEERS IN THE ASSESSMENT PROCESS
Was the lesson interesting? Were we relaxed? If not, why not? · In a classroom setting, report to the instructor on activities and progress. · Evaluate your performance. Were you an effective tutor? Keep a log to help you review your own progress.
It is especially important to take note of how the learner is using her new skills in both her learning and everyday life.
More Tips & Tools Volunteer tutors can use some or all of the following strategies to incorporate informal, ongoing assessment into learning activities. Watch and listen Can a preferred learning style (visual, auditory, tactile) be identified? What does the learner do to help herself learn (repeats things out loud, rereads the same passage, asks questions)? Are some skills easier than others for her to learn? Tutors should keep an eye out for ways the learner is using her new skills in everyday situations. Perhaps she will bring in a list she wrote at home or she will read a notice on the bulletin board. She might also talk about things she has done or things her family has noticed. Talk about how the learner learns Who better to ask than the learner himself? Ask him what makes learning easier or harder, how he likes to learn, what strategies he uses. Talk about how he learned things in the past, what worked and what didn't. Make notes Both the learner and the tutor can keep "learning diaries" that describe the lessons they are working on and the progress the learner is making. It is especially important to take note of how the learner is using her new skills in both her learning and everyday life. If she is just learning to write, she can dictate to the tutor. Note taking also makes an excellent writing activity for the
FOCUS ON ONGOING ASSESSMENT
33
beginning writer. These notes will come in handy for filling in reports or for ongoing assessment meetings with program staff. Written records can help keep track of resources used and activities completed. Diaries can also be used to list things like spelling words or new vocabulary.
Complete those reports regularly! Use ongoing notes and diary entries to help complete regular progress reports. As mentioned in the previous chapter, these reports are valuable components of the overall assessment process. Staff will use them to help decide when it's time for the learner to demonstrate his learning more formally. The tutor and the learner will use them to help with ongoing lesson planning. They also provide a good way to look back and easily identify progress made over time.
Go beyond reading, writing and numeracy skills "Soft skills" such as improved attendance or enhanced selfconfidence are just as important as grammar and math. These Self-Management/Self-Direction skills are an essential part of learning and should be given equal value and consideration. Use ROPES The ROPES method is an excellent way to develop lesson plans. ROPES stands for: Review, Overview, Present, Exercise, Summary. Using this method brings in the elements of good teaching and informal, ongoing assessment and also encourages the learner to participate through self-assessment, particularly in the review and summary phases.
"Soft skills" such as improved attendance or enhanced self-confidence are just as important as grammar and math.
34
THE ROLE OF VOLUNTEERS IN THE ASSESSMENT PROCESS
ROPES46 Review: Tutors review with the learner what he already knows. This can provide useful information about the learner's experiences, attitude and competence and reinforce the importance of prior knowledge and/or experience.
Overview: An overview should describe the major steps the learner will take to reach the goal of the lesson and explore how the learner may be able to use what she learns.
Presentation: The presentation phase includes both the major and minor steps that make up the content, the support knowledge, and the "tell," "show" and "do" steps. At this stage, however, the tutor leads or demonstrates the "do" step.
Exercise: The exercise gets the learner doing. For an exercise to be effective, the learner should first be asked to practice the skill under supervision. Later, the learner should be able to do the exercise without guidance.
Summary: In the summary, the learner describes and demonstrates what she has learned and explains how she plans to use the new skill(s). The summary can be a simple matter of verbal or written questions and answers, or it can involve the learner in demonstrating her competence -- a worthwhile exercise, as most people take pleasure in demonstrating a new skill.
FOCUS ON ONGOING ASSESSMENT
35
Other standard elements that should be included in the lesson plan are: · details of any pre-assessment procedures to be used · instructional techniques to be used · time allotted for each activity · facilities, media, supplies, tools, equipment, etc. required · details of any references and textbooks required · details of any assignments · details of any post-assessment procedures to be used.
Be clear about what you are assessing The tutor and learner should decide ahead of time what they will accomplish in each session. Perhaps they will begin learning a new skill or they will begin to incorporate a set of skills into a real-life situation. Success can only be measured if you know what you are measuring -- setting regular Learning Objectives makes ongoing assessment easier. Of course, the learning objectives must relate to the shortterm goal set out in the training plan. Referring back to the training plan and thinking about how learning objectives relate to it is another way of keeping learning on track.
Use a skills checklist This is an easy and quick way to keep track of the skills a learner has gained. Sometimes checklists are called "inventories" or "progress charts". This type of list can provide a quick snapshot at any given time and also offers a good visual comparison from one point in time to another.
Keep a portfolio Many literacy programs in Ontario encourage learners to keep portfolios of their work. Portfolios can help the learner identify the skills he has learned in a very visible, concrete fashion. Sometimes, however, the learner needs help in deciding what to include in his portfolio. The volunteer tutor can help him determine just which activities and assignments provide good examples of his learning so that the portfolio becomes more than just a collection of work.
Many literacy programs in Ontario encourage learners to keep portfolios of their work.
36
THE ROLE OF VOLUNTEERS IN THE ASSESSMENT PROCESS
Ongoing assessment is the job of both the tutor and the learner. It is important that the learner be able to recognize her own progress and become a full partner in determining her progress.
Keep in touch It is important for the tutor and learner to keep a staff member up-to-date on the learner's progress. When the learner-tutor match is arranged, a regular system of reporting should also be established. It is important for the volunteer and the staff to keep in regular contact so that the learner's progress can be properly documented. Encourage self-assessment Ongoing assessment is the job of both the tutor and the learner. It is important that the learner be able to recognize her own progress and become a full partner in determining her progress. The tutor and the learner should work together to identify the skills learned and how the learner is using those new skills. The next chapter will look at self-assessment in more detail. Encourage reflection How does what the learner has learned relate to his goals? How do learning activities relate to the training plan? Which activities provide a good indicator of a skill learned? Reflecting on learning helps internalize learning. Be flexible Sometimes, things won't go according to plan. If it is taking longer for a learner to grasp a new skill, the volunteer will need to adjust her lesson plans (provide extra practice, try a new teaching strategy). Teaching and learning are not exact sciences, there is no one way that works 100% of the time, and no two learners learn at exactly the same pace or in the same way. The volunteer must use her observations about the learner's skills and learning preferences to help her develop and adapt lesson plans as necessary. Provide feedback to the learner The tutor should provide positive feedback on the learner's work. Positive, constructive feedback can help a learner identify areas of difficulty that she can work on. It is important, however, to direct any criticism at the work, not at the person. Providing this type of feedback helps the student
FOCUS ON ONGOING ASSESSMENT
37
learn to assess her own work. Both of these skills (accepting feedback and self-assessment) are part of the Self-Management/Self-Direction domain and are an essential part of learning. It is also important that the learner understand that she is also criticizing her own work and not the individual tutor or the literacy program.
38 Self-Assessment Self-assessment encourages [the learner] to look at his own learning styles and to see what is effective. It also helps him to identify strategies that work. Assessment helps him to develop more realistic goals.47 TOGETHER, LEARNERS, VOLUNTEERS AND STAFF form the assessment team in each literacy agency. As we have discussed throughout this manual, each person has a role to play in the assessment process. The staff 's role is to conduct formal assessments (initial, ongoing and exit) and to document those assessment results in the learner's file. This is more than just a way of keeping track, it is a measure of program accountability. Program staff is also responsible for providing volunteers with the support they need in their role and for establishing systems and mechanisms that ensure accurate and ongoing reporting of assessment information. The volunteer tutor's role is to provide informal, ongoing assessment as part of literacy instruction. The tutor's observations and input must be provided to program staff in a way that supports the staff 's need for accurate, timely information. The learner's role is to provide input to both the volunteer and the staff by learning how to assess his own progress, skills and abilities. Put simply, self-assessment is a way for the learner to answer the question, "How do I know when I'm learning?"48 It is important, however, to separate two distinct areas of self-assessment: assessing specific skills-related learning and
39 SELF-ASSESSMENT
assessing the "big picture" of learning, which relates more to the Self-Management/Self-Direction domain. It is possible, even likely, that many learners may not look favourably upon assessment at first. They may have had negative experiences in school and will view any form of assessment as a way of highlighting what they don't know. They will probably think about assessment in terms of formal tests and exams, but good assessment goes far beyond that and helps the learner identify strengths and knowledge.
Both program staff and volunteer tutors can help the learner understand how proper assessment can ensure that he is in the right program and learning at the appropriate level. Through self-assessment, the learner can become a more active partner in the overall assessment process; he will discover more about the skills he needs to reach his goals and how to be a full participant in planning his learning outcomes. The Way to Go! Series of bulletins published by the Ontario Literacy Coalition in 2001 provides a good start to discussing assessment with learners.
Both program staff and volunteer tutors can help the learner understand how proper assessment can ensure that he is in the right program and learning at the appropriate level.
The more a learner has to wait and get approval...from someone else -- be it...a teacher or any other authority figure...the less the learner is likely to become an independent, self-regulating, self-disciplined, self-critical, self-loving, self-motivated, self-developing, empowered...member of our society.49 Self-assessment is a skill in itself. The learner probably won't know just what you are asking him to do and will need to practice. Remember to ask open-ended questions that encourage him to think about what, how and why he has learned. Simply asking "Did you learn anything today?" is not enough. Try questions such as "What did you learn today?", "What made it easy (or hard) to learn" and "How have you used your new skills in the past week?" Asking open-ended
40
THE ROLE OF VOLUNTEERS IN THE ASSESSMENT PROCESS
One thing to be sure to talk about is the importance of being honest but not brutal when self-assessing.
questions helps the learner re-create the learning, thus making it easier to identify just what he learned and how he did it.50 One way to begin to get the learner involved in self-assessment is to encourage her to take a role in completing the progress reports that are submitted to program staff. Instead of the tutor filling out the forms on her own, she should go over the report with the learner and include her comments and observations. If the learner is comfortable with writing, this can be also be a learning activity. If the learner isn't able to write out her own comments, she can dictate them to the tutor -- this can also be a learning activity! Sometimes, the learner might know he has made progress but he won't be able to identify a specific skill he has mastered. It is less important that he be able to identify each skill than it is for him to understand how he is using his learning, but if he would like to know more about skill sets, levels and learning outcomes, The Level Descriptions Manual is a good source of information. A staff member should also be consulted if the tutor and learner need help in this area. One thing to be sure to talk about is the importance of being honest but not brutal when self-assessing. Most people are harder on themselves than others are, and learners need to be reminded that they don't need to identify every little thing they are doing wrong. Instead, they should look for what they are doing right and build on those strengths. A good way to learn how to do this is for both the tutor and the learner to assess a particular lesson or skill separately and then compare results. Often the learner will be harder on herself than the tutor was. This activity can be repeated many times and provides a good way for the learner to become more comfortable with self-assessment. The tutor can also use information from this activity to prepare future lesson plans. It is important to follow up on the learner's comments or ideas. For example, if she mentions that she doesn't really understand the meaning of a certain word, the tutor can use the word in a variety of sentences to better demonstrate its meaning. Or, if the learner indicates that she is struggling with how to calculate a percentage, the next lesson could focus on
41 SELF-ASSESSMENT
the cost savings of a 15% off sale. By paying attention to, and incorporating, the learner's suggestions and concerns the tutor will validate the benefits of self-assessment. This in turn will encourage the learner to expand her role in her own assessment and become a more active participant in her own learning. An easy way to help the learner get started assessing her own work is by having her choose what goes into her portfolio. Encourage her to think carefully about what she chooses to include, why it represents what she has learned and how it relates to her short-term goal. Help her to appreciate what she has accomplished.
Another way to help learners recognize their own learning is to ask them to think beyond the tutoring sessions. Encourage them to look at the changes that are happening in their lives and in what they do on a daily basis and how those changes relate to the skills they are learning. They can do this by thinking about what they do differently now or by thinking about things that they can do now that they couldn't before. These are signs of learning. Some examples include: · Reading a magazine about something of personal interest · Filling out forms at work · Following a set of written instructions · Getting a personal e-mail address and using e-mail · Volunteering to help out in the community · Helping children with their homework · Going to the library · Looking up an address or a phone number. Other signs of progress include: · Remembering something that was learned a few days ago (instead of forgetting it right away) · Finding a task easier to do than before; making fewer mistakes
Encourage them to look at the changes that are happening in their lives and in what they do on a daily basis and how those changes relate to the skills they are learning.
42
THE ROLE OF VOLUNTEERS IN THE ASSESSMENT PROCESS
· Doing a task faster than before · Feeling more confident about doing something · Needing less help than before · Explaining to someone else how to do something · Helping someone else do something · Using skills in everyday life · Using skills differently · Doing better on tests and assignments · Feeling less nervous about doing things · Trying something new · Accepting constructive criticism -- from family, friends, tutor and from one self!51 Here are some other ideas that tutors can give learner to help them recognize if they are on track with their learning: · Think back to the goal -- are you there yet? · Write the goal down where you will see it often; post it on the wall; put it on the inside cover of your binder. · Ask yourself what you have learned that day to help you reach your goal.52 Self-Assessment Tips · Start with what's good about the work; be specific. · Then look at what needs to be improved. · Focus on the skills you are learning; that will help you know what to look for. · Think about what the work is for; does it need to be done perfectly, or is it supposed to be a Rough Draft? · Don't just identify what is wrong or "bad"; think about how it could be improved.
· Keep an eye out for repetitive mistakes (you always spell "friend" wrong); that will help make it easier to fix. · Work with a partner -- this can be a fellow learner who is working on the same skills you are. · Accept that it's ok to make mistakes -- that is how we learn!53 A major assumption behind self-assessment is that learners are capable of evaluating themselves, that they have explicit goals, and that a process of self-evaluation will make them more motivated and aware of their own learning process.54
43 SELF-ASSESSMENT
44 Resources* THERE ARE MANY, MANY RESOURCES AVAILABLE to help the volunteer literacy tutor. Resources are not only books and websites, they are people too! Tutors who are interested in learning more about any aspect of literacy should talk with a staff member or ask to browse the program's bookshelves. Here are some of the resources programs might have available. If they aren't available through the agency, they are all available on loan from the AlphaPlus library. Resources can also be ordered from the organizations that produced them. To learn more about demonstrations, look for: · Developing Demonstrations Activities, a package developed as part of training delivered by the Ontario Literacy Coalition in 1999 · Linking Demonstrations with Laubach by Robyn Cook-Ritchie, Laubach Literacy Ontario · On the Level by Lindsay Kennedy, published by CLO · On-line demonstrations builder at http://demonstrations.alphaplus.ca To learn more about assessment, look for: · Assessing Up, Designing Down by Jane Barber, Ontario Literacy Coalition · CABS (Common Assessment of Basic Skills) by Judith Fox Lee and Rose Strohmaier, Literacy Link Eastern Ontario · Common Assessment in the Literacy and Basic Skills Program by the Ministry of Training, Colleges and Universities (Ontario) · Goal-Directed Assessment: An Initial Assessment Process by the Ministry of Education and training (Ontario) · Inside Outcomes, a series of bulletins written by Rose Grotsky for the Ontario Literacy Coalition · Ongoing Assessment Tracking Model for Volunteer Tutoring Programs by Kingston Literacy (commonly referred to as "The Assessment Kit") · Portfolios in Practice by Andrea Leis, Conestoga College · Way to Go!, a series of bulletins from the Ontario Literacy Coalition * This is by no means a complete listing of the resources available nor is it an endorsement of these particular resources. Website addresses were accurate at time of printing.
45 RESOURCES To learn more about volunteer tutoring, look for: · Demystifying Adult Literacy for Volunteer Tutors: A Reference Handbook and Resource Guide by Charlene Ball, Literacy Partners of Manitoba · Handbook for Literacy Tutors by Chris Harwood · Renfrew County Upgrading Program Tutor Manual by Colleen Ryan-Franey and Peggy Bridgland · The Right to Read: Tutor Handbook by Frontier College · Tutor Workshop Handbook by Laubach Literacy Ontario To learn more about skill sets, success markers and literacy levels, look for: · The Level Descriptions Manual by the Ontario Literacy Coalition · Working with Learning Outcomes: Validation Draft by the Ministry of Education and Training (Ontario) To learn more about training plans, look for: · Developing Training Plans for Learners by Jean McKinlay, Peel Adult Learning Centre · Inside Outcomes, a series of bulletins written by Rose Grotsky for the Ontario Literacy Coalition To learn about further literacy training for tutors, try: · Adult Literacy Educator Certificate Program. This program is offered through the continuing or distance education departments of Sault College, Algonquin College and Conestoga College. To do further research, try: · Centre AlphaPlus Centre 2040 Yonge St.,Toronto 416-332-1012 / 1-800-788-1120 http://alphaplus.ca · National Adult Literacy Database (NALD) http://www.nald.ca · New to Adult Literacy in Ontario? by Ann Semple, Literacy Link South Central
THE ROLE OF VOLUNTEERS IN THE ASSESSMENT PROCESS
47
Appendix 1 Two statements of principles for literacy assessment
TWO STATEMENTS OF PRINCIPLES FOR LITERACY ASSESSMENT
49 APPENDIX 1
Principles of Good Practice for Assessing Student Learning56
1 The assessment of student learning begins with educational values. Assessment is not an end in itself but a vehicle for educational improvement. Educational values should drive not only what we choose to assess but how we do so. 2 Assessment is most effective when it reflects an understanding of learning as multidimensional, integrated, and revealed in performance over time. Assessment should employ a diverse array of methods, including those that call for actual performance, using them over time so as to reveal change, growth, and increasing degrees of integration. 3 Assessment works best when the programs it seeks to improve have clear, explicitly stated purposes. Assessment is a goal-oriented process. Clear, shared, implementable goals are the cornerstone for assessment that is focused and useful. 4 Assessment requires attention to outcomes but also and equally to the experiences that lead to those outcomes. Where students "end up" matters greatly. To improve outcomes, we need to know about student experience along the way. Assessment can help us understand which students learn best under what conditions; with such knowledge comes the capacity to improve the whole of their learning. 5 Assessment works best when it is ongoing not episodic. Assessment is a process whose power is cumulative. Improvement is best fostered when assessment entails a linked series of activities undertaken over time. The point is to monitor progress toward intended goals in a spirit of continuous improvement. Along the way, the assessment process itself should be evaluated and refined in light of emerging insights. 6 Assessment is most likely to lead to improvement when it is part of a larger set of conditions that promote change. Assessment alone changes little. 7 Through assessment, educators meet responsibilities to students and to the public. As educators, we have a responsibility to the public to provide information about the ways in which our students meet goals and expectations.
50
THE ROLE OF VOLUNTEERS IN THE ASSESSMENT PROCESS
Principles of Adult-Oriented Learning Assessment
Premise
Principles
Learning is derived from multiple sources.
Recognizes multiple sources of knowing, that is, learning that occurs from interaction with a wide variety of informal and formal knowledge sources.
Learning engages the whole person and Recognizes and reinforces the cognitive,
contributes to that person's
conative, and affective domains of
development.
learning.
Learning and the capacity for self-direction are promoted by feedback. Learning occurs in context; its significance relates in part to its impact on those contexts. Learning from experiences is a unique meaning-making event that creates diversity among adult learners.
Focuses on adults' active involvement in learning and assessment processes, including active engagement in self-assessment. Embraces adult learners' involvement in and impact on the broader world of work, family, and community. Accommodate adult learners' increasing differentiation from one another given varied life experiences and education.
THE ROLE OF VOLUNTEERS IN THE ASSESSMENT PROCESS
51
Appendix 2 Information about focus groups used to research this manual
INFORMATION ABOUT FOCUS GROUPS USED TO RESEARCH THIS MANUAL
53 APPENDIX 2
What is a focus group?
A group of individuals gathered together to discuss and comment on a particular topic. This is done to help a researcher collect opinions, beliefs and attitudes.
Discussion should be comfortable and enjoyable as participants share ideas.
group members assist each other by responding to ideas and comments.
Remember:
There are no wrong answers!
54
THE ROLE OF VOLUNTEERS IN THE ASSESSMENT PROCESS
Sample Letter Sent to Programs before Focus Group Session To: ABC Literacy From: Vicki Trottier Date: July 4, 2001 Re: Focus Group Sessions Further to our phone calls today, we will try to set up Focus Group sessions with your volunteers (and staff are also welcome) to discuss the volunteer's role in the assessment process. Hopefully, we can arrange for a session at 9:00 a.m. in Yourtown on Wednesday, September 12 followed by an evening or late afternoon session in Othertown the same day. Here is the information that you can share with potential focus group members. The definitions and project description are in a separate attachment. And of course, you can also remind them about Tutor Bulletins #1 and #2. The overall goal of the project is to gather information about the role of volunteers in the assessment process so that CLO can produce a resource that will be of use to all of us working in the field. We will meet for approximately one hour, although I can certainly stay longer if we get into some really interesting discussions! The session will have four parts: First, we'll start with a discussion of assessment. I am enclosing a sheet with MTCU definitions. Then, we'll discuss the role your volunteers currently play in assessment -- initial, ongoing and exit. Hopefully, I'll hear about how you find out information about the learners, how you record it and what you do with the information (i.e. how it affects decisions about demonstrations, moving on to next steps, etc.) If you have certain forms or documented procedures, etc., I'd love to see them; hopefully, you'll also give me permission to use them in the resource manual that will be written at the end of the project. Next, we will talk about how you might use volunteers in different ways. In a perfect world, what would the role of volunteers be in the assessment process? If that's different from what you do now, why is it different (e.g. time constraints, training considerations, etc.) Finally, we'll talk a bit about CLO. How can CLO support your program both within the scope of this project and in other areas? I'm not making any promises here -- just gathering information. If you have any questions, please give me a call. I hope that we are able to arrange the focus groups; they have gone well in other areas and they provide me with a lot of information that I'll be using in the resource manual.
THE ROLE OF VOLUNTEERS IN THE ASSESSMENT PROCESS
55
Appendix 3 Tutor Bulletins #1 through 4 These bulletins were widely circulated to the Ontario community-based literacy field for distribution to volunteer tutors. Feedback indicated that they were very well-received by both staff and tutors. They are fully photocopiable and may be distributed freely.
THE ROLE OF VOLUNTEERS IN THE ASSESSMENT PROCESS
65
Appendix 4 Sample Tutor Job Descriptions These job descriptions can be adapted for use in community-based agencies. Please credit the original source as indicated. They have also been included on a CD-ROM.
SAMPLE TUTOR JOB DESCRIPTIONS
67 APPENDIX 4
Role Basic Training Location Hours Duration Duties
Literacy Tutor58 To help an adult non-reader acquire reading, writing, speaking, listening, numeracy and other life skills needed to function successfully in society. This is usually done through a one-to-one relationship that emphasizes personal attention. A fifteen-hour approved basic literacy tutor workshop. Literacy Council Office or any mutually agreed-upon location such as the community center, library, school, church, etc. Twice weekly, two hours each lesson. Preparation time is also required for each lesson. A minimum one-year commitment. · Contact the student to make arrangements regarding the time and place of the lessons. · Notify the office when the first tutoring session has taken place. · Meet regularly and punctually with the student (summer and holiday times excepted). · Maintain the student's confidentiality at all times. · Provide encouragement and support, assisting the student to develop a positive self-image and an enthusiasm for learning. · Set goals with the student and frequently evaluate progress toward them. · Prepare lessons to meet the individual needs and interests of the student. · Inform the Tutor Co-Ordinator if any problems arise. · Keep accurate records of hours tutored and student progress and submit written reports monthly to the Office Co-Ordinator. · Notify the office if tutoring terminates even for a short period of time. · Keep yourself up-to-date on literacy issues by reading newsletters and attending seminars and meetings.
68
THE ROLE OF VOLUNTEERS IN THE ASSESSMENT PROCESS
Qualifications Benefits
A tutor should be dependable, interested in people, sensitive to others, a good listener, literate (professional training not necessary), flexible, patient, optimistic, friendly, non-judgmental, open-minded and have a good sense of humour. · Personal satisfaction · Deepened understanding of values and lifestyles different from your own. · Broadened imagination for creative problem-solving.
SAMPLE TUTOR JOB DESCRIPTIONS
69 APPENDIX 4
Volunteer Adult Literacy Tutor59
Role
· To assist an adult learner to improve basic reading, writing or math skills, usually on a one-to-one basis. · To assist an adult learner to meet his or her needs and personal goals.
Qualifications
· Grade 12 education or equivalent · 18 years of age or older · Proficient and confident in reading and writing · Patient, enthusiastic, creative · Supportive, flexible, dependable
Responsibilities
· Attend foundational tutor orientation/training · Attend in-service training · Understand, accept and work effectively with people from diverse backgrounds · Assist students to meet their self-identified goals · Plan and instruct individualized, relevant lessons · Involve learners in choosing materials and evaluating lessons · Provide Program Co-Ordinator with accurate documentation (tutoring hours, activities, student progress, etc.) · Respect and maintain confidentiality
Support Provided
· Initial and ongoing training · Learning resources · Consultation with Program Co-Ordinator
Time Commitment · 2­4 hours per week · Minimum 6 months
70
THE ROLE OF VOLUNTEERS IN THE ASSESSMENT PROCESS
Volunteer Literacy Tutor60
Overview
The tutor's job is to facilitate the student's learning, not to prescribe or dictate what the student must know. A good tutor is one who: · knows something about adult learners and about teaching literacy and is willing to keep learning him/herself · is responsive to the needs of the individual student · respects the student and appreciates that the student brings skills and prior knowledge to this learning experience · is empathetic, caring and interested in the student · can establish a comfortable non-threatening atmosphere where learning can take place · has good communication skills: i.e., can explain well; understands what it is the student does not know or understand; finds different ways to present material so the student can understand it; gives genuine feedback and constructive criticism; welcomes feedback and suggestions from the student · is creative: able to try or adapt different teaching/learning techniques to keep lessons interesting · is enthusiastic: enthusiasm fosters confidence in the student and a positive attitude about learning · is well-organized for each lesson.
Roles and Responsibilities We expect that volunteer tutors will: · commit to four hours a week for at least six months · attend the basic tutor training offered by the program · maintain confidentiality regarding any personal information, inquiries and documents the student may share with you · notify the Co-Ordinator immediately if an uncomfortable, dangerous or emergency situation should arise.
SAMPLE TUTOR JOB DESCRIPTIONS
71 APPENDIX 4
As well, the volunteer tutor will: · establish and maintain a regular, yet flexible, tutorial schedule · arrange a safe and mutually convenient meeting place with the student · assist in developing and implementing a personalized learning program for the student · develop and use a weekly lesson plan and keep a record of student progress · maintain an accurate record of contact hours and submit it to the office monthly · attend tutor group meetings · inform the student of relevant events within the program and in the literacy field · respect that people have different learning styles and understand that reading and writing are only two skills and that people may have other valuable knowledge and experience. The Program's Responsibilities · provide adequate training, materials and support · maintain regular contact and be available for consultation when you have difficulties or suggestions · keep you informed about the program and about developments in the larger literacy community · facilitate networking opportunities with other tutors.
72
THE ROLE OF VOLUNTEERS IN THE ASSESSMENT PROCESS
Job Description for a Tutor61
Purpose Training Duties Qualifications Location Hours fringe benefits
To teach adults the necessary literacy skills for better survival in our society. New tutor orientation is six hours and introduces methods in Adult Basic tutoring. · After being paired with a student, the following is expected: · Meet regularly and punctually with the student. · Get to know the student's long and short-term goals. · Prepare individualized lessons and present them to the student. · Assign achievable learning tasks that the student can do independently. · Review the student's independent work and try a new approach when needed. · Evaluate the student's work and give feedback. · Compliment the student for his/her efforts. · Encourage the student if he/she has difficulty learning a new skill. · Keep record of student's progress. · Report monthly to the Council office on student's progress and any change in schedule. Tutor must be committed to helping individuals develop literacy skills, be respectful of the student and his/her right to confidentiality, and be flexible, friendly, patient and optimistic. A sense of humour is helpful. In any location suggested by the Literacy Council; i.e. libraries, churches, schools, restaurants, our office, etc. After completing the orientation, the tutor and student should meet once or twice weekly for a total of 3-4 hours. Scheduling is up to you and your student. Heightened perception of the world around you, deepened understanding of values and lifestyles different from your own and satisfaction in helping others achieve their potential.
THE ROLE OF VOLUNTEERS IN THE ASSESSMENT PROCESS
73
Appendix 5 Sample Report Forms These forms can be filled out by the tutor, the learner or the tutor and the learner working together. They have been supplied in an easy-to-use format and can be adapted to suit your needs. They are also available on CD-ROM. Please credit the original source where indicated. Where a form does not have a source, it was created for this manual and should be credited accordingly.
74
THE ROLE OF VOLUNTEERS IN THE ASSESSMENT PROCESS
Tutor and Learner Weekly Summary62
Please fill in this report at the end of every tutoring session. It can be completed by the learner, the tutor or by both working together.
Learner: _______________________ Tutor: _______________________
Date:
_______________________
Total Hours: _______________________
We reviewed...
This week we worked on (new skills)...
What I liked best about this sessions was...
What I liked least was...
Next week, we plan to work on...
(Remember to review your training plan at least every four sessions. Are you on track?) Additional comments:
SAMPLE REPORT FORMS
75 APPENDIX 5
Session Report63
Learner: Tutor:
Date: Time spent:
We reviewed: ____________________________________________________________ _________________________________________________________________________ _________________________________________________________________________ _________________________________________________________________________
This week I worked on... __________________________________________________ _________________________________________________________________________ _________________________________________________________________________ _________________________________________________________________________
With tutor: _______________________________________________________________ _________________________________________________________________________ _________________________________________________________________________ _________________________________________________________________________
Student Reflection:________________________________________________________ _________________________________________________________________________ _________________________________________________________________________ _________________________________________________________________________
Learning Highlights: _______________________________________________________ _________________________________________________________________________ _________________________________________________________________________ _________________________________________________________________________
Tutor Comments: _________________________________________________________ _________________________________________________________________________ _________________________________________________________________________ _________________________________________________________________________
Resources Used: · · · ·
76
THE ROLE OF VOLUNTEERS IN THE ASSESSMENT PROCESS
Exit Report64
Student: Tutor:
Date: Length of time Together:
1. What activities did you find the most helpful and/or enjoyable? Why? _________________________________________________________________________ _________________________________________________________________________ _________________________________________________________________________ _________________________________________________________________________ _________________________________________________________________________ _________________________________________________________________________
2. What activities were not helpful and/or enjoyable for you? Why not? _________________________________________________________________________ _________________________________________________________________________ _________________________________________________________________________ _________________________________________________________________________ _________________________________________________________________________ _________________________________________________________________________
3. Most progress was made with ... _________________________________________________________________________ _________________________________________________________________________ _________________________________________________________________________ _________________________________________________________________________ _________________________________________________________________________ _________________________________________________________________________
4. Other comments: _________________________________________________________________________ _________________________________________________________________________ _________________________________________________________________________ _________________________________________________________________________ _________________________________________________________________________
SAMPLE REPORT FORMS
Student-Tutor Report
Student: Date:
Tutor: Location:
Please complete this form after every fourth session and return to the office. From time to time, please include a sample of your student's work to help us track progress. Encourage the student to help you complete this report.
Date of Session
Length of What we Worked On
Session
(include resources)
How it Went (include successes and difficulties)
APPENDIX 77 5
Date of Session
Length of What we Worked On
Session
(include resources)
How it Went (include successes and difficulties)
THE ROLE OF VOLUNTEERS IN THE ASSESSMENT PROCESS
78
SAMPLE REPORT FORMS
Date of Session
Length of What we Worked On
Session
(include resources)
How it Went (include successes and difficulties)
Is there anything else you would like to tell us? ___________________________________________________________________________________________________ ___________________________________________________________________________________________________ ___________________________________________________________________________________________________ ___________________________________________________________________________________________________ Thank you for your time!
APPENDIX 79 5
80
THE ROLE OF VOLUNTEERS IN THE ASSESSMENT PROCESS
Progress Report65 Your answers to the following questions will help staff address your needs. Work together with your tutor to fill out this form once every four to six sessions. If you are not sure of how to fill it out, please contact a staff member. Learner's name: __________________________________________________________ Tutor's name:_____________________________________________________________ Reporting period: _____________________ to _______________________________ Frequency of meetings: ____________________________________________________ Length of sessions:________________________________________________________ What is your short term goal?
What are your long term goals?
What books and learning techniques have you used? What worked well? Why? What didn't work very well? Why not?
Remember -- progress can be more than just an improvement in your reading and writing skills. Progress can mean you are more confident, your attendance has improved or you participate more in lesson planning. Do you think you have made progress?
SAMPLE REPORT FORMS Why or why not? Please give examples.
81 APPENDIX 5
Please tell us about any problems you have run into. How did you solve them?
Please use the rest of the page for any other comments you would like to make:
82
THE ROLE OF VOLUNTEERS IN THE ASSESSMENT PROCESS
Tutoring Report66
Tutor's Name: Date: Prep Hours:
Learner's Name: No. of meetings since last report: Tutoring Hours:
Please describe the learner's achievements in the following areas: Work-related _________________________________________________________________________ _________________________________________________________________________
Family-related _________________________________________________________________________ _________________________________________________________________________
Community-related _________________________________________________________________________ _________________________________________________________________________
Personal _________________________________________________________________________ _________________________________________________________________________
What specific skills did you work on in the following outcome areas: Communications (reading, writing, speaking & listening) _________________________________________________________________________ _________________________________________________________________________ Numeracy (numbers sense & computation, geometry, measurement, data & probability) _________________________________________________________________________ _________________________________________________________________________ Self-Management and Self-Direction _________________________________________________________________________ _________________________________________________________________________ Please include a sample of the learner's work that you think highlights the skills he/she is working on.
83 Are there any books or other resources that you found especially helpful? Please list them. _________________________________________________________________________ _________________________________________________________________________ Are there any other books or resources that you would like us to help you find? Please list them. _________________________________________________________________________ _________________________________________________________________________ Is there anything else we can do to help? _________________________________________________________________________ _________________________________________________________________________ Please remember to let us know as soon as possible if tutoring stops or if you or the learner has a new address or telephone number. Thank you!
84
THE ROLE OF VOLUNTEERS IN THE ASSESSMENT PROCESS
Student-Tutor Report67
This report can be completed by the student, the tutor or both working together.
Student:
Tutor:
Date:
Short-term Goal:
Math skills worked on
Week 1
Week 2
Week 3
Week 4
Group work, one-on-one or independent study
What went well? Why?
SAMPLE REPORT FORMS
What didn't go so well? Why not?
Week 1
Week 2
Week 3
Week 4
How do these skills relate to your short-term goal? Communications skills worked on
Group work, one-on-one or independent study What went well? Why?
APPENDIX 85 5
86
What didn't go so well? Why not?
Week 1
Week 2
Week 3
Week 4
THE ROLE OF VOLUNTEERS IN THE ASSESSMENT PROCESS
How do these skills relate to your short-term goal?
Self-management /self-direction skills worked on Group work, one-on-one or independent study
SAMPLE REPORT FORMS
What went well? Why?
Week 1
Week 2
Week 3
Week 4
What didn't go so well? Why not?
How do these skills relate to your short-term goal?
APPENDIX 87 5
88
THE ROLE OF VOLUNTEERS IN THE ASSESSMENT PROCESS
Week 1
Week 2
Week 3
Week 4
Please tell us how your new skills are making a difference in your life. For example, are you more confident about your reading? Or are you helping your children with their homework? Please tell us about it: ___________________________________________________________________________________________________ ___________________________________________________________________________________________________ ___________________________________________________________________________________________________ ___________________________________________________________________________________________________
Is there anything else you would like to tell us? ___________________________________________________________________________________________________ ___________________________________________________________________________________________________ ___________________________________________________________________________________________________ ___________________________________________________________________________________________________
Thank you for taking the time to fill in this report! It helps us make sure we are helping you work towards your goals and lets us know about the progress you are making.
SAMPLE REPORT FORMS
My Progress Tracking Sheet This sheet should be filled in at least once each month. Your tutor can help you fill it in if you need help.
Learner: ______________________________________________ Tutor: ________________________________________________ Date: _________________________________________________
My long-term goal: _______________________________________________________________________________________________________ My short-term goal: ______________________________________________________________________________________________________
I believe I am making progress towards my short-term goal
I don't think I'm making much progress

Please explain why you think you are or aren't making progress: ________________________________________________________________________________________________________________________ ________________________________________________________________________________________________________________________
When we developed your training plan, you said you would like to work on certain skills to help you reach your short-term goal. These skills are listed in the boxes below. As you master each skill, shade in the small box. This will help you see your progress more clearly. Be sure to carry over last month's progress on to this sheet!
APPENDIX 89 5
Please use the "Activities" box to tell us how you have been learning these skills. Be sure to include the names of the books and other resources you used. You can also attach samples of your work.
90
THE ROLE OF VOLUNTEERS IN THE ASSESSMENT PROCESS
SKILLS
LEARNING ACTIVITIES
1
2
3
4
5
Communications Reading 1. 2. 3. 4. 5.
1
2
3
4
5
Writing 1. 2. 3. 4. 5.
1
2
3
4
5
Speaking & Listening 1. 2. 3. 4. 5.
SAMPLE REPORT FORMS
SKILLS
LEARNING ACTIVITIES
1
2
3
4
5
Numeracy Number sense & computation 1. 2. 3. 4. 5.
1
2
3
4
5
Measurement 1. 2. 3. 4. 5.
1
2
3
4
5
Geometric Problems 1. 2. 3. 4. 5.
APPENDIX 91 5
1
2
3
4
5
Data & Probability 1. 2. 3. 4. 5.
92
SKILLS
LEARNING ACTIVITIES
1
2
3
4
5
Self-Management/Self-Direction 1. 2. 3. 4. 5.
THE ROLE OF VOLUNTEERS IN THE ASSESSMENT PROCESS
You have been making progress towards your short-term goal. Once you get there, what do you think you would like to do next? ___________________________________________________________________________________________________ ___________________________________________________________________________________________________ Do you have any other comments you would like to make? ___________________________________________________________________________________________________ ___________________________________________________________________________________________________ Thank you for filling out this form. It will help us keep track of your progress towards your goal.
SAMPLE REPORT FORMS
Plan Your Work and Work Your Plan68
Name: Date:
Tutor: Short-term Goal:
Please think about what you would like to work on with your tutor over the next month. Write this in the "skills and activities" section of the chart. Remember to think about how the work you will be doing relates to your short-term goal and your training plan.
When you have finished working on something, mark it off in the "completed" box. This will help make it easier for you to keep track of your learning. It will also help you see what areas you may need to spend more time on.
This sheet will be kept in your file as part of an ongoing record of your progress.
Outcome Reading
Skills/Activities Worked On
Completed!
Writing
APPENDIX 93 5
Outcome Speaking & Listening
Skills/Activities Worked On
Math
Self-Management / Self-Direction
Completed!
THE ROLE OF VOLUNTEERS IN THE ASSESSMENT PROCESS
94
SAMPLE REPORT FORMS
Keeping Track69
Use this box to write about something you did, something you noticed or something that interested you.
Use this box to list some of the new skills or ideas you worked on this month.
Use this box to make a list of the books and resources Use this box to keep a list of new words you have you used this month. Be sure to include any check-ups, learned. Include a definition for each of them. tests or assignments.
APPENDIX 95 5
97 Bibliography Books, Articles, and Other Printed Resources American Association for Higher Education. 9 Principles of Good Practice for Assessing Student Learning, 1996. Available at http://www.aahe.org/principl.htm Ananda, Sri. Equipped for the Future Assessment Report: How Instructors can Support Adult Learners Through Performance-Based Assessment. National Institute for Literacy, 2000. Baker, Janette M. Sharing our Experiences: Learning New Skills. Scarborough: Self-Directed Studies Literacy Program, 1993. Ball, Charlene L. Demystifying Adult Literacy for Volunteer Tutors: A Reference Handbook and Resource Guide. Literacy Partners of Manitoba, 1996. Available online at http://www.nald.ca/clr/demyst/cover.htm Barber, Jane. Assessing Up, Designing Down. Toronto: Ontario Literacy Coalition, 2001. Bartel, Kathleen. A Study of Assessment Devices and Procedures used in Voluntary Literacy Programs. Wisconsin: Literacy Volunteers of America, 1991. The Basic Skills Agency. Assessing Progress in Basic Skills Literacy. London: 1997. Belfiore, Mary Ellen. Adult Literacy Educator Skills List. Toronto: Ontario Literacy Coalition, 1999. Blankenship Cheatham, Judy; Ruth Johnson Colvin and Lester L. Laminack. Tutor: A Collaborative Approach to Literacy Instruction, 7th Edition). Syracuse: Literacy Volunteers of America, Inc., 1993. Boyd, Mary Thompson and Paula Davies. Doing it for Ourselves: A Collection of Resource Tools for Volunteer Tutor Program Coordination. Vancouver: Literacy BC, 1997. Core Literacy Centre. A Model for Learning Assessment in Community-Based Literacy Programs. Waterloo: 1996.
98
THE ROLE OF VOLUNTEERS IN THE ASSESSMENT PROCESS
D'Amico-Samuels, Deborah. Perspectives on Assessment from the New York City Adult Literacy Initiative. New York: Literacy Assistance Center, 1991. Einarson, Nan and Karen Burwell. Resource Package for Community Literacy Groups. 1992. Faessler, Lana. Tutor Training in Ontario. Laubach Literacy Ontario, 1998. Fox-Lee, Judith and Rose Strohmaier. CABS/Common Assessment of Basic Skills: Initial Assessment in 5 Levels, 3rd ed. Kingston: Literacy Link Eastern Ontario, 2000. Gould, Laurie. Intake Assessment Materials: Fundamental Level English and Math. Vancouver Community College, 1999. Harwood, Chris. Handbook for Literacy Tutors. Ottawa-Carleton Coalition for Literacy, 2001. How to Assess Progress in ABE: A Collection of Techniques and Models. (Source unknown) Huebner, Fiona. A Guide for the Development of Policies and Procedures in Ontario's Community Literacy Agencies. Community Literacy of Ontario, 1999. Integrating Assessment, papers presented at NCVER's National Conference, Perth, Australia. National Centre for Vocational Education Research, 1996. Kennedy, Lindsay. On The Level: Demonstrating Skills & Knowledge in Ontario's Community Literacy Agencies. Community Literacy of Ontario, 2000. Kennedy, Lindsay and Fiona Huebner. Skills for the Future. Community Literacy of Ontario, 2000. Kingston Literacy. Ongoing Assessment Tracking Model for Volunteer Tutoring Programs (Revised). 2000. Koehler, Sandra and Peggy Dean. Monitoring Student Progress. Des Plaines: Adult Learning Resource Centre, 19__. Lefebvre, Susan. Building a Learning Plan: Tutor Training. From MTML's Literacy Worker's Training Course.
99 BIBLIOGRAPHY Lockwood, Sheri. Yukon Learn Tutor Training Manual. 1996. Manitoba Education and Training. Adult Literacy Volunteer Tutor Pack, 19__. Manitoba Education and Training. Creative Student Assessment: A Guide to Developing Meaningful Evaluation. 19__. McGrail, Loren, and Laura Purdom eds. Adventures in Assessment: Learner-Centred Approaches to Assessment and Evaluation in Adult Literacy. Volume 1: Getting Started. System for Adult basic education Support, 1991. McGrail, Loren, and Laura Purdom eds. Adventures in Assessment. Volume 3: Looking Back, Starting Again. System for Adult Basic Education Support, 1992. McGrail, Loren ed. Adventures in Assessment. Volume 2: Ongoing. System for Adult Basic Education Support, 1992. McGrail, Loren ed. Adventures in Assessment. Volume 4. System for Adult Basic Education Support, 1993. McGrail, Loren ed. Adventures in Assessment. Volume 5: The Tale of the Tools. System for Adult Basic Education Support, 1993. McGrail, Loren ed. Adventures in Assessment. Volume 6: Responding to the Dream Conference. System for Adult Basic Education Support, 1994. Metz, Elizabeth. "The Issue: Adult Literacy Assessment." ERIC Clearinghouse on Reading and Communication Digest #45. October 1989. Also available at http://www.Indiana.edu/~eric_rec/ieo/digests/d45.html Ontario Literacy Coalition. The Level Descriptions Manual. 2000. Ontario Literacy Coalition. Partners in Common Assessment. 2000. Ontario, Ministry of Education and Training. Goal-Directed Assessment: An Initial Assessment Process. 1997. Ontario, Ministry of Education and Training. Working With Learning Outcomes: Validation Draft. 1998. Ontario, Ministry of Training, Colleges and Universities. Common Assessment in the Literacy and Basic Skills Program. 2000.
00
THE ROLE OF VOLUNTEERS IN THE ASSESSMENT PROCESS
Ontario Training and Adjustment Board, Learning and Employment Branch, Literacy Section. Quality Standards for Literacy: A Practitioner's Guide to the Accountability Framework for the Adult Literacy Education System and Core Quality Standards for Programs. 1995. Osborne, Ann. Recognizing and Accrediting the Learning of Adult Literacy Students. M.Ed. Thesis. Kingston: Queen's University, 1996. Paul, Marianne. Way to Go! (Issues I-IV). Ontario Literacy Coalition, 2001. Paul, Marianne and Robb MacDonald. Sharing the Path: Developing a Common Understanding of Assessment. Kitchener: 2001. Payne, Marianne. Development of a Goal-Oriented Intake and Referral Protocol. Hamilton Adult Basic Education Association of Hamilton-Wentworth, 1999. Radencich, Marguerite C. ed. Adult Literacy: A Compendium of Articles from the Journal of Reading. Delaware: International Reading Association, 1994. Rickard, Patricia L. Assessment in Adult Literacy Programs. Philadelphia: National Center on Adult Literacy, 1991. Rose, Amy D. and Meredyth A. Leahy eds. Assessing Adult Learning in Diverse Settings: Current Issues and Approaches. San Francisco: Jossey-Bass Publishers, 1997. Roussy, Yvonne. Literacy Skills Assessment Project Final Report. Kitchener: Project Read Literacy Network, 1995. Roussy, Yvonne. Taking the First Step: An LBS Referral Assessment. Kitchener: Project Read Literacy Network, 2000. Rowen, Norman S. Notes for the Roundtable: Some Thoughts on Learning Assessment. A Discussion Paper prepared for the Articulation Standards Project Steering Committee, 1996. Ryan-Franey, Colleen and Peggy Bridgland. The Renfrew County Community Upgrading Program Tutor Manual. Renfrew County Community Upgrading Program, 1997.
101 BIBLIOGRAPHY Sanders, William L. and Sandra P. Horn. "Educational Assessment Reassessed: The Usefulness of Standardized and Alternative Measures of Student Achievement as Indicators for the Assessment of Educational Outcomes." Education Policy Analysis Archives, Volume 3, number 6. (March 3, 1995) Available at http://olam.ed.asu.edu/edaa/v3n6.html Schneider, Melody and Mallory Clarke. Dimensions of Change: An Authentic Assessment Guidebook. Greensboro, NC: Peppercorn Books, 1999. Semple, Ann. New to Adult Literacy in Ontario? What Literacy Staff Need to Know. Literacy Link South Central, 2001. Simmons, Alison ed. Adventures in Assessment. Volume 7: The Partnership Project. System for Adult Basic Education Support, 1994. Simmons, Alison ed. Adventures in Assessment. Volume 9. System for Adult Basic Education Support, 1996. Simmons, Alison ed. Adventures in Assessment. Volume 10. System for Adult Basic Education Support, 1997. Skage, Sharon and Marnie Schaetti. Setting the Compass: A Program Development Evaluation Tool for Literacy Programs in Alberta. The Association of Literacy Coordinators of Alberta, 1999. Soltonovich, Irma. Volunteer Tutor/Learner Guide: Literacy Training Guide for Offenders in the Communty. Victoria: John Howard Society of BC, 1995. Spruck Wrigley, Heide. "A Modest Proposal," in Adventures in Assessment. Volume 11. System for Adult Basic Education Support, 1998. Tri- County Tutor: The Newsletter for Tutors and Other Volunteers of Tri-County Literacy Council, Volume 11, no. 1 (Winter/Spring 2000). Trussell-Cullen, Alan. Assessment in the Learner-Centred Classroom. Dominie Press (California), 1998. Tutor Handbook. Thunder Bay Literacy Group, 19__. Valencia, Sheila W., Elfrieda H. Hiebert, Peter P. Afflerback, eds. Authentic Reading Assessment: Practices and Possibilities. Delaware: International Reading Association, 1994. Vella, Jane, Paula Berardinelli, Jim Burrow. How do they Know they Know?: Evaluating Adult Learning. San Francisco: Jossey-Bass Publishers, 1998.
02
THE ROLE OF VOLUNTEERS IN THE ASSESSMENT PROCESS
Wiehe Arnade, Celia. "Using Volunteers as Aides in the Adult ESL Classroom." ERIC Digest. November 1997: National Center for ESL Literacy Education. Available online at http://www.cal.org/ncle/ digests/Voluntr.htm Web-Based Resources http://www.ymcaphilly.org/tutor/index.htm Tutor Resources from the Philadelphia YMCA. Includes introduction to assessment and tutoring tips. http://www.literacyvolunteer.homestead.com "Information and Inspiration for Literacy Volunteers." The focus is on family literacy but some good tips for all tutors. Includes sections for adult literacy and second-language literacy. http://www.research.umbc.edu/~ira/lessweb.htm Adult Education Resources on the Web. An excellent source of links to a variety of lesson plans and ideas. http://www.nwrel.org/learns Northwest Regional Educational Laboratory. Contains a variety of useful resources, including tutor training tips. Be sure to check out "Tutor," a newsletter published three times/year. Although the focus is on tutors for children, many of the techniques apply in adult literacy. http://www.nald.ca/pwc People, Words & Change (Ottawa) website; includes section for volunteers and resources. http://CharityChannel.com/newsletters/vfr Volunteer Management Review. A free weekly electronic newsletter edited by Nan Hawthorne. Contains information use for both volunteers and managers of volunteers. http://www.ncrel.org/sdrs/areas/issues/content/cntareas/reading/li7lk29.htm Literacy Assessment Techniques from the North Central Regional Educational Laboratory. Although designed with children in mind, this site includes many techniques that can be used in an adult setting.
103 BIBLIOGRAPHY http://nces.ed.gov/naal National Assessment of Adult Literacy. American site seeking to: describe the status of adult literacy in the United States; report on national trends; and identify relationships between literacy and selected characteristics of adults. http://www.lcrb.org Literacy Council of Reading-Berks, West Lawn PA. http://www.sanmateopubliclibrary.org/projread Project Read, San Mateo public library CA. Personal communications The following people were very helpful in the research phase of this project. Their insights and comments contributed greatly to the development of this manual. Andrew, Sarah. YMCA career development and Learning Centre, YMCA of Hamilton/Burlington. June 14, 2001. Bentson, Marilyn. Tacoma Community House, Tacoma WA. October 5, 2001. Connelly, Carrie. Wellington County Learning Centre, Arthur ON. June 4, 2001. DeNeire, Cheryl. North Channel Literacy Council. Espanola ON. December 10, 2001. Fleming, Ian. Georgian College, Parry Sound ON. December 12, 2001. Horne, Ashley. Trent Valley Literacy Association, Peterborough ON. July 6, 2001. Marshall, Sheila. Timmins Learning Centre, Timmins ON. August 16, 2001. McCallum, Tracy. Quinte Literacy Group, Belleville ON. June 13, 2001. Mitton, Mary. The Literacy Council of Brantford and District, Brantford ON. July 2, 2001. Morgan, Connie. The Barrie Literacy Council, Barrie ON. June 11, 2001.
04
THE ROLE OF VOLUNTEERS IN THE ASSESSMENT PROCESS
Torvi, Lee. Renfrew County Community Upgrading Program, Eganville ON. June 29, 2001. Wink, Corry. Trent Valley Literacy Association, Peterborough ON. June 12, 2001. Focus Groups Literacy Council of North Timiskaming, Kirkland Lake ON. March 2001. Literacy Network Northeast, Timmins ON. March 2001. North Bay Literacy Council, North Bay ON. August 2001. North York Public Library, Toronto ON. September 2001. Sarnia/Lambton Literacy Council, Sarnia ON. September 2001. Simcoe Literacy Network, Gravenhurst ON. March 2001.
105 References 1 Core Literacy Centre, A Model for Learning Assessment in CommunityBased Literacy Programs (Waterloo), p. 27. 2 Now the Ministry of Training, Colleges and Universities. 3 Ontario, Ministry of Education and Training, Working with Learning Outcomes: Validation Draft (1998), p. 6. 4 Ontario Training and Adjustment Board, Learning and Employment Preparation Branch, Literacy Section, Quality Standards for Adult Literacy: A Practitioner's Guide to the Accountability Framework for the Adult Literacy Education System and Core Quality Standards for Programs (1995), p. 30. 5 Irma Soltonovich. Volunteer Tutor/Learner Guide: Literacy Training Guide for Offenders in the Community (John Howard Society of British Columbia, 1995), p. 8. 6 Alan Trussell-Cullen, Assessment in the Learner-Centred Classroom (California: Dominie Press, 1998), p. 6. 7 The terms "learner" and "student" are used interchangeably throughout this manual. 8 Ontario, Ministry of Training, Colleges and Universities, Common Assessment in the Literacy and Basic Skills Program. 9 Ibid. 10 Working with Learning Outcomes, p. 24. 11 The terms "he" and "she" are used interchangeably throughout the manual to avoid the use of "he/she." This in no way implies a gender preference or bias. 12 Ontario, Literacy and Basic Skills Section, Workplace Preparation Branch, Ministry of Education and Training, Goal-Directed Assessment: An Initial Assessment Process, p. 10. 13 Ibid., p. 1 and p. 11.
06
THE ROLE OF VOLUNTEERS IN THE ASSESSMENT PROCESS
14 Ibid., p. 11. 15 YMCA Philadelphia Adult Education Network website: http://www.ymcaphilly.org 16 Working with Learning Outcomes: Validation Draft, p. 34. 17 Goal-Directed Assessment: An Initial Assessment Process, p. 10. 18 Colleen Ryan-Franey and Peggy Bridgland, Renfrew County Community Upgrading Program Tutor Manual, Renfrew County Community Upgrading Program (1997), Section 7, p. 3. 19 Volunteer Tutor/Learner Guide: Literacy Training Guide for Offenders in the Community. p. 11. 20 Elizabeth Metz, "The Issue: Adult Literacy Assessment," ERIC Clearinghouse on Reading and Communication Digest #45 (October 1989). http://www.Indiana.edu/~eric_rec/ieo/digests/d45.html 21 Janette M. Baker, Sharing our Experiences: Learning New Skills (Scarborough: Self-Directed Studies Literacy Program, 1993), p. 5. 22 Community Literacy of Ontario is currently working on Phase 3 of the Practitioner Training Strategy (Skills for the Future). This work will include the development of a provincial standard for tutor training. 23 http://www.nald.ca/pwc 24 Chris Harwood, Handbook for Literacy Tutors (Ottawa-Carleton Coalition for Literacy, 2001), Section 5, p. 7. 25 Renfrew County Community Upgrading Program Tutor Manual, Section 3, p. 7. 26 Tutor Job Description, Literacy Council of South Temiskaming. 27 Tutor Job Description, Midland Area Reading Council. 28 Fiona Huebner, A Guide for the Development of Policies and Procedures in Ontario's Community Literacy Agencies (Community Literacy of Ontario, 1999), p. 139. 29 Ongoing Assessment Tracking Model for Volunteer Tutoring Programs (Kingston Literacy, 2000, revised), introduction.
107 REFERENCES 30 Tutor's Monthly Report Form, Literacy Council of Norristown, Pennsylvania, http://Norristown_literacy.tripod.com/tutForm.htm 31 Sharon Skage and Marnie Schaetti, Setting The Compass: A Program Development Evaluation Tool for Literacy Programs in Alberta (The Association of Literacy Coordinators of Alberta, 1999), p. 31. 32 Visit these websites at http://www.ci.cambridge.ma.us, http://www.sanmateopubliclibrary.org/projread, http://www.lcrb.org, http://www.www.kentliteracy.org. 33 Lindsay Kennedy and Fiona Huebner, Skills for the Future (Community Literacy of Ontario, 2000). 34 Australia, National staff development Committee, Handbook for Coordinators of Adult Literacy Volunteer Tutor Programs (1995), pp. 72-73. 35 From a conversation with Corry Wink, Trent Valley Literacy Association, June 2001. 36 Celia Wiehe Arnada, "Using Volunteers as Aides in the Adult ESL Classroom," ERIC Digest, National Center for ESL Literacy Education (November 1997), http://www.cal.org/ncle/digests/Voluntr.htm 37 The terms "training plan" and "lesson plan" are not synonymous. A training plan is a document developed by a staff member in consultation with the learner. It provides an overview of the skills the learner will need to demonstrate in order to reach her short-term goal. The lesson plan, on the other hand, is a more detailed (and often less formal) way for the tutor to plan the specifics of the tutoring sessions. 38 Eunice N. Askov, Barbara L. VanHorn, Priscilla S. Carman, "Assessment in Adult Basic Education Programs" in Assessing Adult Learning in Diverse Settings: Current Issues and Approaches, edited by Amy D. Rose and Meredyth A. Leahy (Jossey-Bass Publishers), p. 71. 39 Janet Isserlis, "Volunteer Tutors and Learner Assessment: What Counts Here?" in Adventures in Assessment, Volume 9, pp. 17-21. 40 Mary Mitton, Literacy Council of Brantford and District, in a personal communication; July 2001. 41 Handbook for Literacy Tutors, Section 5, p. 18.
08
THE ROLE OF VOLUNTEERS IN THE ASSESSMENT PROCESS
42 Janet Isserlis, "What you See: Ongoing Assessment in the ESL/Literacy Classroom" in Adventures in Assessment, Volume 2, p. 42. 43 In some situations, the training plan may also need to be revised to reflect more realistic timelines or to incorporate the possibility of additional or different demonstrations of learning. When a tutor and learner think it might be necessary to review the training plan before the time originally agreed upon, they should contact a staff member. 44 Judy Blankenship Cheatham, Ruth Johnson Colvin and Lester L. Laminack, Tutor: A Collaborative Approach to Literacy Instruction (Literacy Volunteers of America, Inc., 1993), p. 117. 45 Handbook for Literacy Tutors, Section 5, pp. 6-7. 46 Charlene L. Ball, Demystifying Adult Literacy for Volunteer Tutors: A Reference Handbook and Resource Guide (Literacy Partners of Manitoba), p. 109. 47 Handbook for Literacy Tutor, Section 5, p. 18. 48 Marianne Paul, Marianne. Way to Go! Issue 3 (Ontario Literacy Coalition: 2001), p. 1. 49 Assessment in the Learner-Centred Classroom, p. 11. 50 Ibid., p. 40. 51 Adapted from Way to Go! Issue 3, pp. 6-7, and A Model for Learning Assessment in Community-Based Literacy Programs, p. 17. 52 Marianne Paul, Way to Go! Issue 1 (Ontario Literacy Coalition: 2001), p. 5. 53 Adapted from Way to Go! Issue 3, p. 9. 54 Paul Trunnell, "Self-Assessment: Doing and Reflecting" in Adventures in Assessment, Volume 2, p. 31. 55 This is by no means a complete listing of the resources available nor is it an endorsement of these particular resources. Website addresses were accurate at time of printing.
109 REFERENCES 56 9 Principles of Good Practice for Assessing Student Learning. This document was developed under the auspices of the American Association for Higher Education (AAHE) Assessment Forum, 1996. It can be viewed in its entirety at the AAHE website, http://www.aahe.org 57 Carol E. Kasworm and Catherine A. Marienau, "Principles for Assessment of Adult Learning" in Assessing Adult Learning in Diverse Settings: Current Issues and Approaches, p. 8. 58 Adapted from Tutor Job Description, Muskoka Literacy Council. 59 Setting the Compass; p. 31. 60 Adapted from Renfrew County Community Upgrading Program Tutor Manual, Section 3, pp. 2-4 61 Adapted from Literacy Council of Reading-Berks, West Lawn PA, http://www.lcrb.org/volunteer.shtml 62 Adapted from Wellington County Learning Centre 63 From Tri-County Literacy Council. 64 From Tri-County Literacy Council. 65 Adapted from the North York Public Library Adult Literacy Program. 66 Adapted from Project Read, San Mateo Public Library, California, http://www.sanmateopubliclibrary.org/projread/pages/ monthlyreport.htm 67 adapted from "Record of Participation" in Paul Trunnell's "The Partnership Project", Adventures in Assessment, Volume 7, 1994, p. 27. 68 Adapted from LBS Program, Georgian College, Parry Sound Campus. 69 Adapted from LBS Program, Georgian College, Parry Sound Campus.

File: the-role-of-volunteers-in-the-assessment-process.pdf
Title: book.frame6
Author: Steve Knowles
Published: Thu Apr 11 19:32:22 2002
Pages: 110
File size: 1.6 Mb


Who thought of that, 2 pages, 0.17 Mb

The Sea-Change, 21 pages, 0.07 Mb

, pages, 0 Mb

, pages, 0 Mb
Copyright © 2018 doc.uments.com