Using mobile phones for vocabulary activities: Examining the effect of platform, G Stockwell

Tags: activities, mobile phone, Language Learning & Technology, Mobile Phones, Vocabulary Activities, learners, vocabulary, PC platforms, mobile platforms, mobile platform, production activities, writing activities, the mobile phone
Content: language learning & Technology http://llt.msu.edu/vol14num2/stockwell.pdf
June 2010, Volume 14, Number 2 pp. 95­110
USING MOBILE PHONES FOR VOCABULARY ACTIVITIES: EXAMINING THE EFFECT OF THE PLATFORM Glenn Stockwell Waseda University While problems such as small screens and inconvenient keypads have been pointed out by researchers (e.g., Thornton & Houser, 2002), we still have little knowledge of how the mobile platform affects the way in which activities are completed and how learners make decisions about using mobile phones. Stockwell (2007b) provided preliminary evidence that learners generally require more time to complete vocabulary activities and achieved slightly lower scores on mobile phones when compared to completing the same activities on desktop computers, but data in the study were limited. The current study examines 175 pre-intermediate learners of English who could choose to complete vocabulary activities on either a mobile phone or a desktop computer to identify the effect of the mobile platform. Data were collected from three cohorts of learners over a three-year period, and learner activity was analysed for the amount of time required to complete activities on both platforms and the scores they achieved for the activities. The results of the study are discussed in terms of how the platform affects learners' ability to complete tasks, whether continued usage contributes to improved performance or sustained usage of the mobile platform over time. Trends across the yearly cohorts were also identified. INTRODUCTION There is no doubt that mobile technologies have started to make their presence felt in the field of education, as can be seen by the increasing number of publications that have appeared in recent years (e.g., Ally, 2009; Kukulska-Hulme & Traxler, 2005). It is not surprising, then, to see mobile technologies also gaining momentum in second language learning environments, and research on Mobile learning is emerging more regularly in the CALL literature, including a special issue of the ReCALL Journal in 2008. Several types of mobile technologies are available, such as wireless laptop computers, portable MP3 players, personal digital assistants (PDAs), and electronic dictionaries, although mobile phones are one technology that has started to attract particular attention from educators. One of the primary reasons for the popularity of the mobile phone is the widespread infiltration of the market, particularly in Japan and the US (see Levy & Kennedy, 2005, for a discussion). This means that there is no need for the institution (or in many cases, the teacher) to provide learners with the hardware in order to incorporate a mobile learning component into their teaching context. Added to this is the fact that mobile phones are relatively inexpensive as compared with, for example, wireless laptop computers, and with functions such as Internet browsers that are available in current mobile phones, the range of possibilities of mobile phones as tools for learning increases even further. While podcasting is another area which has gained in popularity over the past few years (e.g., Ducate & Lomicka, 2009; Rosell-Aguilar, 2007), there are limitations in the interactivity that MP3 players can achieve, generally restricted to playing audio or, more recently, video. In contrast, most modern mobile phones have either e-mail or Short Message Service (SMS) functionality, which means that information can be forwarded to and from mobile phones by teachers or students. Internet-capable mobile phones allow immediate connection to a server, which makes it possible for learners to retrieve updated or specific information as they require it, and for teachers to maintain detailed logs of access. The range of research into the use of mobile phones for language learning has been diverse. In one study, for example, Kiernan and Aizawa (2004) used mobile phones to have learners exchange e-mails with one another in order to teach targeted structures, while Taylor and Gitsaki (2003) required their learners to use the browser function of their phones to perform Internet searches, and Levy and Kennedy (2005) sent learners language learning related text notifications through SMS about what they had learnt during class
Copyright © 2010, ISSN 1094-3501
95
Glenn Stockwell
Using Mobile Phones for Vocabulary Activities
or details of upcoming television programs they wanted the learners to watch. Each of these studies capitalises on different features of mobile phones (e-mail, web browsers and SMS) and illustrates the broad potential of the phone as a learning tool. Mobile Phones and Vocabulary Learning Studies investigating using mobile phones for learning vocabulary have also started to appear in the literature, and the nature of the activities and the focuses of the research have been varied. Browne and Culligan (2008), for example, provide an overview of an environment where learners complete activities on a computer, after which time they can access vocabulary flash cards on their mobile phones that are generated based on items that the system predicts that they need to work on. In their study, a description is given of how the activities are beneficial, specifically that targeted items are provided for learners to study at a time and place that suits them, but details are not given regarding how the system was actually used by the learners. Another example is described by Thornton and Houser (2005), where learners were asked to access video lessons about English idioms from their mobile phones during class time and complete short multiple choice activities about the idioms they had learnt, also on their mobile phones in class. The materials were given a positive evaluation by the learners, who found them both fun and useful. One study that attempted to investigate the way in which learners acquire vocabulary through mobile phones was conducted by Chen, Hsieh, and Kinshuk (2008). Learners deemed to have varying verbal and visual Learning Skills according to an online survey into short-term memory abilities were provided with four different types of annotations for learning English vocabulary depending on their learning preferences determined in the survey. Flashcards were sent to their mobile phones via SMS which included one of four different types of annotation; English word only, English word with written annotation, English word with pictorial annotation, and English word with both written and pictorial annotation. The flashcards were viewed in the classroom and learners were given 50 minutes to learn 24 vocabulary items. In a post-test carried out immediately after the activities on desktop computers in the classroom, they found that the pictorial annotation assisted learners who had lower verbal and higher visual ability to retain vocabulary, at least in the short term. Studies that have looked at actual mobile phone use outside the classroom include research into sending messages to learners' mobile phones by Thornton and Houser (2005) and Kennedy and Levy (2008). Both studies were based on the "push" mode of operation, that is, where teachers control the frequency and the timing of messages sent to learners. Thornton and Houser sent short mini-lessons for learning vocabulary via e-mail to learners' mobile phones three times a day, using new words in multiple contexts to allow learners to infer the meanings. Similarly, learners in Kennedy and Levy's study were given the option to receive messages which presented known words in new contexts and new words in contexts that were familiar to the learners through SMS to their mobile phones on an average of nine to ten messages per week. A survey was administered in both studies, and in each case indicated that learners felt that these messages were very helpful for learning vocabulary, although some indicated that the messages were too frequent. To determine effectiveness, the learners who received the e-mail mini-lessons in Thornton and Houser's study were compared against learners who could access the same materials through a website designed for the mobile phone and learners who were given the materials on paper. The study only ran for a two-week period but showed that the learners who received the e-mails scored better on post-tests compared with the other two groups. No measures of effectiveness were conducted in Kennedy and Levy's study. A limitation plaguing research into using mobile phones for language learning, however, is that much of it occurs in artificial environments, generally within the classroom itself. In order to get a real indication of the nature of mobile learning, it is necessary to view its use in naturalistic settings. An example of where learners' mobile phone usage was tracked outside the classroom was conducted by Stockwell (2008) in a study of 75 pre-intermediate learners of English. Learners were provided with tailored vocabulary activities based on listening activities covered in class, and were able to complete these either on mobile phones through the Internet browser function on their phones or on a normal desktop or laptop computer (PC). Surveys and server log data revealed that learners used the PC in preference to the mobile phone in the vast majority of cases, many of whom indicated from the outset that they did not intend to use the
Language Learning & Technology
96
Glenn Stockwell
Using Mobile Phones for Vocabulary Activities
mobile phone for their vocabulary study, citing problems such as the cost of Internet access, the screen size, the keypad and the study environment as the primary reasons. The study showed that 61% of learners did not use the mobile phone at all, with a further 24% of learners using the mobile phone for less than 20% of the vocabulary activities. Depending on their usage patterns across the period investigated, Stockwell classified learners as non-users, try-and-quit users, sporadic users, balanced users, or heavy users of the mobile phone. Many learners who indicated that they intended to use the mobile phone in a pre-survey either did not do so or used the mobile phone very minimally (and, incidentally, there were also learners who indicated that they did not initially intend to use the mobile phone, but later used it relatively frequently). While the reasons cited by learners such as cost, screen size and the inconvenient keypad shed some light on why some learners chose not to use the mobile phone for the activities, they give us only minimal insight into sporadic or balanced usage of the phone. This leads us to ask what it is that causes learners to choose to use one platform or another at specific times. Obviously, Environmental Factors such as access to a PC at a particular time would be expected to play some role, but are there other factors involved as well? Data from the Stockwell (2008) study tend to indicate that reasons behind platform selection are complex. There was a clear novelty effect, with the mobile phone being used for 17% of the first lesson, but only 3.2% of the last lesson, but the increases and decreases in use across the semester suggest that this was not the only factor contributing to choice of platform. Of interest was the fact that even those learners who did not use the mobile phone provided positive comments about the concept of learning through this platform, but nonetheless, it appears that their concerns outweighed the perceived benefits of using the mobile phone. These studies have suggested that while learners have a positive view of mobile learning, and feel that there are the potential benefits, not all students are willing to engage in it. What is it that leads learners to make such decisions? This question is dealt with in the following section. Effects of the Mobile Platform on Learning Current discussions on the use of mobile devices in Learning Environments generally focus on the affordances of the device, the skills and attitudes of the learner, and the environmental constraints of learning through a mobile platform. In her FRAME model (Framework for the Rational Analysis of Mobile Education), Koole (2009) proposes that mobile learning occurs in an intersection of device, learner and social aspects. Because the device acts as the interface between the learner and the activities, she argues that it is important to assess characteristics such as the physical characteristics (e.g., size and weight), input capabilities (e.g., keypad or touchpad), output capabilities (e.g., screen size and audio functions), file storage and retrieval, processor speed, and the error rates (i.e., malfunctions which result from flaws in hardware, software and/or interface design). In addition, she proposes that learner skills also play a central role, and prior knowledge and experience with mobile devices for learning--as well as feelings towards activities--can either positively or negatively affect the way in which learners engage themselves with mobile-based tasks. Though not directly discussed by Koole, consideration of the environment in which learning occurs and the psychological barriers is paramount (Wang & Higgins, 2006). If learners feel that the mobile environment is not conducive to learning, it can have a detrimental effect on the way in which activities are undertaken. Learners in Stockwell's (2008) study, for example, wrote that the mobile phone was "not a tool for studying" and that they "couldn't get into study mode with the mobile" (p. 260) as reasons for not using the mobile phone to complete the activities. This perception of the problems with the mobile device as learning tool is one that must be overcome if mobile phones are to enter the mainstream. In saying this, however, it is a problem that may eventually cease to be an issue by itself as acceptance of the mobile phone as learning tool becomes more widespread. Where once many students and teachers may have had a less than positive view towards computers for learning, we now see computers used with learners of all ages, and in the spirit of "digital natives" as coined by Prensky (2001), there are learners who perceive learning through computers as more natural than through more traditional means. Until we reach this stage with mobile learning, however, there is a need to design our learning environments around the existing technologies in such a way as to encourage learners to feel comfortable with working within them. Unfortunately, as KukulskaHulme (2005) argues, many of the mobile devices that learners have access to are simply not designed for
Language Learning & Technology
97
Glenn Stockwell
Using Mobile Phones for Vocabulary Activities
educational purposes, meaning that learners find them difficult to use for the activities that teachers expect them to undertake. While some of the blame for this may lie with developers, in many cases it is not that such devices do not exist, but rather that those that are appropriate for specific purposes are too expensive for many learners. The goal for teachers, then, is to be aware of what tools learners possess, and to chose and/or adapt resources to suit these tools. Stockwell's (2008) study showed that even when materials were adapted to their mobile phones, when given a choice of whether to use their mobile phone or a PC for completing vocabulary-learning activities, the overwhelming majority of learners chose the PC. The purpose of the current study was to look specifically at whether there were any features inherent to completing activities on the mobile platform that may have affected learners' decisions to use a PC rather than their mobile phone (and vice-versa). Do they achieve higher scores with the PC so opt to use the PC instead? Alternatively, do learners find that the activities take too long to complete on the mobile phone? Rather than looking at subjective survey data, the current study looks at the actual scores achieved by learners depending on the activity type, the amount of time spent engaged on the activities, and any longitudinal differences that were apparent across the period investigated. In order to investigate these issues, the following research questions were posed: 1. Are there differences in the scores achieved in activities completed on mobile and PC platforms? 2. Are there differences in the time required to complete activities on mobile and PC platforms? 3. Do learners improve in speed and scores over time on each platform? The method used for the study is described in the following section. METHOD Participants The study was conducted over a three-year period with three individual cohorts coming to a total of 175 learners. All learners were enrolled in a compulsory first-year English-language subject in the School of Law at Waseda University, Tokyo, which focussed predominantly on improving listening skills and vocabulary. The three cohorts were made up of students enrolled in a total of seven classes; three classes in 2007 (n = 80), two classes in 2008 (n = 50) and two classes in 2009 (n = 45). All seven classes were taught by the author and the teaching approach and content were the same. The classes were not streamed, and learners were randomly assigned by the faculty's student affairs office. Learners were mostly fresh out of high school, and were all aged between 18 and 21. They were generally considered as being of a pre-intermediate level, with TOEIC scores ranging between 450 and 650, the majority being at the lower end of the range. Most learners were quite motivated to achieve high scores in the subject in order to maintain their GPA (Grade Point Average) to enable them to enter the graduate law school. According to a survey administered at the end of the semester, none of the learners had ever used a mobile phone for language learning before taking the subject. The survey also indicated that nearly two-thirds of the learners had planned to use the mobile phone at the beginning of the semester. Classes were held once a week for a 15-week period, and as class time was dedicated mainly to listening activities, learners were required to study vocabulary outside of class (more details are provided in the following section). Vocabulary activities based on the textbook materials were developed and made available to learners either on PC or on their mobile phones. An orientation on how to use the activities was given in the first class in the semester, which included showing learners how to log in and complete each activity type. Time was also spent on ensuring that learners understood how to complete the activities, and learners were given the opportunity to ask questions about how to use the system on both the PC and the mobile platforms in class. The vocabulary activities were included as part of the assessment for the subject (10% of the overall score for the subject), and learners were told that they could choose between using a PC, their mobile phone, or any combination that they wanted to during the semester. It was emphasised that there was no pressure to use one platform or the other, and that activities completed on either platform would be included in their progress equally. They were also informed that they could switch between platforms at will, and could even start a lesson on one platform and complete it
Language Learning & Technology
98
Glenn Stockwell
Using Mobile Phones for Vocabulary Activities
on the other if they saw fit. They were told in advance of the study that data would be collected and used for research and further development purposes, but that records would be collected anonymously with no information linking their scores to their identity. Scores for individual activities were not included as part of the assessment, and learners were assigned a grade depending on how many of the ten lessons they completed. Records of completed lessons were correlated with real names at the end of each semester so that grades could be assigned, with a score of 10 being awarded if all activities were completed by the end of the semester. Learners were able to access the PC version of the materials from any computer, including the university computer laboratories or their homes. The mobile phone version was also available anywhere that learners had a signal, and provided that they had Internet capabilities on their phones. All 175 learners indicated orally in the first class that they did have a phone with this function. The mobile system was trialled with Internet-capable handsets of each of the three major mobile phone carriers in Japan (NTT DoCoMo, KDDI AU, and Softbank) and no incompatibility issues were apparent. Apart from fixing some minor bugs such as formatting errors, the system remained essentially unchanged across the three-year period. Vocabulary Activity System The vocabulary activity system entitled VocabTutor was the same system that was used in both the Stockwell (2007b and 2008) studies, so only a brief description has been included here. The system was written in PHP and MySQL, and integrated into Moodle, which was used for management of class grades and for provision of the audio passages for the lessons covered in the textbook. When students undertook the vocabulary activities on PC, they were required to log into the Moodle system, and then follow a link to the activities. The mobile system was the same as the PC system in content, but had a simplified interface to fit the smaller screen, and graphic images were removed to make loading faster. Learners accessed the mobile version from a different address on the same server, and logged in using the same user name and password as they used for Moodle. Both the PC and mobile versions accessed the same databases, and regardless of which platform was used for completing the activities, they were recorded in exactly the same way, but a record was kept of which platform was used. The vocabulary activities for each lesson were designed to include more passive activities in the beginning, only requiring learners to select the correct word from a list of alternatives, through to more productive activities at the end, where learners were required to write the correct word in the appropriate tense. There were initially six different activity types included with the system, but one was dropped in the first week of the first cohort because of a programming error, and to maintain consistency was excluded from the two subsequent cohorts. The remaining activity types included: (a) choosing the appropriate word for an English sentence, (b) choosing the appropriate English word for a Japanese meaning, (c) choosing the appropriate English word for an English definition, (d) writing a word in English for an English definition, and (e) writing the appropriate English word for an English sentence. Writing consisted of no more than a single word per question in both the PC and Mobile Systems. This was to maintain a simpler interface when considering the mobile platform, and to keep as much consistency as possible between the mobile and PC platforms. Examples of each of the question types are included in the Appendix. Note that the question type and format was the same for each platform, so the examples given in the appendix apply to both platforms. The system consisted of an intelligent engine that adapted to the learners depending on whether they scored correctly or incorrectly for each vocabulary item. Items for each lesson were presented in random order, and then assigned a competency score depending on learner responses. If the learner answered correctly the first time an item appeared, it was assigned an initial competency score of 6, whereas if they scored incorrectly, the item was assigned an initial competency score of 3. For each correct response, the competency score for each item increased by 1, and decreased by 1 for each incorrect response. An item was considered as "known" by the system if it reached a competency score of 8. These numbers were selected to ensure that even if the learner got an item correct the first time, it would need to be correct a further two times in a row before considered as "known," and that if they got an item wrong the first time it appeared, the learner would need to get it correct a further five times in a row for it to be correct. Even
Language Learning & Technology
99
Glenn Stockwell
Using Mobile Phones for Vocabulary Activities
items considered as "known" were included in the activities periodically to ensure that the learner was still able to answer correctly. Items with a lower competency score were presented to the learner more frequently than those with a higher competency score, and learners needed to attain a competency score of at least 8 with all items in a given lesson before they were able to go on to the next lesson. As a result, those learners who scored more correct answers were able to progress through the lessons more quickly than those that made errors. Each lesson included 13-17 vocabulary items which were selected from the commercially produced textbook. Where multiple-choice questions were given to learners, distracters were automatically generated by the system, matching the required word in both part of speech and tense. Extensive testing was done to ensure that duplicate correct answers did not appear in the questions. The vocabulary activities were linked closely to in-class activities. As mentioned above, the primary focus of the class was listening, and the textbook contained several comprehension activities and communication tasks based on short videos. Each lesson of the textbook contained a number of key vocabulary items which were both highlighted in example sentences and the subject of a simple wordmatching task. Learners could listen to an audio version of the video either directly through the Moodle system, or they could download an MP3 version of the audio that could be transferred to their phone or other MP3 player. The class in the following week included a short listening cloze quiz where the vocabulary items were omitted from a transcript of the video. The vocabulary activities were designed to assist in learning the vocabulary which came up in the lessons to help with both the in-class activities and the listening cloze quiz. Data Collection and Analysis The data were collected through detailed server logs that were automatically kept by the system. No vocabulary pre- or post-test was undertaken as the objective of the study was not to investigate learner development with the vocabulary but rather to identify how learners used their mobile phones for language learning when they had alternative methods of completing the activities, and their performance on both platforms. The server logs kept records of, among other things, the platform the learners used to complete an activity, the lesson number, the type of activity, the time the activity was started and completed, and the score attained for the activity. Other information including the number of attempts on each item, along with the overall accuracy measures of the vocabulary items was also recorded in each learner's profile, but these were not used in the current study. As the combined logs contained over 54,000 entries, the author developed software to organize the data for analysis. Data were broken down into the total number of activities completed by each learner, the platform used, the time taken to complete each activity, and the scores that were achieved. Detailed figures were generated to determine whether learners used one platform or the other consistently, or in bursts across the semester. Comparisons across the three years were also made, which were examined for patterns in mobile usage. The results of the study are presented in the following section. RESULTS The data were first analysed to determine the proportion of mobile phone use by learners. A summary of mobile phone usage across all three cohorts (N = 175) is presented in Figure 1. There was a significant number of learners who did not use the mobile phone at all, but rather elected to complete all activities on the PC. Learners who completed all of the activities on mobile phone (100%) or all activities on PC (0%) are indicated separately. As can been seen in the figure, 60% (105 learners) did not use the mobile phone at all for the activities, and a further 18.9% (33 learners) used the mobile phone for 20% or less of the activities completed. Only very small numbers of learners used the mobile phone for the majority of the activities, with just 3 learners (1.7%) electing to use the mobile phone for all of the vocabulary activities.
Language Learning & Technology
100
Glenn Stockwell
Using Mobile Phones for Vocabulary Activities
Percentage of Students
70.0 60.0 60.0
50.0
40.0
30.0 18.9 20.0
10.0
6.9
5.1
2.9
4.6
1.7
0.0
0
1-20 21-40 41-60 61-80 81-99 100
Percentage of Activities
Figure 1. Percentage of the vocabulary activities completed by the learners on the mobile phone (N = 175).
A breakdown of the mobile phone usage in the three cohorts can be seen in Table 1. The number of learners who did not use the mobile phone at all came to 58.8% in 2007, rising to 78% in 2008, with a sharp drop in 2009 to only 40.4%. A much wider spread of mobile usage was evident in 2009 when compared to earlier years, and although 2009 was the only year in which no learners opted to use the mobile phone for all activities, there was a larger proportion of learners compared with previous years (11.1%) who used the mobile phone for more than 80% of the activities.
Table 1. Percentage of Activities Completed on Mobile Phones in Each Year (N = 175)
Percentage of Tasks Completed on Mobile 100 81-99 61-80 41-60 21-40 1-20 0 n
2007 1 (1.3%) 3 (3.8%) 2 (2.5%) 2 (2.5%) 6 (7.5%) 19 (23.8%) 47 (58.8%) 80
Number of Learners by Year 2008 2 (4.0%) 0 (0.0%) 1 (2.0%) 1 (2.0%) 3 (6.0%) 4 (8.0%) 39 (78.0%) 50
2009 0 (0.0%) 5 (11.1%) 2 (4.4%) 6 (13.3%) 3 (6.7%) 10 (22.2%) 19 (42.2%) 45
Analysis of the data shows that there was not a great difference in the scores achieved as a result of the platform. As Figure 2 illustrates, while the degree of difficulty of the activity seemed to be higher as the tasks became more active (as shown by the lower scores), the scores achieved on both platforms were generally very similar. Some scores were marginally higher on the PC (Activity 3 and Activity 5) and others were marginally higher on the mobile (Activity 4). There was some degree of variation in learner scores for each of the activities, with a standard deviation of 15.44, 22.32, 25.14, 29.98, and 29.26 respectively for the PC, and 15.70, 22.81, 26.29, 32.15, and 32.24 for the mobile, indicating that there was a greater variation in the scores for the more difficult writing activities.
Language Learning & Technology
101
Glenn Stockwell
Using Mobile Phones for Vocabulary Activities
Score
100.0 90.0 80.0 70.0 60.0 50.0 40.0 30.0 20.0 10.0 0.0
Activity 1
Activity 2 Activity 3 Activity 4 Activity Type
PC Mobile Activity 5
Figure 2. The scores achieved for each activity on both PC and mobile phone (N = 175). A general overview of the amount of time required to complete each activity on the PC and mobile platforms for all learners was generated, as can be seen in Figure 3. The figure shows that each of the activities took significantly longer to complete on the mobile phone when compared with the PC, generally requiring around 1.4 minutes more for each activity. Despite the fact that Activities 1, 2, and 3 were multiple choice questions where the learner simply clicked on a radio button before each option, and that Activities 4 and 5 required learners to write complete words in English, there did not seem to be a particularly big difference in time for the production activities compared with the multiple choice.
Minutes
5.0
PC
4.5
Mobile
4.0
3.5
3.0
2.5
2.0
1.5
1.0
0.5
0.0
Activity 1 Activity 2 Activity 3 Activity 4 Activity 5
Activity Type
Figure 3. The number of minutes required to complete each activity on both PC and mobile phone (N = 175).
Language Learning & Technology
102
Glenn Stockwell
Using Mobile Phones for Vocabulary Activities
While general data regarding all 175 participants in the study were informative, comparisons of data specific to the activities on each platform required sufficient activities to have been completed on both the PC and the mobile phone. As such, those learners who completed between 21% and 80% of the activities on mobile phone were selected for comparison, which came to a total of 26 learners. Samples of the usage patterns during the semester have been included in Figure 4. Mobile phone usage was indicated with a circle ("o") while PC usage was indicated with a line ("-"). The column marked "Number" indicates the number of the student in the database records (not their university student number). The results showed that there was a great deal of variation in how the learners used the mobile phone. As the examples shown in Figure 4 demonstrate, there was extended usage of one platform followed by short bursts on the other for some learners (Student 602), while others used both platforms in a relatively balanced manner (Student 667). For learners who mainly used the PC, there were rather random instances of a single attempt on the mobile phone, but this pattern was rare for users who mainly used the mobile. There were also cases where some learners started mainly on one platform, and then over a period of time changed to predominantly use the other, sometimes with occasional use of the original platform until the end of the semester. There was also variation in the total number of activities that were completed, ranging from 95 through to 167. This depended on how correct the responses were, and if there were a lot of errors in the responses, learners were required a do a higher number of activities in order to complete all of ten lessons. The learner who was only required to complete 95 activities achieved perfect scores for almost every activity.
Student 602
Pattern ---o-o--oooooooooooo----oooooo---o-o--oooo-----ooo------ooooo---ooo
667
----ooooo--------------oooooooooooo--------oooooooooooooo----------
Figure 4. Examples of learner usage patterns of mobile phones.
In order to determine how the learners progressed throughout the semester on both the mobile and PC platforms, the mean scores and the time taken to complete each of the ten lessons were calculated and compared. The scores for each of the lessons are presented in Table 2. As can be seen in the table, there is not a great difference between the two platforms, with higher scores being achieved on the PC for some lessons, and higher scores being achieved on the mobile phone for others. There was also very little difference in the scores in terms of improvement across the semester. Because each lesson included different vocabulary, improvement was not expected as a result of having acquired the vocabulary, but rather, the scores were looked at longitudinally to determine whether any "practice effect" of using the activities across an entire semester might contribute to improved scores over time. The scores were slightly higher in the last three lessons overall, but as this is not reflected evenly in the independent PC and mobile scores, it is difficult to conclude that there was any practice effect.
Table 2. Mean Scores (%) for the Activities Broken Down by Lesson (n = 26)
Platform
Lesson
Used
1
2
3
4
5
6
7
8
9
10
PC
81.1 81.0 81.4 84.5 84.6 83.8 85.0 89.7 86.2 87.1
Mobile 83.4 77.5 86.3 79.7 82.2 83.1 82.5 86.7 93.1 89.0
Overall 82.2 79.6 84.3 82.2 83.8 83.4 83.5 88.0 89.5 87.7
The mean amount of time taken to complete the activities showed a much more definitive difference. The overall time has been broken down by lesson in Table 3. The table shows that it took consistently longer to complete the lessons on the mobile phone when compared with the computer, and that this time
Language Learning & Technology
103
Glenn Stockwell
Using Mobile Phones for Vocabulary Activities
difference did not appear to decrease throughout the semester. For the most part, lessons that took slightly less time on the PC also took a comparatively shorter amount of time on the mobile phone, and the difference was relatively consistent at around 1.3 to 2.0 minutes. As with the scores for the activities, there were no real trends that became apparent across the semester. It was interesting to note, however, that only eight of the 26 learners completed the final lesson, Lesson 10, on their mobile phone, preferring to use the computer instead. Similarly, only nine learners used it to complete Lesson 5.This pattern may have been a result of the tests which were held after the 5th and the 10th lessons, and was a tendency that was also seen in the earlier study (Stockwell, 2008). Apart from these two lessons (and a slightly lower figure for Lesson 9), overall usage remained relatively consistent on both platforms throughout the semester.
Table 3. Mean Time (Minutes) Taken to Complete the Activities per Lesson (n = 26)
Platform
Lesson
Used
1
2
3
4
5
6
7
8
9
10
PC
2.23 1.91 2.54 2.13 1.61 2.30 2.01 2.00 1.89 1.60
Mobile 3.97 3.78 4.49 3.46 3.20 3.58 3.62 4.03 3.48 3.16
Overall 3.08 2.68 3.72 2.77 2.18 2.99 2.95 3.15 2.65 2.14
One observation that was noted regarding platform choice was that learners for the most part completed a lesson on one platform or the other, and although some learners occasionally made a single attempt on the other platform midway through a lesson, the data indicated that usage was generally in blocks according to lesson. Examination of the access logs confirms that learners did not tend to leave a long time in between activities within a single lesson, and the majority of learners did the activities for each lesson in quick succession, and then stopped after the lesson was completed. In some cases it appeared that learners decided to swap from the mobile phone to the PC after activities as a result of the amount of time taken to complete the activities. This swapping of platform took a couple of different forms. The first type is exemplified by a learner who started on the activities exclusively using the mobile phone, but after the first two lessons decided to try to use the PC in conjunction with the mobile phone. While there was not a great difference in scores, the learner took significantly longer on the mobile phone for the third and fourth lesson, and then chose to use the PC only for the fifth and sixth lessons, swapping back and forth between the mobile and the PC for the remainder. The second type can be seen in a learner that started the first two lessons using both the mobile phone and the PC, but did not display a great difference in time between the two platforms. For the third lesson, they completed all of the activities on the mobile phone, but took much longer than for the activities done on the phone for the previous two lessons. As a result, they chose to use the PC for the fourth lesson, and completed the activities very quickly, so then stuck with using the PC for the rest of the semester. It should be pointed out, however, that this was not consistent for all of the learners. There were learners who took considerably longer to complete activities on the mobile phone when compared with the PC but did not change to the PC. In one such example, the learner completed the first two lessons using both the PC and the mobile phone, but took much longer to do the activities on the mobile phone than the PC. Despite this, they decided to use the mobile phone almost exclusively from the third lesson onwards for all of the remaining lessons.
DISCUSSION Features of the Mobile Platform The study set out to shed light on the use of a mobile phone as a tool for completing interactive vocabulary activities when compared with completing the same activities on a desktop (or laptop) computer. There were three main questions that the study sought to answer, dealing with the scores achieved and the time taken on each platform, and any longitudinal patterns that occurred across the
Language Learning & Technology
104
Glenn Stockwell
Using Mobile Phones for Vocabulary Activities
semester. The first question looked at the differences in scores achieved in activities completed on both mobile and PC platforms. This showed no consistent difference between the mobile phone and the PC, with some activities scoring higher on one platform and others scoring higher on the other. It was initially expected that learners might score lower on the productive activities (Activity 4 and 5) on the mobile phone when compared with the PC, as the smaller screen and keypad were definitely less convenient for entering text. Surprisingly, no such difference was apparent, with the learners actually scoring slightly higher on Activity 4 on the mobile than the PC, and a very marginal difference in favour of the PC for Activity 5. The second research question focused on the differences in time required to complete activity on mobile and PC platforms. The results of the study clearly demonstrated that, with the activities used in the current study at least, there was a clear difference in the amount of time required to do activities on the mobile phone compared with the computer. Despite the fact that the interface was simplified as much as possible to make the possible effects of the small screen and keypad as minimal as possible, learning through the mobile phone just took much longer. The results indicated that the time taken to complete different types of activity (i.e., multiple choice or word production) did not seem to be greatly affected by the platform, and activities that took longer on PC seemed to take a proportionately longer amount of time on the mobile phone. It is difficult to tell whether the extra time taken to complete the activities on the mobile phone was a result of the mobile phone interface or because of other unforseen environmental issues that might arise when completing tasks on a mobile phone. From an interface perspective, care was taken to ensure that when a question was shown, all of the possible responses to that question could be seen on the screen at the same time. However, if learners wished to check the answers of several questions in an activity, they would have to scroll up and down through the screen in order to see all of the questions, whereas on the PC this could be done with little or no scrolling at all. This scrolling may have contributed to the time taken, particularly if the learner wanted to confirm all questions repeatedly before submitting their answers. Environmental factors which may have contributed to the extra time taken on the activities are more difficult to determine, but it is possible to consider scenarios that might shed some light on why it took longer on the mobile phone compared to the PC. The mobility of the platform means that learners could do the activities in any conceivable environment where the phone can access the Internet, such as in a train, in the library, walking down the street or in a coffee shop. Learners who complete activities on a busy train, for example, may find it difficult to concentrate, as they are preoccupied with other things around them, such as ensuring they do not miss their stop or keeping their balance if they are standing. As a result, they may not do the activity as a single unit where they answer all questions at once, but instead answer one question at a time, with their mind focussed elsewhere in between. This would likely give the impression of taking much longer to complete the activities when in fact the actual time spent inputting was much the same, but with additional periods where the activity window was open without working on it. Obviously it is unlikely that all learners were in non-ideal environments as they completed the activities, but there is the potential that it contributed in some cases. This could only be confirmed, however, by having learners keep a detailed journal themselves of where they did each of the activities on the mobile phone. The final research question sought to determine whether learners improve in speed and scores over time on each platform. The data did not seem to suggest that this was the case, and activities in some lessons took longer than other lessons, regardless of the platform. Similarly, some lessons appeared to be more difficult than others, and learners achieved higher scores in some than others with very little difference between the scores on the mobile phone or PC. Was there, then, any effect evident after completing the activities for a period of time? One interesting observation was that in the first couple of lessons, learners were more likely to swap around between platforms during a single lesson. Once they had gotten used to each platform, however, they were far less likely to change platform midway through the lesson. It appears that the learners were using the initial lessons to determine how they felt about the platforms, and then the subsequent lessons were completed on the platform that they felt best suited their surroundings or feelings at a given time.
Language Learning & Technology
105
Glenn Stockwell
Using Mobile Phones for Vocabulary Activities
Changing Perceptions of Mobile Learning? A significant result of the study was the change in patterns in mobile phone usage over the three years that the study was conducted, and the increase in usage of mobile phones in 2009 may be indicative of a changing perception of mobile phones as a learning tool. In saying this, however, it should be noted that the number of learners who did not attempt to use the mobile platform at all was consistently high, with nearly 60% in 2007, nearly 80% in 2008, and over 40% in 2009. The reasons for the high number of learners who did not use the mobile in 2008 compared with 2007 and 2009 are unknown, as the learners were received information about the platforms in very much the same way as with the other years. When adding the minimal users who completed less than 20% of activities on the mobile into the equation, however, the gap between 2007 and 2008 becomes less conspicuous, with learners who used the mobile for less than 20% of the time (including non-users) coming to 82.6% in 2007 and 86% in 2008. When compared to the figure of 64.4% in 2009, it might be concluded that there is a general increase in mobile use in 2009 compared with previous years. One trend that was more obvious in 2009 compared with previous years was that there were more learners who opted to continually use the mobile phone despite taking significantly longer than the PC to complete the activities. Of these learners there were some who used the PC consistently at the beginning, and then trialled the mobile phone on a couple of occasions intermittently with the PC. Those learners who then chose to continue using the mobile phone would appear to have made a decision that even with the extra time taken to complete the activities, the mobile platform is a viable means through which to do the activities. We can judge from this that learners are quite capable of rationalising their time, and deciding on what is best for themselves in a given situation. They may feel, for example, that the time that might be wasted on the train during commuting may be better spent studying vocabulary, even though it would take far less time on a computer. The time in the train ceases to become dead time to the learner, but rather becomes extra time that can be used productively without needing to take away from other tasks that need to be done when they have access to a computer. Further investigation in the future is necessary to explain this trend, as it has the potential to mark mobile phone use not as something that detracts from time that might be spent on the computer, but rather as a way of creating new time spaces. In this way, just as Thornton and Houser (2005) pointed out that their learners indicated that they preferred to look at e-mail messages sent to their mobile phones at a time when they could concentrate on them, the current lack of usage of the mobile phone may not necessarily be an inherent feature of the technology itself, but rather it may simply be a case of learners looking for an appropriate time and place to use it. While there was an increase in mobile phone use over time, the study showed that numbers remained quite low. What are the possible factors that contribute to the continuing low usage of the mobile phone, with many learners not using it at all? Apart from the obvious factors such as perceived costs and inconvenience of the interface, one feasible suggestion is that learners just found that the PC was sufficient for their learning needs. This is not to say that the mobile phone is an unnecessary tool for language learners, but it should be kept in mind that not all learners will be willing to engage in activities on them. Since learners were aware of the fact that the activities were the same on the PC and the mobile phone, they may have felt that it was just easier to do the activities on the PC while they were doing other things, such as writing reports or checking e-mail. Thus, as is the case with any technology, integration that sees the mobile phone linked with other activities--learning and non-learning--would likely bring about a greater willingness to engage in activities on them. Limitations of the Study The study was not without limitations. Firstly, the system was not updated each year as was originally hoped when it was developed (apart from minor formatting bugs), predominantly due to time constraints. While using the same system did allow for comparison across the three years with the same system, factors such as varying the number of questions on each platform (see the following section) and testing different types of input methods to cater for difficulties with keypad entry were not investigated. Another limitation was that precisely what experience learners had with technology for language learning prior to the study was not known. While a post-questionnaire did reveal that none of the learners had used
Language Learning & Technology
106
Glenn Stockwell
Using Mobile Phones for Vocabulary Activities
mobile phones for learning before the study, experience with other technologies was not determined. It is conceivable that if learners had experience with PCs in advance of the study, they would naturally use the platform that they were more comfortable with, hence this information would have been useful. A further limitation was the study did not specifically identify how many of the learners commuted and how long was spent on commuting. Given the potential importance of commuting time for completing the vocabulary activities on the mobile phone, this information could shed light on whether those who did not use the mobile phone chose not to simply because they did not have relatively unbroken periods of time without a computer in front of them where they could engage in the activities, such as might occur while commuting, or if there were other factors involved. A final limitation was that learners' usage habits with the mobile phone for non-learning purposes were not known. The post-questionnaire identified that learners used their mobile phones for sending personal e-mails, checking news and train information, but questions asking for details such as frequency were not included on the questionnaire. Knowing what learners use their mobile phones for in non-learning environments could provide valuable information to determine how mobile usage might be improved. Implications for Further Research As could be seen from the limitations above, several questions remain regarding learning vocabulary through mobile phones. Firstly, the impact of how the questions are presented on both platforms requires further investigation. In the current study, both the mobile and the PC displayed, depending on the question type, eight to ten questions at a single time. If the number were reduced for the mobile phone, it is possible that it may take less time, and hence learners may be more inclined to use the mobile. It should be noted, however, that if the number of questions per activity are reduced, there would need to be more activities undertaken to get through the vocabulary, hence there is the possibility of more access costs. Further research would shed light onto whether learners are prepared to do more activities--bearing in mind the potential increased cost--in order to have a slightly simpler interface, or whether they prefer to complete the activities on the mobile even though it is likely that it will take longer. Secondly, as was noted in the limitations, the research did not reveal very much about the locations the mobile phones were used for completing the activities. Having learners keep a journal of exactly when and where they used the mobile activities, along with detailed information of how they use their mobile phones for non-learning activities would be very valuable in terms of activity design. For example, if learners frequently check e-mail on their mobile phones, sending reminders to their phones including a link to activities, such as in the "push" mode employed by Kennedy and Levy (2008), may prompt them to complete the activities rather than passively waiting for learners to access them of their own accord. Obviously there are cost concerns (receiving e-mails on mobile phone is not free) that mean that some learners may not want this imposed on them. It would be interesting to see, however, whether providing this option contributes to improved usage. CONCLUSION Mobile learning for language learning has reached a stage where it is starting to move out of the classroom and into the real world. Through mobile phones, we have the potential to provide a rich learning environment for our learners, but there are still issues that must be considered before they can reach their full potential. Obviously, there is still the problem of the lack of willingness to try new mobile technologies, but this is something that may slowly become less of an issue as perceptions change. The fact that activities may take longer on mobile phones compared with computers does not necessarily detract from their usefulness. Just as teachers needed time to find appropriate times and places for using computers in the early days of CALL, the same could be said about the need to find a time and place for mobile phones. Even with computers these concepts have changed as computers became connected to the Internet, and learners no longer needed to visit laboratories but could access materials from home at times that were convenient to them. With the mobile phone, there is an even greater sense of freedom of time and place, but this freedom also can make it more difficult to make decisions about which times and places are the most suitable.
Language Learning & Technology
107
Glenn Stockwell
Using Mobile Phones for Vocabulary Activities
Thus, freedom is about experimentation and making choices. This experimentation and decision-making must occur on the part of both teachers and learners. Many learners in the current study were able to make decisions about how to use the mobile phone as a direct result of trying both the PC and mobile platforms, and determining what they felt was best for them. Examining how learners use mobile phones in natural contexts can help inform teachers in designing activities and materials for learners, who can then experiment to find what works best with their learners. As attitudes change over time, so too will learner preferences and expectations with mobile technologies. Successful use of mobile technologies relies on keeping up with our changing learners, and continuing to give them opportunities to experiment and discover.
ACKNOWLEDGMENTS I would like to express my gratitude to the three anonymous reviewers for their insightful suggestions that led to significant improvements in this paper.
ABOUT THE AUTHOR Glenn Stockwell is Professor in Applied Linguistics at Waseda University, Tokyo, Japan. His research interests include computer-mediated communication, mobile learning, and the role of technology in the language learning process. E-mail: [email protected]
REFERENCES Ally, M. (2009). (Ed.). Mobile learning: Transforming the delivery of education & training. Athabasca: AU Press. Browne, C., & Culligan, B. (2008). Combining technology and IRT testing to build student knowledge of high frequency vocabulary. The JALT CALL Journal, 4(2), 3­16. Chen, N.-S., Hsieh, S.-W., & Kinshuk. (2008). Effects of short-term memory and content representation type on mobile language learning. Language Learning & Technology, 12(3), 93­113. Retrieved from http://llt.msu.edu/vol12num3/chenetal.pdf Ducate, L., & Lomicka, L. (2009). Podcasting: An effective tool for honing language students' pronunciation? Language Learning & Technology, 13(3), 66­86. Retrieved from http://llt.msu.edu/vol13num3/ducatelomicka.pdf Kennedy, C., & Levy, M. (2008). L'italiano al telefonino: Using SMS to support beginners' language learning. ReCALL, 20(3), 315­350. Kiernan, P., & Aizawa, K. (2004). Cell phones in task based learning. Are cell phones useful language learning tools? ReCALL, 16(1), 71­84. Koole, M. (2009). A model for framing mobile learning. In M. Ally (Ed.), Mobile learning: Transforming the delivery of education & training (pp. 25­47). Athabasca: AU Press. Kukulska-Hulme, A. (2005). Mobile usability and user experience. In A. Kukulska-Hulme & J. Traxler (Eds.), Mobile learning: A handbook for educators and trainers (pp. 45­56). London: Routledge. Kukulska-Hulme, A., & Traxler, J. (2005). (Eds.). Mobile learning: A handbook for educators and trainers. London: Routledge.
Language Learning & Technology
108
Glenn Stockwell
Using Mobile Phones for Vocabulary Activities
Levy, M., & Kennedy, C. (2005). Learning Italian via mobile SMS. In A. Kukulska-Hulme & J. Traxler (Eds.), Mobile learning: A handbook for educators and trainers (pp. 76­83). London: Routledge. Ma, Q. (2004). Theoretical and design issues for a computer assisted vocabulary learning program: WUFUN. Proceedings of the Eleventh International CALL Conference (pp. 241­251). Antwerp: University of Antwerp. Prensky, M. (2001). Digital natives, digital immigrants. On the Horizon, 9(5), 1­6. Rosell-Aguilar, F. (2007). Top of the pods--In search of a podcasting "podagogy" for language learning. Computer Assisted Language Learning, 20(5), 471­492. Stockwell, G. (2007a). A review of technology choice for teaching language skills in the CALL literature. ReCALL, 19(2), 105­120. Stockwell, G. (2007b). Vocabulary on the move: Investigating an intelligent mobile phone-based vocabulary tutor. Computer Assisted Language Learning, 20(4), 365­383. Stockwell, G. (2008). Investigating learner preparedness for and usage patterns of mobile learning. ReCALL, 20(3), 253­270. Taylor, R.P., & Gitsaki, C. (2003). Teaching WELL in a computerless classroom. Computer Assisted Language Learning, 16(4), 275­294. Thornton, P., & Houser, C. (2002). M-learning: Learning in transit. In P. Lewis (Ed.), The changing face of CALL: A Japanese perspective (pp. 229­243). The Netherlands: Swets & Zeitlinger. Thornton, P., & Houser, C. (2005). Using mobile phones in English education in Japan. Journal of Computer Assisted Learning, 21(3), 217­228. Wang, S., & Higgins, M. (2006). Limitations of mobile phone learning. The JALT CALL Journal, 2(1), 3­ 14.
Language Learning & Technology
109
Glenn Stockwell
Using Mobile Phones for Vocabulary Activities
APPENDIX. Examples of question types for the vocabulary item "fossil." 1. Choose the appropriate word for an English sentence: Scientists know about the existence of dinosaurs from __________ . 1. floods 2. fossils 3. breakthroughs 4. technologies 2. Choose the appropriate English word for a Japanese meaning: 1. motion 2. generator 3. awareness 4. fossil 3. Choose the appropriate English word for an English definition: A remnant or trace of an organism of a past geologic age, such as a skeleton or leaf imprint, embedded and preserved in the earth's crust. 1. flood 2. fossil 3. emission 4. disorder 4. Write a word in English for an English definition: A remnant or trace of an organism of a past geologic age, such as a skeleton or leaf imprint, embedded and preserved in the earth's crust: __________ 5. Write the appropriate English word for an English sentence: Scientists know about the existence of dinosaurs from __________ .
Language Learning & Technology
110

G Stockwell

File: using-mobile-phones-for-vocabulary-activities-examining-the-effect.pdf
Title: Using Mobile Phones for Vocabulary Activities: Examining the Effect of Platform
Author: G Stockwell
Author: Glenn Stockwell
Subject: Language Learning & Technology
Keywords: Learner Autonomy;Language Learning Strategies;Vocabulary
Published: Mon Jun 14 17:00:01 2010
Pages: 16
File size: 0.18 Mb


, pages, 0 Mb

, pages, 0 Mb
Copyright © 2018 doc.uments.com