Students' adoptions and attitudes towards electronic placement tests: A UTAUT analysis, PJB Tan

Tags: American Journal of Computer Technology, Taiwan, proficiency standards, theories, Expectancy, technology acceptance model, Unified Theory of Acceptance and Use of Technology, Social Influence, independent variables, Venkatesh, placement tests, students', students, factors affecting, learning object, Advance Placement, placement test, traditional classroom tests, National Penghu University, Learning style, information technology, Theoretical extension, English Placement Test, Global Information Technology Report, Angela L. Rollins., questionnaire, administrative decisions, demographic information, Mark D. Shermis.
Content: American Journal of Computer Technology and Application Vol. 1, No. 1, February 2013, PP: 14 ­ 24, ISSN: 2327-2325 (Online) Available online www.ajcta.com RESEARCH ARTICLE Students' Adoptions and Attitudes towards Electronic placement tests: A UTAUT Analysis Paul Juinn Bing Tan National Penghu University (Department of Applied Foreign Languages, National Penghu University) Address: 300 Liu-Ho Rd. Magong , Penghu 880, Taiwan, R.O.C. E-mail: [email protected] Abstract The purpose of this study is to examine core factors affecting university students' attitudes to computerized placement tests (e-placement tests) and corresponding use. Placement tests are a crucial tool for educators to determine students' level of skills, ability, or knowledge. The tests allow students to show that they meet the proficiency standards of a program in order to gain admission and/or to gain additional credit, such as in Advance Placement testing. Therefore, placement tests improve both teaching efficiency and student motivation. Utilizing technology to implement such tests can provide many advantages in terms of distribution, collection, and evaluation. However, doing so could also provoke distress coming in unfamiliar forms, such as personal concerns. This study, based on the Unified Theory of Acceptance and Use of Technology model (UTAUT), analyzes the relationship of students' attitudes and willingness to use e-placement tests. It evaluates four factors affecting students' attitudes: performance expectancy, effort expectancy, social influence, and facilitating conditions. The results confirm that three of the four core constructs (factors) have a significantly positive influence on behavioral intention, which in turn have a positive correlation to actual use. Thus, e-placement tests are more appreciated by the students and more likely to be utilized by them. When students believe that e-placement tests: 1) will lead to academic gains, 2) are easy to use, and 3) are important to their peers or society, use of these tests will increase. In addition, administrators intending to utilize e-placement tests in their schools should incorporate such factors in the placement testing implementation. Copyright © AJCTA, all right reserved. Key words: e-placement tests, Unified Theory of Acceptance and Use of Technology model (UTAUT), higher education, computer based placement test, student attitude. 14
American Journal of Computer Technology and Application Vol. 1, No. 1, February 2013, PP: 14 ­ 24, ISSN: 2327-2325 (Online) Available online www.ajcta.com Introduction Placement tests typically evaluate knowledge and skill proficiency in a specific subject. This is in order to either determine student's level of skill or knowledge or to determine if the student is able to meet certain proficiency standards required by a specific program or organization, such as the Test Of English as Foreign Language (TOEFL). Occasionally, placement testing results in earning additional educational credits, such as in the Advanced Placement Program. Placement tests can be given by individual schools or by a certified external testing organization. These tests provide benefits to students in allowing the students to both accelerate their education or entry into a higher education program thereby improving their educational opportunities. Kay and Knaack (2008) developed the Learning Object Evaluation Scale for Students (LQES-S) to examine the impact, effectiveness, and usefulness of learning objects. People should use it to evaluate e-placement tests, that these are factors to consider when writing the placement tests ­ Their findings showed the LQES-S was a valid instrument which measured three criteria/factors: learning, quality, and engagement of the learning object. Most previous research on placement testing has focused on the content and grading of such tests (Ruth & Murphy, 1988). A study by Russell & Haney (1997) which examined multiple choice, open-ended, and direct writing assessments revealed that using computers significantly improve students' writing scores. Russell & Haney interpretation of the results indicated that accurate writing evaluation needs to consider available test-taking media as well as pure content. In the twenty-first century this issue will become increasingly pressing as computers become more prevalent on college campuses. (Harrington et al. 2000). It is tempting to use them for testing simply because they can be used for testing, but is it the best media to use. The issue of e-testing is not only related to education in U.S., but also in computer proliferate countries in Asia. It is the time for changes in learning, quality and engagement in Taiwan. Test administrators need to recognize that the testing media can affect both student performance and rater reactions. The purpose of this study is to re-examine the model of UTAUT (Unified Theory of Acceptance and Use of Technology) and how it was implemented in Taiwan, where this study was conducted. Taiwan has a well-developed infrastructure of information and communication technology. In terms of e-government, Taiwan was ranked first in the world by West (2005) while Waseda University (2006) ranked it seventh. For network readiness, it was ranked fifteenth by Schwab (2005). For promotions, digital business, and ICT services, the Economist Intelligent Unit (2009) ranked it sixteenth (Tao, 2008). Thus, Taiwan certainly has the technological capabilities to implement e-placement tests. Christensen (1998) investigated the correlation between positive attitudes toward IT and effective technology use. He concluded that additional verification of this relationship could significantly impact the way individuals are trained to use technology. Thus, higher education can not avoid the changes .Most college campuses in Taiwan have suitable availability of computer resources and most students have access to computers both on and off campus. Furthermore, a majority of students are familiar with word-processing, web-browsing, and a wide variety of other software. However, school administrators still need to decide if and in what form to implement placement tests. This study examines students' attitudes towards electronic placement testing or e-placement tests. The objectives of this study are: 1. To examine the attitudes of Taiwanese college students towards e-placement tests 2. To analyze the relationship between these attitudes and actual behavior 15
American Journal of Computer Technology and Application Vol. 1, No. 1, February 2013, PP: 14 ­ 24, ISSN: 2327-2325 (Online) Available online www.ajcta.com 3. To evaluate the applicability of UTAUT to these issues. This study aims to identify factors which affect students' attitudes to technology in placement testing. It also provides an empirical basis for administrative decisions about placement testing. Various theories and UTAUT were reviewed in the construction of this measure. The results of this study are essential to administrators trying to make such decisions. Prior research Venkatesh et al. (2003) hypothesized and empirically supported UTAUT, a model which includes "three direct determinants of intention to use (performance expectancy, effort expectancy, and social influence) and two direct determinants of usage behavior (intention and facilitating condition)". To date, empirical testing of this new UTAUT model is limited (Youngberg et al. 2009). Teaching and learning is just like one coin with two sides. Examining the acceptance of e-learning among university students is necessary using models such as the newly developed E-learning Acceptance Measure (ElAM). Using ElAM to collect information on students' reactions prior to the implementation of any lesson via e-learning or e-testing is vital . A better understanding of students' acceptance of e-learning would increase our understanding of students' computer-related behaviors. Such information would assist teachers in their decisions regarding resources allocation, Instructional Design, and teaching strategies. This knowledge has wide-reaching implications for educational stakeholders (Teo, 2010). Teo's study provided evidence for the factorial structure of the ElAM and it may be useful for educational researchers. UTAUT is a hierarchal, latent variable statistical model (aka Structural Equation Modeling, i.e. SEM) postulating four core constructs which affect behavioral intention which then affects use behavior. In such latent variable models, subjects respond to a variety of related stimuli. This stimuli act as the overt variables, which the UTAUT model then combines into the latent variables, the four core (independent) constructs, and their two dependent variables. The four core constructs, which are the focus of this study, as defined by Venkatesh et al. (2003) are: (Figure 1) Performance Expectancy: "The degree to which an individual believe that using the system will help him or her attain gains in job performance." Effort Expectancy: "The degree of ease associated with the use of the system." Social Influence: "The degree to which an individual perceives that important others believe he or she should use the new system." Facilitating Conditions: "The degree to which an individual believes that an organizational and technical infrastructure exists to support use of the system." 16
American Journal of Computer Technology and Application Vol. 1, No. 1, February 2013, PP: 14 ­ 24, ISSN: 2327-2325 (Online) Available online www.ajcta.com
Figure 1: The UTAUT model for E-placement test adoption for college students in Taiwan.
Performance
Expectancy
H1
Effort H2 Expectancy H3 Social Influence H4 Facilitating Conditions
Behavioral H5
Use
Intention
Behavior
UTAUT was formulated to integrate and unify the various fragmented theories that previously existed on individual's acceptance of new technologies. To this end, previous models were analyzed and compared. Then conceptual similarities of these models were combined to construct UTAUT. The UTAUT model incorporates and organizes eight previous theories of individual attitudes and behavior (Venkatesh et al., 2003): Theory of Reasoned Action (TRA) Technology Acceptance Model (TAM) Motivational Model (MM)- Theory of Planned Behavior (TPB) Model Combining the Technology Acceptance Model and the Theory of Planned Behavior (C-TAM-TPB) Model of PC Utilization (MPCU) Innovation Diffusion Theory (IDT) Social Cognitive Theory (SCT) Table 1 shows how the core constructs of UTAUT include the constructs of the other theories and provides the sources for those theories.
17
American Journal of Computer Technology and Application Vol. 1, No. 1, February 2013, PP: 14 ­ 24, ISSN: 2327-2325 (Online) Available online www.ajcta.com
Table 1:The constructs mentioned in IDT, TRA, TAM, TPB, C-TAM-TPB, MPCU MM and SCT
Core Constructs
Constructs and Theories
References
Performance Expectancy Perceived Usefulness (TAM/TAM2 and (Davis, 1989; Davis et al., 1989)
C-TAM-TPB)
Extrinsic Motivation (MM)
(Davis et al., 1992)
Job-fit (MPCU)
(Thompson et al., 1991
Relative Advantage (IDT)
(Moore and Benbasat, 1991)
Outcome Expectations (SCT)
(Compeau and Higgins, 1995)
Effort Expectancy
Perceived Ease of Use (TAM/TAM2)
(Davis, 1989; Davis et al., 1989)
Complexity (MPCU)
(Thompson et al., 1991)
Ease of Use (IDT)
(Moore and Benbasat, 1991)
Social Influence
Subjective Norm (TRA, TAM2, TPB and (Fishbein and Azjen, 1975; Ajzen,
C-TAM-TPB)
1991; Davis et al., 1989; Taylor and
Todd, 1995)
Social Factors (MPCU)
(Thompson et al., 1991)
Image (IDT)
(Moore and Benbasat, 1991)
Facilitating Conditions Perceived Behavioral Control (TPB and (Ajzen, 1991; Taylor and Todd, 1995)
C-TAM-TPB)
Facilitating Conditions (MPCU)
(Thompson et al., 1991)
Compatibility (IDT)
(Moore and Benbasat, 1991)
DeLone and McLean's D&M IS Success Model (2003), and Schwarz' Core-Outcome and Acceptance-Outcome Model (2004) research may inform the IS community of the social, cognitive, cultural, and contextual antecedents for a future parsimonious "Unified" (Venkatesh et al., 2003) technology acceptance model. Thus, an "amalgamation" (Glass et al., 2004) of models may better inform and direct future IS researchers by facilitating the linkage of user acceptance to individual and organizational outcomes.
Table 2:Comparisons between traditional classroom tests and E-tests
Characteristic Time and place limits
Classroom Time and location dependent
E-tests Anytime-anywhere
Physical ­ limited scale
Unlimited
Teaching content
Teacher-centered
Student-centered
Personalization
Push approach
Pull approach
One learning path ­ lowest Learning pace and path
common denominator
determined by user
Learning style
Rigid
Flexible
18
American Journal of Computer Technology and Application Vol. 1, No. 1, February 2013, PP: 14 ­ 24, ISSN: 2327-2325 (Online) Available online www.ajcta.com Given both that e-placement tests are a rapidly emerging technology and that the four core constructs seem relevant to academic issues, it is important to verify the applicability of the UTAUT model to this topic. Once verified, the model can then also lend insight into the determining factors of student perceptions of these tests and the relevant factors for administrators considering their implementation.
Method Data for this study was collected using a questionnaire survey. Survey questions were adopted from Venkatesh and associates (2003). The survey questions are listed in the appendix. The surveys were pre-tested on four students and evaluated by two independent professors, then modified according to their comments. The final survey demonstrated high reliability. The survey collected demographic information and data on the overt variables for the four constructs in the model using a 5-point Likert scale (1=strongly disagree, 5=strongly agree).The questionnaire was accessible via MY3Q questionnaire website (http://www.my3q.com) and sent to a random sample of Taiwanese college students. A total of 230 students were surveyed using a standard questionnaire. There were only 196 that were valid. The sample size, 196, is close to the sample size of prior Technology Acceptance Model (TAM) research (Venkatesh and Davis, 2000; Venkatesh et al., 2003). In addition, the sample size was more than five times the number of variables being analyzed in structural equation model (Hatcher, 1944). Table 3 presents demographic information of the respondents. Of note, more females than males participated in the study. SPSS software was used to examine the data and perform reliability, correlation, and regression analyses.
Table 3: Demographics of respondents
Gender
Valid data of Participants (n=196)
Male
83
Female
113
Percentage (%) 42.35% 57.65%
Hypotheses Using the four core constructs of the UTAUT model, five hypotheses were made: H1: Performance expectancy positively affects subjects' intention to take e-placement tests. H2: Effort expectancy positively affects subjects' intention to take e-placement tests. H3: Social influence positively affects subjects' intention to take e-placement tests. H4: Facilitating conditions positively affect actual use behavior. H5: Subjects' intentions to take e-placement tests positively affect actual use behavior. Moderating effects of age, experience, and volunteering were not included as independent variables in our study. Given that the subjects were all freshman Taiwanese university students, there would be little variation in these factors. . Results Reliability analysis indicated that the overt variables outcomes were consistent to the hypothesized results. Table 4 shows that Cronbach's Alpha for all constructs ranged from 0.76 to 0.95, thus exceeding the recommended level of 0.70. 19
American Journal of Computer Technology and Application Vol. 1, No. 1, February 2013, PP: 14 ­ 24, ISSN: 2327-2325 (Online) Available online www.ajcta.com
Table 4: Reliability of research variable Performance Expectancy Effort Expectancy Social Influence Facilitating Conditions Behavioral Intention Use behavior Cronbach's Alpha
Cronbach's Alpha 0.780 0.748 0.759 0.818 0.791 0.575 0.904
Convergent validity and discriminated validity were assessed using Pearson correlation Analysis. Guidelines suggested that factor loadings greater than 0.50 (Hair et al., 1998) or a stricter criterion greater than 0.70 (Fornell, 1982). All of the factor results in this research study were higher than 0.50 and many were above 0.70. Every item was loaded significantly (p<0.01 in all cases) on its constructs. Therefore, all criteria in the results have acceptable reliability and convergent validity. Correlation results are showed in Table 5.
Table 5: Correlation of adoption factors
PE
EE
SI
FC
BI
UB
PE
1.000
EE
.701**
1.000
SI
.664**
.626**
1.000
FC
.671**
.706**
.703**
1.000
BI
.695**
.642**
.655**
.766**
1.000
UB
.487**
.445**
.519**
.512**
.528**
1.000
**. Correlation is significant at the 0.01 level (2-tailed).
Regression analysis indicated the influence of performance expectancy, effort expectancy, and social influence on
behavioral intention. T-tests indicate that all the coefficients were positively significant. The results are summarized in
the table below.
Table 6: Regression of adoption factors on intention to use

t-value
PE
.327
3.416***
EE
.158
2.126*
SI
.225
2.963**

.756
Adjusted RІ
.571
*p<0.05 **p<0.01 ***p<0.001
20
American Journal of Computer Technology and Application Vol. 1, No. 1, February 2013, PP: 14 ­ 24, ISSN: 2327-2325 (Online) Available online www.ajcta.com Regression analysis also demonstrated the positive significance of behavioral intention in predicting use behavior. However, facilitating conditions did not show statistical significance. These results are shown in the table below.
Table 7: Regression of intention to use behavior

FC
.266
BI
.323

.554
Adjusted RІ
.307
*p<0.05 **p<0.01 ***p<0.001
t-value 1.936 2.449*
To summarize, the results of our study for each hypothesis were: H1: Performance expectancy positively affects subjects' intention to take e-placement tests. ­SIGNIFICANT H2: Effort expectancy positively affects subjects' intention to take e-placement tests. ­SIGNIFICANT H3: Social influence positively affects subjects' intention to take e-placement tests. ­SIGNIFICANT H4: Facilitating conditions positively affect actual use behavior. -NOT SIGNIFICANT H5: Subjects' intentions to take e-placement tests positively affect actual use behavior. ­SIGNIFICANT
Discussion The results support the applicability of the UTAUT model to the analysis of students' attitudes and behavior regarding e-placement tests. Parties wishing to implement e-tests need to consider students' awareness of the advantages, convenience, and social importance of e-placement tests. When students appreciate the academic advantages of e-placement tests, they are more likely to want to take them. When such tests appear easy to use and access, students will be more interested in them. When those around them, such as parents and friends, also appreciate the value of these tests, students' attitudes towards the tests will be better. When students have a more positive attitude to such tests, they will be more likely to actually use them. However, our results did not sustain the significance of facilitating conditions as predicted by UTAUT. Possibly, such conditions are less significant in an academic setting where student activities are driven more by performance demands and ability evaluations and less subject to conditions facilitating convenience or comfort. Overall, the results support that the UTAUT model provides a good tool for understanding and applying technology in an academic setting.
Limitations and suggestions This study was carried out on a small sample of freshmen Taiwanese students. While Taiwan is extremely well-developed technologically, the results of this study should be confirmed with a larger sample size, variety of ages, and across a variety of cultures. It is also important to establish whether technological development of a given culture affects the results outcome As the results did not confirm the importance of facilitating conditions to student behavior perhaps the survey questions should be rephrased to determine what aspect of facilitating conditions are pertinent in an academic setting and/or to student behavior. Further research should examine what methods are successful in improving the 21
American Journal of Computer Technology and Application Vol. 1, No. 1, February 2013, PP: 14 ­ 24, ISSN: 2327-2325 (Online) Available online www.ajcta.com independent variables of this model. In other words, how can test implementers improve students' performance, effort expectancies, and increase awareness of the tests' social influence. As we have seen, successful implementation of e-placement testing requires positive attitudes for the above mentioned variables. Future research should determine the best ways to improve these attitudes. Appendix 1: Survey Questions Performance Expectancy (PE) PE1: I believe that taking e-placement tests could improve my academic results. PE2: I think that taking e-placement tests could improve my academic performance. PE3: E-placement tests could accelerate my academic performance. PE4: E-placement tests could be beneficial to my Learning activities. Effort Expectancy (EE) EE1: E-placement tests would probably be easy to use. EE2: E-placement tests should be easy for me. EE3: E-placement tests should facilitate test completion EE4: Using e-placement tests is understandable. Social Influence (SI) SI1: People who are important to me think that I should use e-placement tests. SI2: People who affect my learning think that I should use e-placement tests. SI3: I expect to use e-placement tests because people around me do. SI4: Not using e-placement tests is falling behind others. Facilitating conditions (FC) FC1: I sense the need to use e-placement tests to advance my education. FC2: I have the knowledge to take advantage of e-placement tests. FC3: E-placement tests are suitable to the way I like to do things. Behavioral Intention (BI) BI1: I intend to use e-placement tests in the future. BI2: I would use e-placement tests to assess my abilities. BI3: I plan to take an e-placement test within the next 3 months. Use Behavior (UB) UB1: I have used an e-placement test to identify my level in school. UB2: I have used an e-placement test to assess my abilities. UB3: I have used an e-placement test to prove my abilities. 22
American Journal of Computer Technology and Application Vol. 1, No. 1, February 2013, PP: 14 ­ 24, ISSN: 2327-2325 (Online) Available online www.ajcta.com References [1]Ajzen, I. (1991) The theory of planned behavior. Organizational Behavior and Human Decision Processes, 50, 179-211. [2]Christensen, R. (1998). Effect of technology integration education on the attitudes of teachers and their students. Unpublished Doctoral dissertation, University of North Texas, Denton. [3]Compeau, D. R. & Higgins, C. A. (1995) computer self-efficacy: Development of a measure and initial test. MIS Quarterly, 19, 189-211. [4]Davis, F. D. (1989) Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly 13, 319-340. [5]Davis, F.D., R.P. Bagozzi and P.R. Warshaw,. 1989.. User acceptance of computer technology acomparison of two theoretical models. Management Science,35( 8). 982 -1003. [6]Davis, F.D., R.P. Bagozzi and P.R. Warshaw, 1992. Extrinsic and intrinsic motivation to use computers in the workplace. Journal of Applied Social Psychology, 22, 1111-1132. [7]DeLone, W. H., & McLean, E. R. (2003). The Delone and Mclean model of Information Systems success: A ten- year update. Journal of management information System, 19(4), 9- 30 [8]Economist Intelligent Unit, 2009. The 2009 e-readiness ranking. http://graphics.eiu.com/pdf/E-readiness%20rankings.pdf [9]Elaine Youngberg, David Olsen, Karina Hauser, Determinants of professionally autonomous end user acceptance in an Enterprise Resource Planning system environment, International Journal of Information Management 29 (2009) 138- 144 [10]Fishbein, M. & Ajzen, I. (1975) Belief, attitude, intention, and behavior: An introduction to theory and research. Massachusetts, Reading, MA. [11]Fornell, C. A. (1982). Second generation of multivariate analysis. Methods (Vol. 1). NY: Praeger Special Studies. [12]Glass, R. L., Ramesh, V., & Vessey, I. (2004). An analysis of research in computing disciplines, Communications of the ACM, 47(6), 89- 94 [13]Hair, J. F., Anderson, R. E., Tatham, R. L., & Black, W. C. (1998). Multivariate data analysis (5th ed.). NJ: Prentice-Hall Inc. [14]Hatcher L. A step-by-step approach to using the SAS(R) system for factor analysis and structural equation modeling. Cary, NC: SAS Institute; 1994. [15]Kay, R. H., & Knaack, L. (2008). A multi- component model for assessing learning objects: The Learning Object Evaluation Metric (LOEM). Australasian Journal of Educational Technology, 24, 574- 591 [16]Michael Russell & Walt Haney (1997). Testing writing on computers: An experiment comparing student performance on tests conducted via computer and via paper-and-pencil. Education Policy Analysis Archives, 5.3 Available: http://epaa.asu.edu/epaa/v5n3.html [17]Moore, G. C. & Benbasat, I. (1991) Development of an instrument to measure the perceptions of adopting an information technology innovation. Information Systems Research, 2 , 192-222. [18]Ruth, Leo, & Murphy, Sandra. (1988). Designing writing tasks for the assessment of writing. Norwood, NJ: 23
American Journal of Computer Technology and Application Vol. 1, No. 1, February 2013, PP: 14 ­ 24, ISSN: 2327-2325 (Online) Available online www.ajcta.com
Ablex.Schwarz A. H. (2004). Defining information technology acceptance: A human- centered, management-
oriented perspective [19]Schwab, K. (2005).
Global
Information
Technology
Report,
World
Economic
Forum.
Available:+Competitiveness+Report>.
[20]Susanmare Harrington., Mark D. Shermis., Angela L. Rollins., The Influence of Word Processing on English
Placement Test Results, Computer and Composition 17, 197- 210 (2000) [21]Tao, Y.H., 2008. Typology of college student perception on institutional E-learning issues: An extension study of a teacher's typology in Taiwan. Comput. Educ., 50: 1495-1508.
[22]Taylor, S. & Todd, P. (1995) Understanding information technology usage: A test of competing models. Information Systems Research, 6, 144-176. [23]Teo, T., Development and validation of the E- learning Acceptance Measure (ElAM), Internet and Higher Education
(2010), doi:10.1016/j.iheduc.2010.02.001 [24]Thompson, R. L., Higgins, C. A. & Howell, J. M. (1991) Personal computing: Toward a conceptual model of
utilization. MIS Quarterly, 15, 124-143. [25]Venkatesh V, Davis FD. (2000) A Theoretical extension of the technology acceptance model: four longitudinal field
studies. Manag Sci;45(2):186-204 [26]Venkatesh, V., Morris, M. G., Davis, G. B. & Davis, F. D. (2003) User acceptance of information technology: Toward a
unified view. MIS Quarterly, 27, 425-478. [27]Waseda University of e-Government (2006). World e-government ranking. [28]West, D., (2005), Global e-government 2005, Brown University. .
24

PJB Tan

File: students-adoptions-and-attitudes-towards-electronic-placement.pdf
Title: Microsoft Word - This is my final vevision(A)
Author: PJB Tan
Author: Mohammad
Published: Fri Apr 5 10:52:04 2013
Pages: 11
File size: 0.14 Mb


, pages, 0 Mb

Chelan/Douglas, 1 pages, 0.41 Mb

UNDER NEW OWNERSHIP, 1 pages, 0.98 Mb
Copyright © 2018 doc.uments.com