The online recruitment system orsee 2.0-a guide for the organization of experiments in economics

Tags: experiment, subject pool, ORSEE, subjects, the experiment, registration, experimenters, participant, experimenter, sessions, session, laboratory experiments, experiments, the session, Recruitment System, participation data, Recruitment Process, installed, Laboratory Experiment, Financial support, economic experiment, Economic Experiments, invitations, system registration, self-description, self description
Content: The Online Recruitment System ORSEE 2.0 - A Guide for the Organization of Experiments in Economics Ben Greiner Version 2.0.2, November 13, 2004 Abstract We discuss several issues regarding the organization of economic laboratory experiments such as subject pool, recruitment, scheduling, and show how we solved them with the help of the Online Recruitment System for Economic Experiments (ORSEE) version 2.0. With this integrated software experimenters have a free, convenient, and very powerful tool to organize their experiments and sessions in a standardized way. Key features are: PHP/MySQL application, multiple language/ laboratory/ subject pool/ experimenters/ experiment types/ experiment classes support, attribute query selection, random recruitment, experiment calendar, automated reputation system, automated invitation and rule based reminder mailing, subjects manage their own account, overview about registration state, user rights management, pdf output and mailing, complete logging and statistics, and customizable layout. A test system has been installed in order to visually support the reader while reading the manual (www.orsee.org). I would like to thank Philipp Euskirchen, Jens GroЯer, Imke Jungjohann, Nikoletta Kiss, Carsten Schmidt, and participants at the ESA European Meeting 2003 in Erfurt and the ESA World Meeting 2004 in Amsterdam for valuable comments and suggestions on this paper and software, and the Strategic Interaction Group at the Max Planck Institute in Jena, Germany for testing the very first version of the system. Financial support from the Max Planck Society is gratefully acknowledged. University of Cologne, Department of Economics, Albertus-Magnus-Platz, D-50923 KЁoln, Germany. Tel.: +49/221/470-6116, Fax: +49/221/470-5086, E-mail: [email protected]
Contents
1 Introduction
5
2 Methodological Issues
6
2.1 Subject Pool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
2.2 Recruitment Process . . . . . . . . . . . . . . . . . . . . . . . . . 7
2.3 No-shows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
2.4 Multiple Experimenters and Laboratory Booking . . . . . . . . . 9
2.5 Moral Issues and Privacy . . . . . . . . . . . . . . . . . . . . . . 9
2.6 Administration and Organization . . . . . . . . . . . . . . . . . . 10
3 System Overview
10
3.1 Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
3.1.1 General Features . . . . . . . . . . . . . . . . . . . . . . . 10
3.1.2 technical features . . . . . . . . . . . . . . . . . . . . . . 12
3.2 License . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
3.3 Basic System Design . . . . . . . . . . . . . . . . . . . . . . . . . 14
3.4 Conducting a Laboratory Experiment . . . . . . . . . . . . . . . 14
4 The subject area
15
4.1 Rules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
4.2 privacy policy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
4.3 General Registration . . . . . . . . . . . . . . . . . . . . . . . . . 16
4.4 Invitation and Registration for Experiments . . . . . . . . . . . . 17
4.5 Calendar . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
4.6 FAQ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
5 The Experimenter Area
21
5.1 Laboratory Experiments . . . . . . . . . . . . . . . . . . . . . . . 21
5.1.1 Registration of the Experiment . . . . . . . . . . . . . . . 21
5.1.2 Registration of Sessions . . . . . . . . . . . . . . . . . . . 24
5.1.3 Assigning Participants . . . . . . . . . . . . . . . . . . . . 26
5.1.4 Inviting Participants . . . . . . . . . . . . . . . . . . . . . 28
5.1.5 Control of the Registration Process . . . . . . . . . . . . . 29
5.1.6 Finishing a Session . . . . . . . . . . . . . . . . . . . . . . 30
5.1.7 Ending the Experiment . . . . . . . . . . . . . . . . . . . 31
5.2 Maintaining the subject pool . . . . . . . . . . . . . . . . . . . . 32
5.2.1 Temporarily Registered Participants . . . . . . . . . . . . 32
5.2.2 Adding New Participants . . . . . . . . . . . . . . . . . . 32
2
5.2.3 Edit Participants . . . . . . . . . . . . . . . . . . . . . . . 32 5.2.4 Delete Participants . . . . . . . . . . . . . . . . . . . . . . 33 5.2.5 Edit Deleted Participants . . . . . . . . . . . . . . . . . . 33 5.2.6 Automated Participant Exclusion . . . . . . . . . . . . . . 33 5.3 Calendar . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 5.4 Downloads . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34 5.5 Logs and Statistics . . . . . . . . . . . . . . . . . . . . . . . . . . 35 5.5.1 Log Files . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 5.5.2 Statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 5.6 E-mail Processing . . . . . . . . . . . . . . . . . . . . . . . . . . . 36 5.7 Languages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36 5.7.1 Edit Language Symbols . . . . . . . . . . . . . . . . . . . 37 5.7.2 Add Language . . . . . . . . . . . . . . . . . . . . . . . . 38 5.7.3 Delete Language . . . . . . . . . . . . . . . . . . . . . . . 38 5.7.4 Edit Basic Data . . . . . . . . . . . . . . . . . . . . . . . 38 5.7.5 Export Language File . . . . . . . . . . . . . . . . . . . . 38 5.7.6 Import Language File . . . . . . . . . . . . . . . . . . . . 38 5.8 User Rights Management . . . . . . . . . . . . . . . . . . . . . . 39 5.9 Reputation System . . . . . . . . . . . . . . . . . . . . . . . . . . 40 5.10 Regular Tasks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 5.11 Other Options and Settings . . . . . . . . . . . . . . . . . . . . . 42 5.11.1 My Profile . . . . . . . . . . . . . . . . . . . . . . . . . . . 42 5.11.2 User Management . . . . . . . . . . . . . . . . . . . . . . 43 5.11.3 General Settings . . . . . . . . . . . . . . . . . . . . . . . 43 5.11.4 Default Values . . . . . . . . . . . . . . . . . . . . . . . . 46 5.11.5 Default Mails . . . . . . . . . . . . . . . . . . . . . . . . . 49 5.11.6 Default Texts . . . . . . . . . . . . . . . . . . . . . . . . . 51 5.11.7 Experiment Classes . . . . . . . . . . . . . . . . . . . . . 52 5.11.8 Experiment Types . . . . . . . . . . . . . . . . . . . . . . 52 5.11.9 Laboratories . . . . . . . . . . . . . . . . . . . . . . . . . 53 5.11.10 Professions and Fields of Studies . . . . . . . . . . . . . . 53 5.11.11 Sub-Subjectpools . . . . . . . . . . . . . . . . . . . . . . . 54 5.11.12 FAQs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55 5.11.13 Help . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55 5.11.14 Public Content . . . . . . . . . . . . . . . . . . . . . . . . 56
6 Installation and command line configuration
57
6.1 Prerequisites . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
6.2 Download and Installation . . . . . . . . . . . . . . . . . . . . . . 57
3
6.3 The Main Configuration File . . . . . . . . . . . . . . . . . . . . 58 6.4 Regular Tasks - cron . . . . . . . . . . . . . . . . . . . . . . . . . 59 6.5 Web Server Statistics - webalizer . . . . . . . . . . . . . . . . . . 60 6.6 Styles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61 6.6.1 HTML Framework . . . . . . . . . . . . . . . . . . . . . . 61 6.6.2 Colors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
7 Tips and Tricks
64
7.1 Two definite subgroups in one session . . . . . . . . . . . . . . . 64
7.2 Handle newcomers on the lab's door . . . . . . . . . . . . . . . . 64
4
1 Introduction Laboratory experimentation has been a growing field in economics for the last decades.1 But the more experiments have been conducted in economics, the more the issue of an appropriate methodology and organization has been arisen. At the moment, there are the following items which are commonly agreed to be symptomatic for economic experiments (compared to human subject experiments in psychology and other Social Sciences)2: · Subjects are payed for their participation. · Payment should reflect subjects' performance in the experiment, i.e. the strategy space should translate to the payoff space.3 · Subjects should be volunteers motivated by the experimenters' payment. · Subjects should not be deceived. However, there is a wide variety in the procedures of maintaining a subject pool and organizing experiments. This paper deals with the topic of the organization of economic experiments, and how this work can be supported and standardized by the use of the Online Recruitment System for Economic Experiments (ORSEE). The specific aims of ORSEE are: · to simplify the organization of economic laboratory experiments · to standardize the procedures of experiment organization · to depersonalize the experimenter-subject interaction · to allow the conduction of simple internet experiments · to provide information and statistics about the subject pool4 1For an introduction into experimental methodology and an overview on history and topics of experimental economics see Davis & Holt (1993), Friedman & Sunder (1994), and Kagel & Roth (1995). 2For a discussion of differences between economics and psychology and in economic and psychological methodology see Camerer (1996), Hertwig & Ortmann (2001) (with Open Peer Commentary), Rabin (1998), Rabin (2002), Zwick, Erev & Budescu (1999). 3See Harrison (1989). 4Out of the 11 points which should be included in the experimental procedures section of every experimental paper sent to Econometrica (see Palfrey & Porter (1991)), ORSEE directly provides information about 4 of them: subject pool and recruiting procedures, number of subjects per session, experience of subjects, and the when and where of the experiment. 5
In the following, we point out the principles which guided the design of the system, based on experimental methodological reasoning. However, note that this work is not about how to conduct an economic experiment. ORSEE has been implemented and is online in Jena, Germany since March 2003. Currently, it is used at five institutions.5 In order to support the reader while going through the manual a test system has been installed.6 The following section discusses methodological issues regarding the organization of experiments and the recruitment of subjects. Section 3 gives a brief overview of the recruitment system and a short example on how laboratory experiments can be conducted in ORSEE. Section 4 and 5 discuss the Subject and Experimenter Area of ORSEE in full detail. The last sections contain a manual for installation and configuration of the system as well as some practical tips and tricks. Before starting, some terms used throughout this manual should be defined: A 'session' is defined as processing an experiment at a certain time at a certain location. An 'experimenter' is a person who conducts and/or administrates an experiment. A 'subject' is a person who is recruited to participate in an experiment. 2 Methodological Issues 2.1 Subject Pool The subject pool is the most precious resource for experimental economists. Standard economic theory suggests that there are no differences in economic decision making between old and young or female and male people. But as standard theory has proven to make wrong predictions in many economic situations, this is not the final truth, as well.7 However, most researchers in experimental economics are using undergraduate student subjects in their studies, because they are cheap due to low opportunity costs, are easy to recruit, have mostly little knowledge in the topic 5Humboldt University Berlin, University of Bonn, University of Erfurt, University of Cologne, and Max Planck Institute for Research into economic systems in Jena. 6http://www.orsee.org 7Ball & Cech (1996) provide an overview of subject pool effects on results in experimental economics. They found no evidence for subject pool differences when using student populations from different universities, no to little evidence for students vs. professionals as experimental subjects, mixed evidence when testing for effects of gender or psychological attributes, but evidence for culture, experimental experience, and economic education. They conclude that subject pool differences in economic situations depend on the design and nature of the economic setting studied, ranging from anonymous markets with many agents to distribution games where fairness considerations might be involved. 6
being tested experimentally, and show steep learning curves. One conclusion drawn from the education effect seems to be that most experimenters try to avoid using doctoral students as subjects. The fact that registration for experiments and sessions is possible over email and the internet only restricts the subject pool to people who have access to these technologies. Since most experimenters use students as subjects, and in most countries universities provide e-mail and internet to their students, we consider this as a minor drawback of the system. To avoid that always the same subjects with often and immediate access to their e-mail register for experiments ORSEE provides the important feature to invite a random draw from the subject pool, or to exclude subjects who have participated at certain other experiments. In addition, ORSEE collects important data when subjects register with the database. Subjects' history of participation at experiments is saved. The experimenter can then invite subjects according to specific characteristics, i.e. field of study or gender. 2.2 Recruitment Process ORSEE is designed to organize experiments with paid volunteers.8 However, the payment is still in the experimenters control. As well, ORSEE does not allow for classroom experiments (where subjects don't register before the experiment) and does not collect information about subjects' participation at certain university courses.9 ORSEE was also designed to keep personal experimenter-subject interaction in the organization process as minimal as possible by the use of a depersonal- 8Psychologists, who care more about the representativeness of a subject pool than experimental economists, often use undergraduate subject pools (USP's), which require all students in an introductory psychology course to participate. Some studies found that volunteers are not representative for the student overall population, which is explained by the selfselection of volunteers (Dixon 1978, Jackson, Procidano & Cohen 1989, Jung 1969, Rosenthal & Rosnow 1975). Eckel & Grossman (2000) tested for differences in behavior between volunteers (who turn up at a particular time and location, motivated by promises of financial reward) and pseudo-volunteers (recruited from a class in which the experiment was to be conducted immediately, but where students could leave if they wanted) in dictator experiments. They found that pseudo-volunteers were more generous, and that for them religious and altruistic preferences had a greater impact on gift-giving. However, other studies suggest that paying the volunteering subjects might increase their representativeness (Rush, Phillips & Panek 1978, Wagner & Schubert 1976). Recently, Camerer & Hogarth (1999) surveyed 74 studies comparing experiments with and without monetary incentives. They concluded that providing monetary incentives matters for tasks where performance responds to effort and effort responds to incentives. For a detailed discussion of the necessity of paying experimental subjects see Binmore (1994), Harrison (1989), and Smith & Walker (1993). 9For a recruitment system allowing for this see for instance 'E-Recruit' (http://erecruit.sourceforge.net). 7
ized database system, generic e-mails and web sites, and institutional e-mail addresses for communication.10 Addresses and e-mail templates can be edited in ORSEE. Psychological studies show that a description and even the name of an experiment can have an impact on subjects' voluntary self selection to experiments (Jackson et al. 1989, Senn & Desmarais 2001, Saunders, Fisher, Hewitt & Clayton 1985, Silverman & Margulis 1973). To avoid this bias in ORSEE, the type of experiments conducted at an institution should be only described once (at a publicly accessible 'Experiment Rules' page), and invitation e-mails are based on generic templates. Regarding the experiment name, ORSEE distinguishes between two types: the internal, meaningful name to identify the experiment in the experimenter area (i.e. 'Five Person Circular Ultimatum Game'), and a public name used for communication with subjects via e-mails and website (i.e. 'Exp416'). Both can be chosen freely. 2.3 No-shows Every experimenter has to deal with the problem of no-shows or late-shows. To avoid this, most institutions use the mean of over-recruitment, i.e. they invite more subjects than actually needed for the specific experimental session. The number of reserve participants is determined by experience. ORSEE uses the same approach. The participant statistics section provides a monthly statistic of no-shows. For every session the experimenter can determine the number of reserve participants to be invited. All subjects are treated equally, there is no distinction between subjects invited for the session and subjects invited as reserve. At the laboratory's door, we use the simple rule of first come, first serve. The subjects coming last, but still in time, as well as the participating subjects receive a show-up fee intended to cover their opportunity costs to show up. The amount of this fee may vary between countries. After each session the experimenter has to fill in the show-up and participation data. To motivate subjects to show-up, a reputation system is implemented in ORSEE, which provides a statistic of no-shows to subjects and experimenters. This should be accompanied by a rule, which excludes subjects who did not show up a certain number of times from further participation. This rule might be set by the laboratory staff and announced at the public 'Rules page'.11 10To our knowledge, there are only two studies testing for experimenter-subjects effects in the recruitment procedure. Coutts & Schneider (1975) find no overall effects of experimenter's sex and subject's sex, but that male volunteers were more likely to show up when recruited by a male experimenter, while Senn & Desmarais (2001) note that both sexes are more likely to sign up for a sex research study when recruited by a male. 11In Jena we use the rule '3 no-shows and you are out (of the subject-pool)'. 8
2.4 Multiple Experimenters and Laboratory Booking Most of the experimental studies in economics are joint papers between two or more authors, and laboratories are regularly used by more than one experimenter. To comply with these facts, ORSEE allows for multiple experimenters. Each experimenter receives his own login, password, and certain privileges (i.e. sending e-mails, editing experiments). The experiments, she is involved in, are listed at a special page. The dates and times of sessions have to be coordinated to prevent clashes. After an experimenter has created the session, it immediately appears in ORSEE's internal calendar. If the time of the session clashes with another, the experimenter gets a feedback. The internal experiment calendar is sent out to the experimenter pool once a week by e-mail. Thus, the rule for allocating laboratory space in ORSEE is 'first come, first serve'. However, by maintaining a second, system-external calendar publicly accessible by experimenters one may use another rule. To facilitate collaboration and learning between experimenters, ORSEE allows to share knowledge via linking of papers and uploads of instructions and programs. In this way, an experimenter can profit from the experience of others in the team. 2.5 Moral Issues and Privacy In Europe there are many laws dealing with the privacy of personal data, and in the U.S. university human subject committees regulate the use of students in experiments. Unlike psychologists, economic experimentalists have agreed to not deceive their subjects in any way (see Hertwig & Ortmann (2001)). For the organization of experiments this means that subjects should be invited without false promises. The public part of ORSEE includes a 'Rules' page where the laboratory should announce how experiments are conducted at their institution. In our view, no deception means also that subjects should know about the data collected in the recruitment database and in the experiment and what it is used for. By using individualized URLs ORSEE ensures that subjectsubject anonymity in the participant database is completely in participants' control. For laboratory experiments, ORSEE is exclusively a recruitment system, meaning that there is no direct connection between the data collected during the registration process and the decision data recorded in the experiment. At the end, the experimenter herself is in charge of subjects' privacy. To announce the privacy policy of the institution, ORSEE provides a publicly ac- 9
cessible 'Privacy Statement' page. In the general registration procedure subjects have to accept both the experiment rules and the privacy policy.12 2.6 Administration and Organization Regarding the organization of laboratory experiments, the objection should be to control the process and the subject pool data as much as possible. However, when privacy or other moral issues are involved, this is not always possible (see previous section). The experimenter can edit the complete participant data by hand, including creation and deletion of new participants. She can observe the whole recruitment process and the number of participants assigned, invited, registered, showed up, and participated for each experiment and session. Monthly subject pool statistics are provided by e-mail to the group and are immediately accessible in the experimenter area of ORSEE. However, although each experimenter can organize own experiments herself, the system cannot solve all tasks that are needed to administer the subject pool. An administrator responsible for the subject pool and recruitment system is needed, who does tasks like searching for doubletons in participant data or controlling that experimenters fill in all participation data. 3 System Overview In this section, we first provide a list with the features of the Online Recruitment System for Economic Experiments. Then we shortly describe the basic system design and show how laboratory experiments are organized in ORSEE. 3.1 Features 3.1.1 General Features · multiple laboratory/subject pool/experimenters/experiment types support · multiple experimenter/subject language support · easily configurable via the web interface · customizable layout 12In Jena, we additionally require subjects to sign the experiment rules (which include insurance clauses) at their first experiment participation. This serves as an informed consent. 10
· experiment classes to organize experiments by topic · structured public and internal experiment calendar including lab reserva- tion for maintenance etc. · overview about registration state · experimenters are informed about session's state by automated e-mails · calendar and session's participants lists in printable format (pdf) · upload / download section to enhance collaboration between experimenters · regular e-mails with experiment calendar and subject pool statistics to experimenters · attribute query selection (e.g., select female participants, select partici- pants who have not participated in experiment X) · random recruitment of subjects out of subject pool · reputation system (number of no-shows, i.e. the number of times a par- ticipant registered for a session but did not show up) · automated warning e-mail to subjects on no-show · configurable automated subject exclusion based on reputation · automated mailing for registration process · subjects are informed by automated e-mails about the sessions they reg- istered for · rule based automated session reminder mailing · subjects are able to manage their own account (without password, using an individualized URL) · comfortable upgrade function to import data from older versions · open source 11
3.1.2 Technical Features · LAMP application ­ runs on Linux ­ Apache webserver recommended ­ uses MySQL database ­ implemented in PHP · data is completely separated from the application · recommended system: Linux on i386/i686 processors, other unixes and Windows Server should work as well · further requirements: PHP on command line, webalizer for usage statistics, cron daemon for regular jobs 3.2 License The Online Recruitment System for Economic Experiments is available under a special open source license called 'Citeware'. Specifically, the source code may be copied, modified and distributed under terms complying with the Open Source Definition of the Free Software Foundation. However, the use of derivative products is restricted in a way that any academic report, publication, or other academic disclosure of results obtained with the use of this software will acknowledge the software's use by an appropriate citation of the paper named in the license. The license in full text: Copyright (c) 2003,2004 Ben Greiner ([email protected]). Author(s): Ben Greiner ORSEE is citeware. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: 1. The use of derivative works is restricted in the following way in accordance with standard academic citation practices: 12
Any academic report, publication, or other academic disclosure of results obtained with the use of this software will acknowledge the software's use by an appropiate citation of the following publication (or an updated version): Ben Greiner: An Online Recruitment System for Economic Experiments. In: Kurt Kremer, Volker Macho (Eds.): Forschung und wissenschaftliches Rechnen 2003. GWDG Bericht 63, Gttingen : Ges. fr Wiss. Datenverarbeitung, 79-93, 2004. 2. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. 3. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. 4. The names of the author(s) and/or the software must not be used to endorse or promote products derived from this software without prior written permission. 5. Products derived from this software may not be called "ORSEE", nor may "ORSEE" appear in their name, without prior written permission of the author(s). THIS SOFTWARE IS PROVIDED "AS IS" AND ANY EXPRESSED OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR(S) BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 13
3.3 Basic System Design The system consists essentially of two views: the experimenters' view on the one hand and the subjects' view on the other hand. Experimenters create experiments which may consist of several sessions and invite subjects. Invited subjects may connect themselves with one of the experiments' sessions in order to participate. We distinguish between 5 different independent states (flags) of a participant with regards to a certain experiment: 1. Assigned: The participant is allowed to register for a session of this experiment. 2. Invited: The participant received an invitation e-mail from the system. 3. Registered: The participant is registered for a certain session of a labora- tory experiment. 4. Showed-up: The participant was at the right location at the right time. 5. Participated: The participant actually played the game. There are four states of a specific session: 1. Not complete: There are not enough participants. 2. About complete: There are as much participants as explicitly needed for the experiment, but not enough reserve participants. 3. Complete: The number of required participants plus the reserve is reached. 4. Finished: All data was filled in for the session. The participation data will be used for the reputation system. 3.4 Conducting a Laboratory Experiment In this section we will briefly describe the procedure of organizing a laboratory experiment with ORSEE.13 When organizing a laboratory experiment, the experimenter first has to log in in the adminstration area and to register the experiment she wants to 13A complete and fully detailed manual how to organize experiments is given in section 5.1. 14
conduct. She fills in the public and internal name of the experiment, a description, the type of experiment, and the experiment's class. She assigns the experiment to herself and/or to other experimenters and chooses who should get the automated e-mails generated by the system. Then she registers each of the planned sessions with date and time, laboratory, experiment duration, number of participants needed and over-recruited, the time of registration and when the reminder should be sent to registered participants. Next, she assigns subjects registered in the database to her experiment. When doing so, she can use different queries including experience, experiment participation, number of no-shows, field of study and profession, a random sub-sample and so on. Once participants are assigned, she sends an invitation e-mail which lists the sessions' dates and times and includes a link to the subjects' individualized registration page. Following this link the subject can choose one date out of the sessions available. When registering for a session, a confirmation e-mail is sent out to the subject. When the registration period expires, the system checks the state of registration for the experiment. For each session, the experimenter receives an e-mail informing him about the number of subjects registered, and having attached a pdf-file containing the list of the names of participants. In case of too few registrations the experimenter may now extend the registration period, or cancel the session at the very end. After a successful registration process the experimenter conducts her experimental session in the laboratory. She fills in the show-up and participation data for all participants. When all data is filled in, she marks the session as finished, and its data will be included in the calculation of reputation score for the participants. When all sessions are done, the experimenter marks the experiment as finished, and it will be listed in the "old experiments" section. 4 The Subject Area The subject area of the system consists of different pages, which are described below. They are available in different languages, depending on the number of languages installed (see section 5.7 on page 36). The default language for the subject area might be set on the general options page (see section 5.11.3 on page 43). 15
4.1 Rules This page displays the experiment rules set by the experimenters. These can include rules for laboratory experiments, internet experiments, video experiments, or online surveys. The rules should contain some sentences about the reputation system used, and the experimenters' policy of handling no-showups and late-comers. They should also contain information about the normal procedure of an experiment, if there are payments or not and so on. At our institution, we require subjects to sign these rules before their first experiment participation, so that they serve as an informed consent. 4.2 Privacy Policy To deal with legal requirements and to provide trust to the potential participants one should add here the policy of the institution regarding the data in the recruitment system and the data collected in experiments, too. 4.3 General Registration Before the registration participants have to choose their own subgroup. The page will only show up if different subgroups are defined. Subgroups are useful because they provide a tool to administrate different populations, for instance mailing lists for internet experiments only or laboratory subjects in different cities. Moreover, one can write a different registration page at another institution's web page, and assign people registering on this page to a certain subgroup (see also section 5.11.11 on page 54). After having selected their subject pool, people see the rules for experiments and the privacy policy. They have to accept both by clicking on the acceptance button before coming to the registration page (Figure 1). On the registration page people are asked to enter their personal data. Only the e-mail address, the last name, and the first name are required. A text shown above the form should inform potential participants that providing additional information about themselves can lead to more invitations. The data asked for includes gender, phone number, begin of study, field of study, and profession (depending on subgroup). After submitting the REGISTRATION FORM by clicking the button, the data is checked for doubletons with already registered participants and will be inserted only in a temporary table. The candidates are informed that they got an email to the account given to confirm their registration. This is done to avoid nonsense registrations. 16
Figure 1: Participant Registration Form. In their confirmation e-mail the participants receive a link to click on. This brings them to a page which informs them about the successful confirmation of their registration. Now the data will be inserted in the regular participant table. Every e-mail participants get from the system contains a link in the footer, which leads to the participant data editing page. Here a form similar to the registration form allows the user to change his data or to unsubscribe in order not to get further invitations. To keep database integrity the account is not deleted internally. If the subject tries later to register again with the system, the system recognizes him and an experimenter can reactivate his account by hand. At the Jena privacy policy page we declared that we will delete personal data on written request. 4.4 Invitation and Registration for Experiments An experiment invitation e-mail (see Figure 2) includes a personalized link which leads to the participant's experiment registration page (see Figure 3). This page consists of three parts: · a list of future sessions of all experiments the participant has been invited for, yet has not participated or registered so far and for which the registration period has not expired (see section 5.1.2 on page 24) · a list of the future experiment sessions the participant has already regis- 17
Figure 2: Experiment invitation e-mail. tered for · a list of former sessions a participant was registered for, including a summary of finished sessions (see section 5.1.6 on page 30) While the two latter only have informational character, the first list contains small register buttons on the right side. If a user clicks on one of these buttons, she registers for the corresponding session. A text above the page should tell the participants about the binding character of the registration. At the same time an e-mail is sent out to the participant's e-mail address to inform him again about the successful registration, again containing the date, time and location of the session. At a time specified on the administration page for the session (see Section 5.1.2 on page 24) a subject has registered for (e.g., 48 hours before the session starts), a session reminder e-mail is sent out to the participant. ORSEE provides no mean for subjects to deregister from a session. We rather encourage subjects to check thoroughly if they are available for the time of the session at the moment of experiment registration. However, if there are reasons beyond his control, the participant can write an e-mail as a reply to his registration e-mail, and the experimenter will deregister him. 18
Figure 3: Experiment registration page. 4.5 Calendar The calendar contains an overview of all experiments and their sessions (Figure 4). The information about a session contains the public name of the experiment (which can be switched off, as well, see Section 5.1.1 on page 21 and Section 5.11.3 on page 43), the time, date and location of the session and the status of the session. The latter is denoted as free places if the registration time has not expired and there are free places left, and as completed otherwise. That means, that sessions which are not full already but whose registration period has expired are marked as complete at the subject area. In the calendar, sessions are presented with different background colors for each experiment. This eases the separation of experiments for subjects, since every participant can only register for one session of an experiment. However, you might want to present all experiments in the same color (probably in conjunction with hiding the experiment name). Then, simply define just one color in the color list for the public calendar (see Section 6.6.2 on page 62)). 19
Figure 4: Experiment calendar in Subject Area. 4.6 FAQ Subjects regularly have questions about details of the registration procedure and other features. To avoid answering all the questions several times, there is a page containing 'Frequently Asked Questions'. They are multilingual as well and administrable through the experimenter section (see Section 5.11.12 at page 55). 20
5 The Experimenter Area The experimenter area is accessible via the subdirectory /admin.14 If you are not logged in yet, you are asked to enter your username and password. Note: Case matters for username and password. After logging in you enter the main screen of the experimenter area (Figure 5). From there you can jump to the different functions described below. Your current location is marked at the navigation area on the left side. In the following we will explain the main function of the system by providing a tutorial concerning the organization of a normal laboratory experiment. Figure 5: Experiment overview in experimenter area. 5.1 Laboratory Experiments 5.1.1 Registration of the Experiment To register a new experiment choose Create new from the experiment's part at the navigation bar or from the experiment overview page. You carefully have to fill in the fields which are explained below (see Figure 6). 14You find the Experimenter Area of a test system at http://www.orsee.org. You may use the username 'test' and the password 'test' to log in. 21
Figure 6: Creation of a new experiment. Name: State the internal name of the experiment. This name will only be used at the internal pages and is therefore not accessible by possible participants. Take a meaningful name so that you can later quickly find and identify your experiment. Public name: In this field the publicly used name of the experiment has to be given. You should choose a generic, abstract name, which nevertheless allows for non-ambiguous identification, like Experiment 24. The public name can be hidden in the public calendar, see Section 5.11.3 on page 43) for details. Description: Describe your experiment in short sentences. Together with uploaded instructions this can help other experimenters to benefit from your knowledge. In addition, it allows to determine experiments on which subjects should not have participated when registering for your experiment. The description will not be shown in the subject area. Type: ORSEE distinguishs between external and internal experiment types. They can be configured in the 'Options' section (see Section 5.11.8 on page 52 of this manual). In the selection list name is the external type, which is the experiment type subjects subscribe to. The name in the brackets is the internal experiment type, i.e the way the experiment has to be organized. In version 2.0 ORSEE only knows the internal type of laboratory experiments, but in later versions we will add online-surveys and internet experiments. 22
Class: To organize experiments by topic, the system allows to distinguish several classes of experiments (e.g. dictator games, public goods games etc.). These can be configured in the options section (see Section 5.11.7 on page 52). Experimenter: Select the user names of the experimenters involved in the experiment. The experiment will appear in the My Experiments list of the related experimenters. Experiment access restricted?: If you check this box, only experimenters assigned to this experiment will be able to access the experiment's pages and options. This checkbox will only show up if experiment restriction is allowed in the system, which can be set in the general options section. Get emails: Select the e-mail-address of the experimenters where all messages of the system regarding this experiment should be sent to. E-mail sender address: This address will be the sender of all e-mails from the system to the assigned/invited/registered participants of this experiment. That means, that this is the address they will reply to if they have questions or comments. You can specify only one e-mail address. Experiment finished?: When setting up the experiment you leave this Box unchecked, of course. Hide in participants statistics?: If you don't want that this experiment is considered when calculating no-show-up numbers for participants statistics, then check this box. This is useful when you add data of old experiments which you conducted before you started with ORSEE and its recruitment system. Hide in public calendar?: Normally, all laboratory experiment sessions will be listed in the public calendar. But sometimes you don't want to show the sessions date to the public, i.e. when you invite two definite subgroups at the same time. If you check this box, the sessions of this experiment will be hidden. Link to Paper: When finished with the paper, you can provide the full URL to allow other experimenters to download the paper. After filling in the fields as described above, click on add. The page returns with the message Changes saved. If you now click on Experiments ­ Overview, you will see your just created experiment listed under Experiments without dedicated sessions. You can access this experiment for further settings by clicking on its name. For laboratory experiments the experiment's main page consists of three parts: first, the basic data as entered above is listed. By clicking on Edit basic data you jump to a form similar to the one you used to create the experiment, 23
where you can make changes. If you click on Upload file, you can exactly do this: upload files like instructions, programs etc. This can be useful for your colleagues as well as for you (just imagine, you are in the lab and forgot your instructions ...). The second part lists the sessions of the experiment with some additional information. The last part is dedicated to the subjects of your experiment: here you can administer their assignment and invitations. 5.1.2 Registration of Sessions To register a session click on the Create new link in the sessions part of your experiment's page. You see a new form containing the following fields (see Figure 7): Figure 7: Creation of a new session. Date, Time and Laboratory: Fill in exactly what you think you should fill in. Duration of experiment: This value has to be given in a hh:mm format and is used twice in the system: first, it will be checked whether this session overlaps with other sessions in the same laboratory. It is possible to ignore the warning message, but you should of course only do so, if you plan to hold two sessions at the same time. Second, this time will be shown in the public calendar as the maximum time the session will last. 24
Session reminder : Choose the number of hours before the session starts, where you want to be sent out a reminder e-mail to the registered participants. This time should of course be later (i.e. a lower number) than the time stated in the field Registration end. Send reminder on ...: In this field you state the rule which should be applied at the time the session reminder should be sent. You have three options: · when as much part. registered as needed plus reserve, else manually: The session reminder e-mail will only be sent out if your session is completely full, i.e. when the number of registered subjects is equal to the number of needed subjects plus the number of reserve participants. Otherwise, you will only receive a warning email and can send the session reminder manually from the session's participant list (see Section 5.1.5 at page 29). · when as much participants registered as needed, else manually: Here the system ignores the number of reserve participants. If the number of needed subjects is met, the reminder will be sent out; if not, see above. · in any case, don't ask : The reminder will be sent out irrespective of the number of subjects already registered. The default rule can be set in the options - default value section. Needed participants: State the number of participants you exactly need to carry out the experiment. Reserve participants: To handle the case of no-show-ups, one should of course invite more people than actually needed. Registration end: Choose the number of hours before the start of the session, when you want the registration period to end, i.e., after the time you are stating here participants are no longer able to register for the session. Of course, you can also extend this time at a later stage. Remarks: This is only a small notepaper for you. Session finished?: Leave this box unchecked when setting up a session. By clicking on the Add button the session will be registered. You get back the same form with the message changes saved. If there is a overlap with another session in this laboratory or another laboratory reservation, a warning message will be shown, too. Your session will now be listed in the session part of the experiment's main page. The Edit link allowed you to change the settings. Below the sessions' date and time you see the words Registered subjects and three numbers. The first number shows the number of subjects already registered for the session, the first 25
number in brackets the required number of participants for the experiment itself, and the second number in brackets the required number of reserve participants. If you click on the words Registered Subjects, a list of the subjects registered for this session will be shown, which is empty. You should now register all your other sessions before turning to the last part of the experiment's main page. At the experiment overview page your experiment is now listed in the section Experiments with dedicated sessions. 5.1.3 Assigning Participants The third part of your experiment's main page contains a list of participant states as described above (see section 3.3 on page 14), i.e. you get informed about how much participants are assigned to your experiments, how much invitations you sent out, how much subjects have registered for sessions, showed up and participated. At time, there should only be zeros. The selection process of participants will be done when you assign participants to the experiment. To do this, choose Assign Subjects from the options part. You get a human readable form to perform a query into the database (see Figure 8). The default is to select all possible subjects not yet assigned to your experiment. By activating the different clauses (by checking the small box next to the number) and filling them in you can define your selection. To find the right selection, just try to formulate an intact sentence by choosing its different parts from this menu (i.e. "Select all ... where gender is 'female' ... and where field of studies is 'economics' ... and without subjects who have participated at one of the following experiments: 'Dictator Game','Ultimatum Game' ... , but at maximum 100 subjects"). At the current state, the following query modules are implemented in ORSEE: · freetext for last name, first name and e-mail address · number of no-shows · number of registrations · subjectpool · gender · begin of study · field of study 26
Figure 8: Assigning participants. · profession · participation at experiments of certain classes · participation at former experiments · assignment to other experiments · random subset of certain size These modules allow the experimenter to restrict the subject pool according to his needs. Particularly the possibility of a random draw ensures that not always the same subjects with immediate access to the internet register for the experiments. After your selection, click on Search and Show. You get the list of all participants satisfying your selection clause to check your query. You can simply assign all of the listed people by clicking on the appropriate button, or assign only selected participants by checking them and clicking on the button Assign only marked participants. After submitting your choice, you get a feedback message on how many participants were assigned and will return again to the assignment form. After making your participant assignments you can see their number at the described list in the last part of the experiment's main page. 27
5.1.4 Inviting Participants After having registered sessions and assigned participants, you probably want to inform them about being selected for participation. The recruitment system includes the feature to send out automated messages to your aimed participants. To do this, click on Send invitations on your experiments' page in the bottom participant part. Figure 9: Sending invitation e-mails. For every language you allow for participants, you see the following fields to fill in (cf. Figure 9):15 Subject: Fill in the subject of the invitation mail. Mail text: In this field you can insert the body of the invitation message. If you enter this page the first time, the default text is shown. If you change the text it will be saved and presented if you return to the page. To rebuild the original default text just clean the whole text window and click on save. You may use variables in the text. The most important ones are the list of available sessions (#sessionlist#) and the link to the registration page 15The configuration of languages is described in Section 5.7 at page 36. Default mail texts can be configured in the options section, see Section 5.11.5 on page 49 of this manual. 28
(#link#). The other variables including first and last name (#fname#,#lname#) are means to personalize e-mails. Underneath the described fields you see some options. The one with the fewest consequences is the Save button, it just saves the mail text and subject. By clicking on Mail preview the texts will be saved, too, and you will be taken to a page containing a good approximation of the resulting e-mail. Being back and having checked the e-mail preview, there are two ways to send out e-mails. First, you can send them only to the people who have not received one before, which should be the normal case. Second, you can mail the message to a greater audience including the subjects who have already got an e-mail for this experiment, but have not registered for a session of the experiment so far. When sending out a mail for the first time, both options will lead to the same result. Note: You should always check the mail text via the preview before sending out the e-mail for a nonempty session list. If there is no session with registration period that ends in the future, then the list will be empty and a quite meaningless e-mail would result. The time needed for sending the e-mails depends on the system's configuration and the number of e-mails to send out. ORSEE uses an internal mail queue to send out e-mails. That is, the messages are not send directly, but rather put in a table. A so called 'regular task' will process the mail queue regularly and send out a certain number of e-mails each time (see Section 5.6 on page 36 for details). When ready with all the work, you will be redirected to the invitation page and a message appears informing about the number of e-mails sent and the time needed. After receiving your e-mail, the participants can register for a session as described above in section 4.4 on page 17. 5.1.5 Control of the Registration Process Some of you might not be satisfied by the convenience an internet based registration process provides or might simply not trust the potential participants. In that case, you have the option to monitor the registration process and to change everything by hand. As already mentioned above, a summary statistics about the registration process is given in the bottom part of your experiment's page which is focused on the whole experiment. In the center part of the page one statistic is given for 29
each session separately. By clicking on the links behind the participant states you reach a page with a list of the corresponding participants. Clicking the Registered subjects link in the session area is almost the same as clicking on it in the participants area, with the difference that you only get the participants registered for the corresponding session. At all participant lists you can download a printable PDF version of the list by clicking on Print version. Assigned (invited) subjects: To keep the page as short as possible the emerging list contains only those assigned (invited) subjects who have not yet registered for a session. You can register some or all or none of them for a certain session by hand. To do this, you check the boxes on the right (you can also use the Check all/Uncheck all buttons) and choose the session in the list at the bottom. By clicking on the Change button you (try to) register them for the session. Before writing to the database a check is performed if the number of the new participants together with the already registered subjects exceeds the capacity (participants needed + reserve) of the session. If this is the case an error message appears and you have to deselect some of your choices. However, to allow for flexibility, you can overrule the check by deselecting the small checkbox at the bottom of the table (Yes, the one denoted Check for free places in session). The registered subjects will not appear again in this table, from now on you can administer by using the following options. Registered and showed up subjects and subjects participated: In principle, the tables you get for this three options are similar, except, that for each option only the appropriate subjects are shown and that in the list for the subjects participated you cannot change the show-up status. (Subjects participated always showed up.) In the table itself you get the necessary information to handle the session: names, e-mail addresses, phone numbers, and participation history. You can move subjects within the experiment from one session to another by changing their session field and clicking on the change button. To de-register participants (in case they cannot come) move them to a session called No Session. 5.1.6 Finishing a Session Finally, there should be enough participants in every session. If you have an internet connection from the laboratory to your experiment recruitment server, you can do the following steps just in time when the participants arrive at their 30
Figure 10: Session overview and experiment statistics. location, if not, do it later. This is essential to give the whole system a little amount of -- meaning and consistency. To keep track of participation, click the Registered subjects link of your session and fill in the right columns. For showed-up subjects (the persons who get a show-up fee from you) mark the shown-up box, and for persons actually participating mark the participated box. At the end click the 'Change' button to save the results. After you have checked the list, go to the Edit page of the session and check the Session finished? box. From now on, this session will be included in the calculation of the subject's participation history. 5.1.7 Ending the Experiment If you have run all sessions and finished them in the database, go to the Edit page of your experiment and check the box Experiment finished?. The experiment will be moved from the Current Experiments list to the one containing Old Experiments. 31
5.2 Maintaining the subject pool Although most of the work updating participant data should be done by the participants themselves, it is especially useful for problematic cases to edit the data by hand. To do that, several functions are provided. 5.2.1 Temporarily Registered Participants On the Overview page you see a short summary about the participants in the database, separated by experiment type subscriptions. By clicking on the Registered but not confirmed link you see a table containing all registrations not confirmed yet. By a click on Delete the corresponding row will be deleted immediately. 5.2.2 Adding New Participants When you choose Add participant on the participant overview page or Create new on the participant section at the navigation bar you are directed to a form to add a participant. In fact you get a form similar to the form presented to participants when they register with the system (see section 4.3 on page 16). Depending on the selected sub-subject pool you should either choose the main field of studies or the profession of the subject. Note, that a participant has to be subscribed to at least one experiment type. You can directly assign the participant to an existing session. Do this by checking the small box at the bottom of the form and selecting the desired session. A message will inform you about the changes after you have clicked on the Add button. 5.2.3 Edit Participants By clicking on this link you will first get to a page containing the same search form as used to assign participants to an experiment (see section 5.1.3 on page 26). You can search participants by using this form or immediately click on the button bringing you directly to a list of all participants in the database. At the end of each row in the resulting list there is an Edit link. By following this you can change the participant's data or delete him. At the bottom of the participant's edit page a complete experiment history of the participant is shown, including registrations to current experiments. 32
5.2.4 Delete Participants By clicking on the Delete button on the edit page of a participant, you will receive a confirmation page containing the participants data and three options. You can cancel this action, delete, or exclude the participant. When you delete a participant, a flag will be set in the database that this participant will not be available for experiments in the future. However, current registrations will not be affected by this. By explicitly excluding a participant an additional flag will be set. This is just to remember that this participant was not self-selecting to quit the database but was deleted for some serious reason. (It's a good practice to note this reasons in the 'remarks' field.) You can access and edit the data of deleted and excluded participants via the Edit deleted participants option on the participants overview page. 5.2.5 Edit Deleted Participants Via this option you can edit the data of deleted or excluded participants or reactivate them by clicking on the button 'Resubscribe' and confirming on the next page. 5.2.6 Automated Participant Exclusion ORSEE does allow to automatically exclude subjects from further registrations for experimental sessions based on their number of no-shows. To enable this feature you have to configure the corresponding regular task (see Section 5.10 at page 41) and to set your rules in the general options section (see Section 5.11.3 at page 43). You may switch on the feature that participants who did not show up at an experimental session automatically get a warning e-mail from the system. 5.3 Calendar The calendar in the experimenter area provides more information than the calendar in the subject area (see Figure 11). For every experiment it contains the internal name, the date and time, the experimenter, the number of already registered participants compared to the number of required and reserve participants, and a link directly to the participants list of this experiment. Again, a click on the Print Version link delivers a nice format pdf file of the calendar ready for printing. In the calendar, the sessions of different experiments are presented in different colors, which can be configured in the color file of the current style (see 33
Section 6.6.2 at page 62). Figure 11: The internal calendar in the Experimenter area. At the calendar page you might reserve laboratory time not occupied by experimental sessions for maintenance tasks and other outages. Do do this choose the link 'Reserve laboratory time' at the top of the calendar page. You are presented a small form where you have to fill in the laboratory concerned, start and ending time of the reservation, your username and a reason for laboratory time reservation. The reservation can be done for several days. After adding the reservation you are immediately informed if the reservation time clashes with experimental sessions or other reservations. To edit or delete a reservation you can click on the 'Edit' link in the calendar entries for this reservation. 5.4 Downloads This module provides a tool to share your knowledge with other experimenters at your institution. You can upload files in categories like instructions, programs, presentations, papers etc. Experiment related files should be uploaded directly from the experiment's main page, while general files can be uploaded at the download page by following the link 'Upload general file'. At the upload 34
page you are asked for the category the file belongs to, the name the file should be given in the download section, and the path to the file on your local computer. The file size is restricted by your server and PHP configuration and the limit set in the general options section (see Section 5.11.3 at page 43). There you might also configure the categories available. 5.5 Logs and Statistics 5.5.1 Log Files In ORSEE all actions by subjects, experimenters and the system are logged to the database. This provides means to retrace changes in the database. At the statistics section you may browse these log file. Actions are sorted by descending time, and long tables are separated on different pages. The tables present the time of the action, the participant/experimenter who executed the action, the name of the action and if available, the target of the action. By clicking on the 'Restrict' link you can restrict the view of the log file to actions of a certain participant/experimenter, to certain types of actions or to certain targets. A select field and a button on the top right of the page allows to delete entries older than the specified time. You should use this from time to time for experimenter actions and regular tasks to reduce the database size. However, please note that the system's statistics described below rely on the data in these log tables. 5.5.2 Statistics ORSEE provides comprehensive on-the-fly statistics about the system, the web server use and your subject pool. In version 2.0, system statistics contain only participant actions, but will be extended in the next versions. The participant statistics can be presented as verbatim text, html tables, graphics or combined tables and graphics (we recommend to use the latter form). At the top of the page a statistic about the used subject pools is presented. All other graphs and tables can also be restricted to a certain subject pool. In ORSEE 2.0 statistics include gender, profession, field of studies, begin of studies, experiment participation per month, experiment experience by count, no-shows by count and by month (Figure 12 shows a screenshot). A text version of the subject pool statistics are regularly sent out by e-mail to subscribed experimenters. 35
Figure 12: Subject Pool Statistics. Webserver statistics are produced with the help of the program 'webalizer' generated as a regular task. Installation is described in Section 6.5 at page 60. 5.6 E-mail Processing To improve performance of the system when organizing sessions, ORSEE uses an internal mail queue to send out bulk mails. Specifically, when sending invitations mails, for instance, these mails are not send directly, but to a queue table of the database. A regular system task processes this mail queue every 5 minutes and sends out 50 e-mails each time. The time span can be configured with the regular task, the number of e-mails in the general options section. E-mails processed through the mail queue are invitation mails, session reminders and bulk mails. No-show notices, general and experiment registration confirmations and all e-mails to experimenters are sent on-the-fly and not using the mail queue. 5.7 Languages ORSEE allows to install as much languages as you want. You may download them from www.orsee.org. The standard languages shipped with the ORSEE 36
packages are English and German. You may add your own language to the system. Languages are maintained in a huge table in the ORSEE database. Particularly, (nearly) every human readable output of the system comes from this table, that is e-mails and all phrases on the HTML pages. Thus, the options for e-mails, default texts, help texts, public content, laboratories, subject pools, experiment types, experiment classes, experiment invitation e-mails, fields of studies, professions, and FAQs are all multilingual, i.e. have to be set for all installed languages of the system. You can to this at the specific pages of the concerned items. In the section Options/Languages you can edit the phrases (the words and sentences used in ORSEE) and the languages themselves. At the languages main page a list of the languages installed is presented. For each language you can choose if it is available in the public area and if participant can choose this language for their communication. If you enable the public availability, visitors of the public area of your recruitment system can switch to this language to view the pages. If you enable this, you should revisit all items listed above for their correctness. At least one language has to be selected, which is the public standard language set in the general options section. To change the public standard language to another one, enable the new language here, choose it as the public standard at the general options page, and disable the old one here. If you enable the participant availability, you will have to edit all experiment invitation e-mails in this language. At least one language has to be selected, which is the public standard language. 5.7.1 Edit Language Symbols The appropriate link in the language's row leads you to the language edit page. You may browse the language symbols using the alphabet or search for language items using the search form. Each row of the resulting symbol table contains the symbol's name, it's value in the currently used language, and a text field with the value of the symbol in the language to edit. After having changed the contents of the text fields you can save the changes by clicking the 'Change' button. Following the 'Edit' link you are presented a form which allows you to change the term in all languages installed. 37
5.7.2 Add Language To add a language, click the button on the languages main page. You will have to provide a shortcut for that language, the name of the language in that language, and the language the new language should be based on. That means, that when created, the new language will copy the language terms of it's base. When you want to create a very new language, you may now edit the language terms for the newly created language. To import a language from an ORSEE language file use the import function described below. 5.7.3 Delete Language At the language deletion page, you will have to select the language which you want to delete. This cannot be your current language. You should select another admin and public standard language before. Second, you will have to state to which language participant and experimenters which use this language should be assigned to. After a click on the button you have to confirm the deletion, after confirmation the language will be deleted. 5.7.4 Edit Basic Data On this page you can change the language's name and import or export a language file of this language as describe in the following sections. 5.7.5 Export Language File The language export page presents you a link to the ORSEE language file of the selected language. You download and save the file by right click and choosing 'Save link as ...'. The format of the ORSEE language file is simple: It contains the symbol's names and values. Lines and variables are separated by unique terms. The file is base 64 encoded to allow compatibility with all operation systems. Note: Only language symbols, default email texts, default texts and help texts will be exported. 5.7.6 Import Language File The language import dialog gives you three options: to update your language, to upgrade it and to do both at once. 38
'Updating' means that only language symbols already defined in the system will be imported from the file. Existing terms will be overwritten by the values from the file. Use this option to install a new language on this system from a language file of at least the same version. The language 'Upgrading' option is provided to ease the upgrade of your ORSEE version. Often, a new version of ORSEE only has minor changes, i.e. no changes on the database structure. Then, an version upgrade is done by just replacing the program files and installing new defined language symbols. The latter is done by the 'upgrade' option. Symbols not existing or empty on your system will be installed, but symbols already existing will not be overwritten. The third option does both at once: it installs new symbols and overwrites the old ones. Use this when upgrading the system and at the same time installing a new language. 5.8 User Rights Management ORSEE comes with a complete user rights management. The users of the administration area can be assigned to different 'Admin types' on the user's edit page (see Section 5.11.2 at page 43). For each type the user who has the right to edit the administrator types (normally the system administrator) can specify the functions which user's of this type are allowed to access. Some of these function rights are hierarchical, meaning that some functions may require other functions to be allowed. To edit the administrator types browse to Options/Admin Types. You will be presented a list of installed administrator types with their assigned rights. By clicking on the 'Edit' link you may edit a type. Use the 'Create new' button to add a new type. On the edit page for a type, you see a text field for the type name and a table containing all available rights of the system. The second column presents the name of the right, the third includes a small description, and the forth column lists possible precondition rights for the right concerned. The small checkbox in first column indicates if the right is enabled for this type or not. You may change the assigned rights by checking and unchecking the boxes. By clicking the 'Change' button at the bottom of the page you save your changes. Use the 'Delete' button to delete the type. Note: The use of experiment access restrictions (see Section 5.1.1 on page 21) may overlap with these user rights. That is, to access a function for an experiment, (1) this function must be allowed for the user's type, and (2) the user must have the right to access the 39
experiment, if the right to override experiment restrictions is not enabled. The default types delivered with ORSEE are the following:16 · nobody: This type is not even allowed to login into the administration area. Use it for example for users who should only receive calendar and subject pool statistics from the system. · visitor : A 'visitor' is allowed to login and to see experiment, session and participant data and statistics. However, she is not allowed to change any data in the system. · showup updater : Some laboratories use student assistants to update the participation data for conducted experimental sessions. This user type is allowed to edit the participant lists for experimental sessions and to upload files for experiments. · experimenter : For the 'experimenter' type all functions which are needed to organize an experiment in a completely configured system are enabled. · admin: This type is thought for system administrators. It allows for adding and editing users (and their type), to override experiment restrictions, delete experiments and non-empty sessions, edit language phrases, regular tasks, and to edit the most options of the system. · installer : For the installer all functions which normally are only needed when setting up or upgrading the system are provided. These include language installation, data import, editing general options and default values and adding/deleting of most option items. · developer : The 'developer' has access to all functions of the system. The extra rights compared to the 'installer' are only functions which are eventually needed when developing the system, i.e. when changing the code. In the user right list these functions are highlighted with 'dev!'. 5.9 Reputation System ORSEE provides the feature of a reputation system for subjects' reliability. After the conduction of sessions experimenters fill in the participation data. From this the software regularly calculates a reputation score, the number of 16These types are organized hierarchical, that means that every type has all rights of the former types. Other, mixed organizations are possible as well. 40
no-shows, for each subject. This score is shown in participant lists to experimenters and at their personal registration page to subjects. Experimenters may exclude subjects from invitations based on the reputation score in the participant experiment assignment query. In the options section you may set up the automatic exclusion feature. Here you have to state from which number of no-shows a subject should be excluded. You can choose if the subject should receive a warning e-mail on every no-show and should be informed in case of exclusion. The exclusion tests runs as a regular task (see below). 5.10 Regular Tasks The regular tasks are tools to process functions of the system not initiated by a user action. To set them up you have to configure a cron job as described in Section 6.4 on page 59. Regular tasks can be configured by browsing to Options/Regular tasks. First, you see a list of regular tasks available for the system with information about their due-date and the time of their last execution. You may run a regular task immediately by clicking the 'Run now' button in the corresponding row. By installation default, all regular tasks are disabled. The following tasks are available: · check for noshow warnings: Checks if participants did show up at finished experimental sessions and sends a warning e-mail to the ones who did not show up that they will be excluded after a certain amount of no-shows. Configurable in Options/General Settings. · check for participant exclusions: Checks whether participants did not show up a certain amount of times, excludes them from further experiment registrations and sends a notification message to the concerned participants. Configurable in Options/General Settings. · check for registration end: When the registration period for an experimental session has been expired, this function will send out a notification e-mail to the experimenters of the concerned experiment with information about the current state of registrations for the session. Registration period configurable at the session's edit page. · check for session reminders: At the time specified with the session properties, a reminder e-mail will be sent out to subjects registered for this 41
session. The sending of the reminder is based on a rule set at the session's edit page. · process mail queue: Sends the next block of invitation, bulk and reminder e-mails from the internal mail queue. The number which should be send each time is configurable in Options/General Settings. To allow for the mentioned kinds of e-mails you must enable this task. · run webalizer : Runs the program 'webalizer' on the system's web server access log file and dumps the output to the 'usage' folder to be accessed from 'Statistics/Server usage statistics'. Please consult Section 6.5 on page 60 and Options/General Settings for installation and configuration. · send experiment calendar : Sends an e-mail with the experiment calendar for the current month (PDF file) to all experimenters subscribed. · send participant statistics: Sends an e-mail with actual subject pool statistics to all experimenters subscribed. · update participants history: To improve performance, summary statistics of participants' number of experiment registrations and no-shows are computed regularly by a system task instead of being calculated on-the-fly. You must enable this regular task to allow the system to work properly. 5.11 Other Options and Settings 5.11.1 My Profile Here every user of the administration area can edit her own settings. First and last name, e-mail-address, preferred language for pages and e-mails can be set. The user can opt-in to receive the periodical experiment calendar and subject pool statistics. The user may also look at the list of rights assigned to her and change her password. Note: You should handle your password very carefully. This system is connected to the internet. And a system is only as secure as its doors. An experimenter account usually has the right to change nearly everything saved in the database. Thus, by using a bad password or by not keeping it secret you do not only jeopardize you own data, but as well the data of your colleagues. 42
Your password will be immediately encrypted using the unix crypt algorithm when arriving at the server and also saved encrypted in the database. A good password consists of 8 letters including numbers and special characters. 5.11.2 User Management Users of the administration area of ORSEE can be edited in 'Options/Edit administrators'. You see a list of registered users with some of their properties. On the edit page for a user, the administrator sees the same form as the users in the profile area (see section above), but with some additional items: · Username: The login name of the user. We recommend to use only lower case letters and no spaces. · Type: The user's type. For details, see Section 5.8 on page 39. · Is experimenter? : When this option is enabled, the current user is listed at all experiment edit pages as a possible experiment owner and experiment e-mail receiver. · Password: You may change the password of the user here. This field is only processed if non-empty. You may delete a user using the 'Delete' button. However, to keep the user's information in the system we recommend not to delete users but to set them to a type like 'nobody' with no login rights and unsubscribe her from calendar and statistics e-mails. 5.11.3 General Settings You find the general options page in the section Option/General settings. It should be sufficient if you edit these settings once - when you install or upgrade the software. Beware that all of these settings have an impact on the system's functions and appearance. The following items can be edited: · Administrator Standard Language: Choose here the standard language for the administration area. Registered users might choose their own language out of the languages installed in their user profile. · Public Standard Language: The public standard language is the language which will be used for the public pages when visitors browse to your site. 43
The public standard language must be available at public pages and as participant language (see Section 5.7 at page 36). · Exclusion Policy: Max. Number of No-Shows: This number is used in warning e-mails and for tests if a participant should be excluded or not (see below). · Send warning email on no-show? : State here if e-mails to subjects who did not show up at an experimental session containing a warning about the maximum number of no-shows should be sent out. To make that work you have to enable the corresponding regular task as well. · Automatically exclude participants after max no-shows? : State here if participant who did not show up the maximum number of no-shows should be excluded automatically by the system. You have to enable the corresponding regular task for that. · Inform excluded participants about automated exclusion? : State here if participant who were excluded by the automated function should get an information e-mail about that. · Allow restriction of experiment page access to experimenters? : As a security feature ORSEE allows that experiment owners can restrict the access to experiment relevant pages to experiment owners. Only users who have the corresponding user right (e.g. system administrators) may override this. You may enable or disable the possibility to restrict experiment access here. · First part of each page's title tag? : This is the first part in the title of each html page. For example, if it is set to 'ORSEE', then the title for the options main page is 'ORSEE: options', which will appear on browser title and tabs on the user's computer. · Type of sending emails: On web servers who have no DNS entry, i.e. are only known by their IP address, the original PHP mail function does not work because it does not set a valid so-called 'envelope sender address'. When this problem arises ORSEE may try to directly use the sendmail function of your system. So, first try to send e-mails with this option set to 'direct'. If this does not work, try 'indirect'. · If indirect: path to sendmail program/wrapper? : When using the 'indirect' type of e-mail sending (see above), we need the path to the sendmail 44
program/wrapper on your system. On Linux servers the program is normally located in '/usr/sbin/sendmail'. If not, you may find out with the command 'locate sendmail'. · Number of mails send from mail queue on each processing: ORSEE's internal mail queue is processed regularly by a corresponding system task. Fill in here the number of e-mails which should be sent on each processing. · Path to server log file (access.log)? : To deliver web usage statistics for your ORSEE system, we need the path to your webserver's access log file. This should be readable for your user account, of course. On dubiety ask your web server administrator. · Style for Public Area: Choose here the style for the public accessible pages of your ORSEE installation. The list of styles is taken from the /styles directory. On how to install styles see Section 6.6 on page 61. · Style for Administration Area: Select the style which should be used for the administration area. · Installed GD Version: For graphs in subject pool and system statistics ORSEE uses the GD functions implemented in PHP. Thus, the PHP installation should be compiled with GD support. However, in most standard binaries of PHP this is the case. Because from version 2.0 GD supports some more functions needed for pie charts, you may select 'ї= 2.0' here when your installed GD version is greater or equal version number 2.0. In all other cases choose 'Ў2.0' here or you will get broken pie charts. However, GD version 2.0 also works with the latter option. To find out your local GD version, try 'php -i -- grep -i "GD Version"' on command line. · Stop public site: If you set this to 'yes' then visitors of the subject area will be redirected to an error page stating that the system is currently maintained. However, when you are logged in as an administrator you are still allowed to visit the subject area. Thus this feature might be useful when you are working on the database or installing/upgrading. · Default registration subject pool? : Select here the default subject pool out of all your different sub-subjectpools which should be applied to participants if no subject pool is given (see Section 5.11.11 on page 54 for details). 45
· System support email address: Fill in here the e-mail-address of your system. This address will be used for support requests by participants as well as for the field sender-address in all e-mails sent out by the system. Indeed, this should be a valid e-mail address. · Upload categories: Include here categories to which files can be assigned when users upload them. The category names have to be separated by commas, with NO spaces within. Note, that the default category names (instructions, data files, programs, paper, presentations, other) are also defined as language symbols, making them multilingual. If a category name is not defined as a language symbol, simply the name itself will be shown on the pages. · Upload max size in bytes? Fill in here the maximum allowed size for uploaded files. Note that the file size is further restricted by your local webserver and PHP configuration. · Participant calendar: Hide public experiment name? In the public calendar, you might want to hide the public experiment name. For example, you might fear that your subjects talk to each other about their experience in a specific experiment. When hiding the experiment name and choosing just one background color for all experiments, they cannot learn from the calendar if they participate in the same experiment or not. · Emailed PDF Calendar: Number of months included? Choose here the number of months which should be included in the regularly sent experiment calendar e-mail. 5.11.4 Default Values On the page Options/Default Values you may configure all values preselected in item creation forms and other settings. Particulary, these are the following: · Default administrator type? : The type which should be selected by default when a new administration area user is created. · Participant statistics: list: ORSEE may show the system and participant statistics in different formats. We recommend to use 'HTML+Graphics'. · Log files: entries per page? : The number of rows which should be shown on each page in the log files area. 46
· PDF Calendar: title font size? : The font size of the title in the PDF version of the experiment calendar. · PDF Calendar: table entry font size? : The font size of the calendar entries in the PDF version of the experiment calendar. · PDF Participant list: title font size? : The font size of the title in the PDF versions of participant lists. · PDF Participant list: table entry font size? : The font size of the rows in the PDF versions of participant lists. · List of experimenters: number of columns? : On the experiment edit pages, ORSEE shows a list of all experimenters registered in the system in a small table. State here the width of that table, i.e. the number of columns used. · Participant query: list of experiment classes: number of columns? : In the participant query form, a list of all defined experiment classes is shown. State here the width of that table, i.e. the number of classes listed per row. · Participant query: list of old experiments: number of columns? : State her the same number for the list of experiments. · Participant query: number of old experiments to show in limited view? : To not overburden the query form, ORSEE shows only the recent experiments in an experiment list. State here how many experiments should be classified as recent. · Participant query: default size of random subset? : For the random recruitment ORSEE selects only a subset of participants. Fill in here the default size of the subset shown in the query form. · Laboratory: opening time? : State the opening time of your laboratories, i.e. the time when the first experiment may be started. · Laboratory: closing time? : When should the last experiment be finished each day? · Laboratory: max number of participants? : How much participants fit in your biggest laboratory? 47
· Laboratory/Experiment Session: default number of participants? : State here the default number of participants for an experimental session to be used on the session creation page. · Experiment session: max number of reserve participants? : What is the maximal number of reserve participants to be invited for an experimental session? · Experiment Session: default number of reserve participants? : What is the default number of reserve participants? · Experiment Session: max duration in hours? : What is the maximum time an experimental session should last? · Experiment Session: duration minute steps? : For session duration: which minute steps should be allowed in the form? · Experiment session: default duration? : What is the default session duration time? · Experiment Session: registration end: max hours before session? : State here the maximum number of hours before the session time the registration period should end. · Experiment Session: registration end: steps for hours before session? : In which steps the number of hours for registration end should be shown on the session edit page? · Experiment Session: registration end: default hours before session? : Fill in here the default end time of the registration period. · Experiment Session: default for "send session reminder on"-condition? : Which is the default rule applied to a session when checking if the session reminder should be sent out. For an explanation of these rules please refer to Section 5.1.2 on page 24. · Experiment Session: send session reminder email: max hours before session? : State here the maximum number of hours before the session starts the session reminder e-mail should be sent. · Experiment Session: send session reminder email: steps for hours before session? : In which steps the number of hours for the session reminder should be shown on the session edit page? 48
· Experiment Session: send session reminder email: default hours before session? : Fill in here the default time for sending session reminders. · Experiment Session: "session start"-field: years backward? : On the session creation page you have to specify the time of the session, including the year. Since we cannot show all years since 0 B.C., state here the number of years the list should go backward from the current year. · Experiment Session: "session start"-field: years forward? : State here the number of years the list should go forward from now. · Participant form: "begin of studies"-field: years backward? : The same as the option above, but for the 'begin of studies' field in the participant form. · Participant form: "begin of studies"-field: years forward? : See above. 5.11.5 Default Mails In this section you can edit the texts for all e-mails sent out by the system as well as the default e-mail text for experiment invitations. You will have to state the texts for all languages installed. You will see a list of all e-mail texts in all installed languages. To change a text, follow the appropriate 'Edit' link on the right. In the texts you may use some variables explained below. All variables are enclosed by '#'. The following variables can be used in almost all e-mails to registered participants, we call them subject variables below: · #lname#: Last name of participant · #fname#: First name · #gender#: Gender · #field of studies#: Field of studies · #profession#: Profession · #begin of studies#: Year of begin of studies · #email#: E-mail-address · #phone number#: Phone number The currently used e-mail texts are: 49
· admin calendar mailtext: The body text of the regular PDF calendar email sent out to experimenters. · admin mail footer : A footer which should be attached to the text body of all e-mails sent to administrators/experimenters. · admin participant statistics mailtext: The text before the statistics listing in the regular subject pool statistics e-mail. · admin registration notice: The text for the e-mail sent out to experimenters when the registration period for their experiment has been expired. Variables: #experiment name# - the internal name of the experiment, #session name# - the date and time of the session concerned, #status# - the registration status (enough or not enough?), #registered# - the number of registered participants for that session, #needed# - the number of needed participants, #reserve# - the number of planned reserve participants. · admin session reminder notice: The e-mail text for notices to experimenters that the session reminder e-mail was sent to registered subjects. Variables: #experiment name# - the internal name of the experiment, #session name# - the date and time of the session, #nr participants# - the number of registered participants, #disclaimer# - possibly empty, an explanation if the session reminder e-mail was not sent (e.g. too few registered participants). · default invitation internet: Invitation e-mail template for internet experiments. Currently not used in ORSEE 2.0, but will be used in later versions. · default invitation internet: Invitation e-mail template for internet experiments. The final text of the invitation e-mail can be edited when sending out the e-mails from the experiment's main page. Variables: subject variables, #link# - a link to the registration page for the participant. · default invitation online-survey: Invitation e-mail template for online surveys. Currently not used in ORSEE 2.0, but will be used in later versions. · public experiment registration: A confirmation e-mail sent out when a participant has registered for an experimental session. Variables: subject variables, #experiment# - the public name of the experiment, #session# 50
- the date and time of the session, #laboratory# - the name of the laboratory where the session will be conducted, #location# - the address of the laboratory. · public mail footer : A footer which will be attached to all e-mails which are sent to registered participants. Variables: #edit link# - a link to the personal data edit page for the participant. · public noshow warning: The text of the warning message sent out to participants who did not show up for a session they had registered for. Variables: subject variables, #experiment name# - the public name of the experiment, #session date# - the date and time of the session, #lab name# - the name of the laboratory where the session has been conducted, #max noshows# - the number of no-shows after that a participant will be automatically excluded from further registrations. · public participant exclusion: The e-mail informing participants that they are automatically excluded from further registrations for experiments. Variables: subject variables, #number noshowup# - the number of times the participant has not shown up. · public session reminder : The text of the session reminder e-mail. Variables: subject variables, #experiment name# - the public name of the experiment, #session date# - the date and time of the session, #lab name# - the name of the laboratory where the session has been conducted, #lab address# - the address of the laboratory. · public system registration: The e-mail which will be sent out when a subject has registered in the system. It should include the data entered by the subject at the registration page as well as a request to confirm the registration. Moreover a small e-mail footer should be included, since the public mail footer will not be attached to this e-mail. Variables: subject variables, #invitations# - the type of experiments the subject has subscribed to, #registration link# - the link which the subject should follow to confirm his registration. 5.11.6 Default Texts In this options section the default texts for online surveys and internet experiments can be defined. However, since surveys and internet experiments are not implemented in ORSEE 2.0 yet, this section is not of interest by now. 51
5.11.7 Experiment Classes For recruitment purposes, experiments in ORSEE are organized by types of the recruitment procedure (see section below). Additionally experiments can be organized by topic in so-called 'experiment classes'. This could be for example 'dictator game', 'public good game', 'others' etc. At the experiment edit page you can assign an experiment to a specific class. These classes can be used to search in the list of old experiments for experiments or to exclude participants who have participated at experiments of that class. At the page you will see the list of already defined experiment classes. Use the 'Create new' button to add a new class and the 'Edit' links to change to existing ones. You will have to specify the description/name of the class for each language installed. 5.11.8 Experiment Types Experiment types are used to organize experiments by type of the recruitment procedure. ORSEE distinguishes between external and internal experiment types. Internal types are nothing else than types of experiment organization and conduction. They are depended on the ORSEE software version and are used internally to organize the recruitment procedure. For example, for 'laboratory' experiments we need to create sessions, invite participants which then can register for that sessions. For online surveys we need to create the questionnaire with questions, items, default answers and so on, and invited participants can directly participate at the survey over the internet. External types are types for which participants agreed to get invitations for. For instance, the external type 'video experiments' is considered internally as 'laboratory experiment' (see examples below). An external type can be assigned to more than one internal types, and more than one internal types can be assigned to one external type. Two examples: 1. "Video experiments" are organized like laboratory experiments, so their internal type is "laboratory". However, we require participants to agree explicitly to be invited for laboratory experiments. Thus, we create a new external type, which links to the internal type "laboratory", additional to the external type "(Normal) Laboratory experiments". 2. We have the two internal types of "online-survey" and "internet" experiments. Internally, these two types are treated differently. (Although the "internet" type is not implemented yet.) But to the participants, they 52
are the same, because both are simply conducted over the internet. Thus, we create one external type "Internet Experiments", which links to both internal "online-survey" and "internet". Participants who agree to get invitations for "Internet Experiments" will be considered in both types of internal experiments. On the (external) experiment type creation/edit page, you are asked for the internal name of the type and a short description. Then you have to state, to which internal experiment types this external type translates to. Answer the question: To which of the internal types participants subscribed to this external type will be invited? After that you have to fill in the name of that type for all languages installed in the system. Use the 'Change' button to the data. 5.11.9 Laboratories ORSEE is capable to manage multiple laboratories. On the session creation page you will have to specify the laboratory where the session will be conducted. To make ORSEE work correctly you have to define at least one laboratory. Use the 'Create new' button to add a laboratory, and the 'Edit' links to change existing ones. On the edit page you will have to fill in an item name (which is used only internally in the database and should consist of lowercase letters and underscores only), the laboratory name and it's address for all languages installed. Note, that the first line in the laboratory text field is considered as the laboratory name, and the rest as the laboratory address. Both are used throughout the system. 5.11.10 Professions and Fields of Studies When registering with the system, subjects get a participant form to fill out. In this form they are asked for their main field of studies and/or profession (depending on the subject pool they self-selected in). In this area you create and administer the professions and Fields of study which will be shown in the appropriate selection list. The following description is for professions, but holds true as well for fields of study. On the overview page you get a list of all professions known by the system, in all languages installed. Using the 'Create new' button you can add a new profession, and following the 'Edit' links you may change a profession entry. On the edit page, you simply have to state the profession's name in 53
all languages installed in the system. To delete a profession, use the 'Delete' button on the edit page. 5.11.11 Sub-Subjectpools Sub-Subjectpools (or 'subpools' called in ORSEE slang) are a mean to organize your subject pool in groups. For each subpool you may set some options for the corresponding participant form. Participants self-select to these subpools by choosing a self-description in the first step of the system registration process. The overview page shows the subpool names, description and the number of subjects assigned to the pool. When creating or editing a subpool, you have to make the following statements: · Name: State here the (informative) name for the subpool. It will be used only in the administration area. · Description: A short description of the subpool for internal use. · Registration page type: State here if participants who self-selected to this pool should be asked for their main field of studies or their profession or should have the option to fill in one of them. · Can request invitations for : Select the external experiment types (see Section 5.11.8 on page 52) to which participants who self-selected to this subpool may subscribe to get invitations for that type. Select at least one experiment type. · Show subpool at registration page: Declare if the self-description (see below) of that subject pool should be shown at the registration page. · Registration page options: For each language installed on the system you have to fill in here a sentence which serves as a self-description for that subpool (see also the examples below). For example, in Jena there are four subpools maintained: · Laboratory Students: Registration page type - 'student', invitations for laboratory, internet and online surveys, show at registration page - yes, self description - I'm a student and I can participate at laboratory experiments in Jena. · Internet Students: Registration page type - 'student', invitations for internet and online surveys, show at registration page - yes, self description - I'm a student, but I cannot come to Jena. 54
· Laboratory Others: Registration page type - 'working', invitations for laboratory, internet and online surveys, show at registration page - yes, self description - I'm not a student, but I can participate at laboratory experiments in Jena. · Internet Others: Registration page type - 'working', invitations for - internet and online surveys, show at registration page - yes, self description - I'm not a student and I cannot come to Jena to participate at laboratory experiments. There is one subpool named 'non specified' created by default. This subpool with id 1 will never be shown at the registration page. When you decide not to use sub-subjectpools at all, simply leave it as it is. If you want to use this feature, create at least one own subpool and declare it as the default subject pool in the 'general settings'. If there is only one own subject pool declared to be shown at the registration page, then the first step of the registration process (the self-selection) is skipped and the new participant is immediately assigned to this subpool. For more than one own subpool marked to be shown on the regsitration page, the page shows a list of the self-descriptions and asks for selecting one of them. When you delete a subpool using the 'Delete' button on the subpool edit page, you have to state to which other subpool it's subjects should be moved. 5.11.12 FAQs 'Frequently Asked Questions' sections (FAQs) are commonly used to reduce the e-mail support traffics to users. The FAQs and their answers you create in this section are shown at the FAQ section at the public area of ORSEE. The overview shows the questions in all languages of the FAQs registered in the system together with the number of persons who voted that this FAQ answered their question. In the public area the questions are sorted by this criterium. On the creation/edit page for a FAQ simply fill in the question and the answer to it for all languages installed. 5.11.13 Help ORSEE comes with an online help system. At some places a small 'help' link leads to a page explaining the item concerned. You may change these help texts in the help options area, again for all languages installed. However, creating new help items is a developer task, since it is necessary to change the program code to add new help links in the system. If you find it necessary to do that, 55
please notify the author with the corresponding place and texts, so that it can be included in the next ORSEE version. 5.11.14 Public Content In this section you can edit the site specific content at the Welcome (mainpage welcome), Rules (rules), Privacy Policy (privacy policy), Impressum (impressum), Contact (contact) and Temporarily Closed (error temporary disabled) pages in the public area and on the Welcome page (admin mainpage) in the experimenter area. The overview page lists the content name and the content itself in the installed languages. Following the 'edit' links you may change the content. Note, that you are allowed to use HTML tags to format the content. 56
6 Installation and command line configuration 6.1 Prerequisites You need · a webserver like Apache · PHP >= 4.2.2 installed to be interpreted by the webserver and to be run on command line, on newer versions with the following separate modules: GD module (php-gd), MySQL module (php-mysql), Session module (phpsession) · the MySQL database >= 4.02 · the cron daemon (a must on all unix systems) · the webalizer log file analyzer All of this programs are part of standard Linux distributions like RedHat, SuSE and others. However, ORSEE should run on Microsoft Windows Servers as well, since Apache, PHP and webalizer run there as well. Only for cron you will have to find a substitute. I'm sure there are some. 6.2 Download and Installation The installation of ORSEE is straightforward: · Download, · unpack, · import database, · edit main configuration file, · and done. You may download the systems' source code from our website at www.orsee.org. Unpack the tarball in your webservers' path. Then change to the install directory. Here you find the file INSTALL.howto, which gives you detailed instructions on how to install the system. Normally, for a new installation you first have to create your MySQL database and to import the database structure from the file install/install.sql by typing 57
mysql databasename -uusername -ppassword < install.sql Thereafter, you just have to edit the file settings.php as described in the section below. Right after you can login into the system using the username orsee install and the password install. The URL to the admin area of your ORSEE installation should be http:///admin Then edit all the options as described in the previous sections, install the cron job and configure webalizer (see below), and you are ready to start. If you upgrade your system from an older version of ORSEE, please refer to the file UPGRADE.howto which includes detailed directives on how to upgrade to the current version from all previous versions. What to do when upgrading depends on the version gap between your old and the new version. Two procedures are used: · When the database structure has not changed between the versions, you simply install the new version and take the database configuration from the old version. You may have to copy your styles to the new installation and to upgrade your languages. · When the database structure has been changed between the versions, install the new ORSEE system with a new database as described in this section, and then run the data import procedure from the options menu. Here you have to select the old version number and most likely to give some information on how to deal with some of the old data. Then ORSEE reads in the data into the new database. The file CHANGELOG will keep track of all code and database changes between versions. 6.3 The Main Configuration File The major configuration file where you have to set up a couple of variables is /config/settings.php In the following we will explain the different variables. In PHP a variable is set by the command: $varname="value"; 58
The quotation marks are not required but should be used for text entries. $settings root to server: Here you have to specify the full path starting with / and ending with your webserver's document directory. $settings root directory: This is the relative path from your server's document root to the system's root directory. The string has to start with a "/" if ORSEE is located in a sub-directory, and to be empty if it resides in the document root itself. $settings server url: Fill in the URL to the document root directory of your webserver. Note, that to web address of the ORSEE system is calculated by concatenating the strings http://, $settings server url and $settings server url. $site database host: The host your database is running on. When on the same computer, it's 'localhost'. $site database database: The name of the database of your recruitment system. $site database admin username: The username to access the database. $site database admin password: The password to access the database. $site database type: The database type. Currently we only support MySQL, but there might be support for other RDBMS in the future. $site database table prefix: Per default, all tables in the ORSEE database start with the prefix 'or '. If you want to use different names, change the table names and the database and then set this variable to your new prefix. $settings stop admin site: Stating here "y" allows nobody to access the system's administration area via a web browser. This may be useful if you do database changes by hand. All other configuration can be done within the administration area of the system. 6.4 Regular Tasks - cron To enable the regular tasks feature in ORSEE, we need to install a so-called 'cronjob' which regularly runs a script which checks whether a task is due and starts it when required. Cronjobs on Unix systems are managed by the cron daemon and configured in a crontab file. To list your currently installed crontab, type crontab -l If you have no cronjobs installed yet, you may just install the crontab file which comes with ORSEE in the install/ directory named crontab-for-orsee 59
by typing crontab crontab-for-orsee If there are cronjobs installed already, and you don't want to miss them, just copy the lines from the file crontab-for-orsee, call the editor vi on your crontab by typing crontab -e and paste in the lines at the end. Some vi commands: i - insert from here, ESC - leave insert mode, x - delete char, dd - delete line, :x - save and exit. The cronjob configured for ORSEE reads as this: Every 5 minutes, test whether the file admin/cron.php in ORSEE's system path is readable, and if this is true, change to the admin/ directory and run the cron.php script via PHP, but without any output. If you are familiar with cron, you may indeed change the job configuration to fit your needs. Note, that the cron job is running under the user name you have installed it. Thus, all files in your ORSEE system directory should be readable for that user. 6.5 Web Server Statistics - webalizer To generate web usage statistics, ORSEE may run the program webalizer on the access log file of your web server. The analyzer program webalizer is a standard package in most Linux distributions. To configure the webalizer output, ORSEE takes the configuration template from usage/webalizer.template, processes it and creates a webalizer.conf file in the usage/ directory out of it. The configuration in the default template should should be sufficient for most users. However, you may change the template file to fit your needs (check the webalizer documentation on that). The strings enclosed by '#' are variables which are replaced automatically by ORSEE when processing the file. You should not change the lines starting with 'HTML', because they make sure that the statistics can be accessed only from the administration area after authentication. The output of webalizer is written to the usage directory. As you may have noted, the system user who creates the webalizer statistics needs to have write access to the usage directory. That is, when the statistics are created via the corresponding regular task, the user running the cron job (see section above) has to have write access. However, when running the statistics generation manually from the Options/Regular Tasks area, the 60
user the web server is running under needs to have write access. In the best case you use the same user for both tasks (cron and webserver). If not, make sure that the directory is writable for both (or at least for the cron user). 6.6 Styles In ORSEE you may change the look and feel of the HTML output by employing styles. You may use different styles for the public and the administration area. All styles are located in the styles/ directory in ORSEE's system root. Every subdirectory represents a style. To create a new style, just copy one of the old ones and edit the new directory. cd styles cp -r oldstyle newstyle In the following we first describe the files which are taken for HTML output, and second we discuss the color configuration. 6.6.1 HTML Framework For each style, there are the following files which deal with HTML-Output: Stylesheets: We use standard CSS stylesheet files here. · stylesheet.css: This is the standard file which will be linked in the section of almost all HTML pages. · stylesheet print.css: This stylesheet is used for pages prepared for printing, for instance the list of future experiments a participant has registered for. HTML headers and footers: The idea here is the following: When generating a HTML page, ORSEE first processes some code, then prints out the tag, the section and the starting tag. Right after it will include the html header.php file and print it out. The header file should include the navigation list (see below). Then, ORSEE generates it's output depending on the current function. Right after, it includes the html footer.php file, and concludes with printing the closing tag. The files concerned are: · html header.php: The HTML header for standard pages. · html footer.php: The standard HTML footer file. 61
· help html header.php: The HTML header file for pages shown in a small window, as help texts and FAQ answers. · help html footer.php: The footer for small windows. In the first two files you will have to use once a PHP command: the call of the navigation generation function. The function will print a table with the navigation to the output. Is is called by The variable orientation may have the values 'vertical' or 'horizontal', with 'vertical' being the default. The boolean variable icons can be 'true' or 'false', indicating whether small icon graphs from the icons/ directory should be shown in the administration area. The default is 'true'. When variables are not given, the defaults are taken, i.e. a call of produces a vertical navigation with icons. Other files and directories: The file colors.php is described in the next section. All other files can be images used on your page. To link them from your HTML headers and footers you should use relative links like ../style/mystyle/myimage.jpg That makes your style generic to be used also on the places. The directory icons/ contains image files in PNG format. These icon images are used for navigation in the administration area. The images starting with 'lang ' are used as flags for language links on the Welcome page, if existent. 6.6.2 Colors To specify the colors used in your style, edit the colors.php file in your style folder. The format of the file is as follows: · Each line is interpreted as one color configuration. Empty lines or lines starting with # are ignored. · Color definition lines start with the color item name, followed by a ':' and then the color value. Whitespaces are trimmed at both ends. · Color values might be hexadecimal numbers like '#0000FF' or color names like 'blue'. 62
· For some color specifications you will have to give a list of color values, separated by ','. These lists are specified in the file. · In the color files shipped with ORSEE, the color items are explained throughout the file. 63
7 Tips and Tricks 7.1 Two definite subgroups in one session Imagine you intend to conduct a gender experiment and to invite half boys and half girls. To do this within the system, you create two different experiments: one for the male and one for the female subjects. Now you register every session twice in each of the both experiments. Remember to specify only the half of the needed subjects in each created session. You can ignore the error message telling you about another experiment at the same time in the same laboratory. Now, indeed, you assign only girls to the first and boys to the second experiment, and voґila: you have to definite subgroups. 7.2 Handle newcomers on the lab's door Imagine there is a person at the lab's door, who is not registered for your experiment. But at the same time you have so many no-shows that you need another subject. Now, do the following: 1. Don't forget to save the data you already filled in. 2. Let the person show his id card. Search the participants data for his name by going to Participants/Edit participants. 3. If you find him, copy his e-mail address, go to the assign page of your experiment and assign the new person to your experiment. Then click on Assigned participants at the end of your experiment's main page and register the person for your session. 4. If you don't find him, create a new participant. At the bottom of the page, choose your session. 5. Now the person should be listed at your session's participant list. 64
References Ball, S. B. & Cech, P.-A. (1996), `Subject pool choice and treatment effects in economic laboratory research', Research in Experimental Economics 6, 239­292. Binmore, K. (1994), Game Theory and the Social Contract, Camebridge, MA: MIT press. Camerer, C. (1996), Rules for Experimenting in Psychology and Economics, and Why They Differ, in W. Albers, W. GuЁth, P. Hammerstein, B. Moldovanu & E. Van Damme, eds, `Understanding strategic interaction: Essays in honor of Reinhard Selten', Berlin: Springer Verlag, pp. 313­327. Camerer, C. F. & Hogarth, R. M. (1999), `The effects of financial incentives in experiments: A review and capital-production framework', Journal of risk and uncertainty 19(103), 7­42. Coutts, L. M. & Schneider, F. W. (1975), `Recruitment of experimental subjects', Perceptual and Motor Skills 41, 142. Davis, D. D. & Holt, C. A. (1993), Experimental Economics, Princeton, NJ: Princeton University Press. Dixon, P. N. (1978), `Subject recruitment incentives, personality factors, and attitudes towards experimentation in a simultaneous intentional-incidential learning task', Journal of General Psychology 99, 99­105. Eckel, C. C. & Grossman, P. J. (2000), `Volunteers and Pseudo-Volunteers: The Effect of Recruitment Method in Dictator Experiments', Experimental Economics 3, 107­120. Friedman, D. & Sunder, S., eds (1994), Experimental methods : a primer for economists, Camebridge: Camebridge University Press. Harrison, G. (1989), `Theory and misbehaviour of first-price auctions', American Economic Review 79, 749­762. Hertwig, R. & Ortmann, A. (2001), `Experimental practices in economics: A methodological challenge for psychologists?', Behavioral and Brain Sciences 24(3), 383­451. Jackson, J. M., Procidano, M. E. & Cohen, C. J. (1989), `Subject pool sign-up procedures: a threat to external validity', Social behavior and Personality 17, 29­43. 65
Jung, J. (1969), `Current practices and problems in the use of college students for psychological research', The Canadian Psychologist 10, 280­290. Kagel, J. H. & Roth, A. E., eds (1995), The Handbook of Experimental Economics, Princeton, NJ: Princeton University Press. Palfrey, T. & Porter, R. (1991), `Guidelines for Submission of Manuscripts on Experimental Economics', Econometrica 59, 1197­1198. Rabin, M. (1998), `Psychology and Economics', Journal of Economic Literature 36(1), 11­46. Rabin, M. (2002), `A Perspective on Psychology and Economics', European Economic Review 46, 657­685. Rosenthal, R. & Rosnow, R. (1975), The Volunteer Subject, New York: John Wiley & Sons. Rush, M. C., Phillips, J. S. & Panek, P. E. (1978), `Subject recruitmemt bias: The paid volunteer subject', Perceptual and Motor Skills 47, 443­449. Saunders, D. M., Fisher, W. A., Hewitt, E. C. & Clayton, J. P. (1985), `A Method For Empirically Assessing Volunteer Selection Effects: Recruitment Procedures and Responses to Erotica', Journal of Personality and social psychology 49, 1703­1712. Senn, C. Y. & Desmarais, S. (2001), `Are Our Recruitment Practices for Sex Studies Working Across Gender? The Effect of Topic and Gender of Recruiter on Participation Rates of University Men and Women', Journal of Sex Research 38, 111­117. Silverman, I. & Margulis, S. (1973), `Experiment tilte as a source of sampling bias in commonly used 'subject pool' procedures', The Canadian Psychologist 14, 197­201. Smith, V. L. & Walker, J. M. (1993), `Monetary Rewards and Decision Cost in Experimental Economics', Economic Inquiry 31, 245­261. Wagner, M. E. & Schubert, D. S. P. (1976), `Increasing volunteer representativeness by recruiting for credit or pay', The Journal of General Psychology 94, 85­91. Zwick, R., Erev, I. & Budescu, D. (1999), The Psychological and Economical Perspectives on Human Decisions in Social and Interactive Contexts, in 66
R. Zwick, I. Erev & D. Budescu, eds, `Games and Human Behavior: Essays in honor of Amnon Rapoport', New Jersey: Lawrence Erlbaum Associates, pp. 3­20. 67

File: the-online-recruitment-system-orsee-20-a-guide-for-the-organization.pdf
Title: orsee_2_0_2_041113.dvi
Published: Sat Nov 13 15:28:49 2004
Pages: 67
File size: 0.68 Mb


, pages, 0 Mb

Augustana XXV, 24 pages, 0.12 Mb

, pages, 0 Mb

RESEARCH OF NOTE, 6 pages, 0.58 Mb
Copyright © 2018 doc.uments.com