clinical staff, handover, information transfer, Auckland City Hospital, patient safety, Auckland, Heuristic evaluation, clinical information systems, information technology, usability evaluation methods, Usability Evaluation, patient management system, evaluation method, Evaluation approaches, evaluation methods, electronic whiteboard, Philip Carter, clinical improvement, Clinical Handover Improvement Project, clinical improvements, Lisa Cunningham Auckland District Health Board Park Road, David Kaufmann, Auckland University of Technology Private Bag, process improvement, process improvements, Survey questions
A Usability Perspective on Clinical Handover Improvement: CHIPS with Everything ? David Kaufmann, Philip Carter, and David Parry School of Computing and Mathematical Sciences, Auckland University of Technology Private Bag 92006, Auckland 1140, New Zealand [email protected]
, [email protected]
Emma Parry, Lucy Westbrooke, Victoria Carlsen, Katie Groom, and Lisa Cunningham Auckland District Health Board Park Road, Grafton, Auckland 1001, New Zealand Abstract Support for clinical handover remains a challenge for Health Informatics. Obstetric Handover is an important multi-disciplinary activity that can have important consequences for patient safety. Although IT systems can be used to support handover, the improvement of handover is not a purely IT-based problem. This project took place in the context of a clinical improvement process designed to formalise and improve handover in a busy delivery unit. This study used investigative tools from the usability domain in order to understand the usability requirements of a complete socio-technical system - the handover process. This work demonstrated the feasibility of using such tools and illustrated potential usability problems and solutions in the clinical handover process. Changes were made in IT systems, the organisation of the handover and the Physical Environment
. Evaluation of the modified approach is being conducted, in the light of some usability issues already discovered. Usability evaluation tools appear useful and practical methods for assisting with clinical improvements in a complex and demanding area. 1. Introduction Clinical handover remains a problematic area for Quality Improvement
. The Clinical Handover Improvement Project (CHIP) was initiated at Auckland City Hospital (ACH) in order to improve this process, reduce the occurrence of adverse events and increase efficiency. The project includes a number of methods to identify and assess methods for clinical process improvement; An intervention based on the OSSIE guidelines , evaluation of the quality of information transfer
via a questionnaire and video recording, pre and post intervention and usability assessments of the software and environment of use as part of the handover process. 1.1. Clinical Handover According to the Australian Commission on Safety and Quality in health care
, Clinical Handover is "the transfer of professional responsibility and accountability for some or all aspects of care for a patient or group of patients, to another person or professional group on a temporary or permanent basis." A professional group can consist of various clinical staff, such as midwives, nurses, doctors and/or allied health professionals. Clinical Handover provides a significant risk for patient safety as it could potentially lead to adverse effects such as discontinuity of care or administration of the wrong medication. In the last 15 years many of the working schedules of Resident Medical Officers have changed. Shorter hours and a move to shift based rosters has resulted in increased numbers of doctors who are not so well known to each other and more Clinical Handover episodes during any given time-frame. These changes have the potential to increase the chance of Adverse Events occurring. A survey conducted by McCann, McHardy & Child  at Auckland City Hospital found that 72 out of 73 medical practitioners had experienced at least one clinical problem due to poor clinical handover in the three months preceding the survey. As a result, much work has gone into the improvement of the clinical handover and among others, process improvements and utilisation of software has been suggested. In addition, because of the complexity and potential for serious error associated with clinical handover, any software-supported approach must have excellent usability.
1.2. Use of OSSIE guidelines The OSSIE Guide to Clinical Handover Improvement  provide a set of empirical guidelines for the improvement of the clinical handover process. These guidelines emphasise clinical involvement and documentation of current issues in the process, in order to engage clinical staff. 1.3. Background to CHIPS Handover continues to be a poorly performed process, in clinical care and research in this area is extremely active Poor handover and the associated clinical risk to patients have been identified as an increasing problem throughout all specialities in medicine. There has been a great deal of literature generated on this topic in the past 5 years. The topic has been highlighted with the change in working hours of Resident medical officers (RMOs) and hence subsequent multiple shift changes reducing continuity of care and necessitating frequent handover of what can often be complex and extensive information. A recurring theme in the literature is the lack of appropriate "tools" for handover . A number of working directives and subsequent guidelines have been generated to aid in standardising handover, including the SHARING pro forma established for the dynamic environment of the labour ward and associated units . There are three main areas that are being addressed in the CHIPS project. 1.3.1. The Handover Environment A 2010 study in neurocritical care by Lyons et al demonstrated that distractions affect information transfer and cause inefficiency .There are numerous studies supporting the use of standardised tools for handover. McCann et al showed that staff involved in handover highlighted the importance of a set location,standardised handover sheet and training related to handover . While Jorm et al showed substantial improvement in the handover of outstanding jobs and overall patient data with the introduction of a standardised proforma . Porteous et al have developed a clinical handover checklist which goes by the acronym iSoBAR (identify, situation, observations, background, agreed plan, read back) . This, and other variants, has been incorporated into the OSSIE guide to clinical Handover Improvement generated by the Australian Commission on Safety and Quality in Healthcare . 1.3.2. Information Technology
Support The increasing use of electronic notes highlights the availability of potential methods to support information transfer. A survey by Hannan et al into the sustainability of medical morning handover showed a strongly positive result to the introduction of IT facilities to the morning handover meeting . Studies also demonstrate that verbal handovers or verbal with note taking are less effective at complete information transfer and retention than printed hand outs with relevant patient information [9, 10]. 1.4. Aim of this research The study aims to produce a set of usability requirements, that extend beyond the user interface of the IT system that can be used for guiding future development of computer-supported handover. It is hoped that this work can identify successful approaches to improving clinical practice
in the area of handover and also be useful in identifying appropriate methods for the evaluation of computer-assisted handover systems. This work uses some of the tools that are familiar (but not exclusive) to usability evaluators including observation, heuristics and requirements gathering approaches, in order to study a clinical activity that includes computer and non-computer based operations. Sociotechnical approaches  drive the overall process with the realisation that handover is a network process, that is collaborative and fluid , which must be investigated using empirical methods. As an illustration, consider that the clinical handover in this case can involve around 20 people and therefore 190 separate relationships between participants, some of which will not be mediated via the computer system
. Thus a solely Computer-centric analysis would never adequately appreciate the complexity of the domain and the interpersonal relations
hips involved. We hope to show that this approach is in fact effective and simple in practice to both the researchers and clinical staff. The overall study protocol and timeline is shown in Figure 1.
Figure 1 - Relationship between this research and the CHIPS implementation 1.5. Usability Evaluation / Observations A large number of different usability evaluation methods have been used in the context of clinical information systems
, giving a clear testimonial of the complexity and differences in view. Generally speaking, each evaluation method can be classified into one of two categories. On one side, there are "user based" approaches which, to some degree, all involve prospective users in the evaluation. On the other hand, there are "expert based" approaches which solely rely on usability experts and do not involve users. Despite the large number of proposed Usability Evaluations methods, practitioners seem to have a preference for a small number of them. Various contemporary papers (e.g.) indicate that Heuristic evaluation and Usability testing are two of the most frequently applied methods, in both practice and research. Other sporadic but less frequent applied methods are Questionnaire, Interviewing and Cognitive walkthrough. 2. Materials and Methods 2.1. The CHIPS intervention Two procedural approaches were modified from the OSSIE approach. SHARING (Figure 2) guides the Clinical Handover Process. This includes: Staff introductions, information about unit bed occupancy and clinical areas where information should be made available to the handover including outlying patients who might otherwise be omitted. Figure 2 SHARING: Protocol for running the handover
Figure 3 IBAAAR: Information about each patient The second approach was IBAAAR (Figure 3). This is the information that should be handed over about each patient. These lists have also been printed onto a small laminated card, which can be attached to the ID lanyard, and have been issued to all clinical staff working in this area. In effect, SHARING, deals with who should speak, and IBAAAR deals with what they should say. These points were modified from previous approaches by the CHIPS steering group. The aim of the intervention is to demonstrate an improvement in handover efficiency and effectiveness and hence reduce clinical risk. 2.1.1. Setting The setting of the project was National Women's Health , Auckland City Hospital. The Doctors handover occurs at 08:00 and 22:00 on the labour ward. The handover is timetabled to take 30 minutes. 2.1.2. Participants Many people are involved in the handover process. The minimum expected number of participants are four House Surgeons, four Obstetric and Gynaecology registrars (two from the outgoing shift and two from the incoming shift) and two specialists, In addition the Charge midwife of Delivery Unit, Clinical Midwife Associate (senior midwife available for the whole of Women's Health), one or two anaesthetists and the Charge theatre nurse. In addition obstetric physicians and private specialists may also need to be present and there are often medical students. 2.1.3. Software System A "whiteboard" application (shown in Figure 4) is used to support the handover process. However this application obtains data from a patient management system and is not primarily designed to support workflow or handover. In many cases staff use printouts from this system to record handover information, by writing relevant information by hand on them. Major redesign of this system is currently not in scope for the project , but minor changes e.g. reordering of fields are possible. 2.2. Evaluation approaches Multiple methods were used in the study and a triangulation approach was used in order to attempt to add rigour to the findings  .The chosen evaluation methods and their application used were; User observation, stakeholder interviews, a survey and heuristic evaluation. Note that, at this stage, a formal heuristic application has not yet been applied to the whiteboard software.
Figure 4 - The current software approach 2.2.1. Method 1 User Observation Ten user observations have been conducted in both the initial as well as the updated handover environment. The procedure was informed by Literature as well as three pilot observations. Due to the real world character of the environment and lack of subject specific knowledge (i.e. medical knowledge), the chosen evaluator role was the one of an "onlooker". This meant that there was no direct interaction with the participants during use (e.g. "Think aloud" technique). Due to the advantages of video recording, all observed handovers have been recorded on video. The recording of the handover also meant that the evaluator did not necessary had to attend in person. However, the handovers have been attended in person whenever possible as this allowed for the recording of additional observations that were outside the angle of the video camera (e.g. events that occurred on the screen of the handover system or reactions of participants in the background who were not visible on the video). The video recordings were analysed using a 17 point check list both pre and post change implementation. 2.2.2. Method 2 Stakeholder Interviews A set of stakeholder interviews has been conducted with various handover participants. The aim was to interview at least one role each . The questions asked at the interviews were the eleven questions suggested by the Australian Commission on Safety and Quality in Health Care [1, pg 2, section 1-3] for the "Organisational Leadership" phase of a handover improvement project. One additional question has been added which particularly referred to the patient spreadsheet of the initial/updated handover system. The need for the additional question has been realised after the pilot stakeholder interview as the eleven existing questions were not suitable to obtain specific information about the current handover system. For that purpose, a printout of the handover system spreadsheet has been taken to the stakeholder interview. Since interview participation was on a voluntary basis and off working time (see constraint 1), it was crucial to keep the interview duration in the range of 5-10 minutes. This has been achieved by the restriction to the 11+1 questions and by voice recording the interviews, which did not require the interviewer to take notes. Also, enquiries have been kept to a minimum and were only used in case where it was absolutely necessary in order to understand the given answer. The interview questions are shown in Figure 5 below.
Stakeholder Interview Question 1.1) What is your definition of clinical handover? 1.2) What are the functions of clinical handover? 1.3) What do you think the transferring of responsibility and accountability during handover means? 2.1) Can you please discuss how handover is currently conducted in your department? 2.2) What do you think are the positive aspects of your current handover process? 2.3) What do you think are the negative aspects of your current handover process? 2.4) How do you think your current handover process could be improved? 2.5) What information do you require for continuity of patient care during your shift? 2.6) How could information technology help you with handover? 3.1) How did you learn how to do handover? 3.2) How do you think handover should be taught? 4) How could the current spreadsheet be improved? Figure 5 - Stakeholder interview questions
2.2.3. Method 3 Survey A written survey was conducted by Dr Victoria Carlsen pre- and post-intervention (e.g. change of handover environment, process and system). The survey has been handed out to the participants immediately after the handover and consisted of thirteen question which addressed the areas "information transfer", "environment", "process" and "handover system". Participants could enter their answers by placing a cross on a 10cm visual analogue scale where the two ends of the scale represented the opposite answer possibilities (e.g. Always vs. Never). The survey questions are shown in Figure 6 below. While the survey has been conducted by the CHIP programme and the questions have been tailored towards the aim of measuring the effectiveness of the applied changes, the collected data
could still give valuable information in regards to usability and therefore assist in the process of identifying usability requirements. Ethical approval was obtained from the Northern Region ethical committee with code NTX/11/EXP/031.
Survey Question 1) Are Delivery Unit/WA handovers run effectively? 2) Is all essential information on patients handed over? 3) Is patient safety compromised by the current handover process? 4) During handover do you know the other members of the team? 5) Is the current environment where handover takes place, the DU workroom, appropriate? 6) Would changing the environment help improve the handover process? 7) Is the current electronic whiteboard useful for handover? 8) Is the data on the electronic whiteboard relevant to handover? 9) Do doctors currently handover in a systematic and organised manner? 10) Would a pro forma be useful for DU/WAU handovers? 11) Are outliers appropriately handed over? 12) Who should attend handover? 13) Please add any comments regarding handover if you wish. Figure 6 - Survey questions
Very Effective Not Effective
Helpful Not Useful Relevant
Not Helpful Very Useful Not Relevant
No scale. List of eleven roles that
could be circled.
No scale. empty space
3. Results As the second part of the study had not yet taken place at the time of submission, the main results at present are those extracted from the initial video observation. Data from both sets of observations will be available at the conference. 3.1. Pre-intervention observations One of the main areas of observation was the identification of the "working surfaces" used by the participants. This refers to the sources of information used and the areas where information is recorded. 3.1.1. Working Surface (current) A critical dimension of usability can be described briefly as the `working surface'. The working surface is the medium (screen, whiteboard, paper) on which `work' is done. Work is some processing of information and this may involve interaction between people. Visibility, accessibility and control all have significant impacts on the success of the process. The use of a small computer screen
caused a number of `work arounds' that made the process opaque and broke the integrity of the data. There is extensive use of paper by various members of the handover as a means of providing an aide-memoire for the issues concerning patients to be handed over. Printed patient lists from the current electronic whiteboard are used as a surface where additional notes can be added. 3.1.2. Working Surface (future) It is doubted that a handover system can be the sole working surface. People need printouts and they will write notes on them. If the information is on a note sheet, then this becomes the working surface. If the working surface has to become the handover system, then it needs to be ensured that as much information is in there as possible. 3.1.3. Lack of information In many cases, there is critical and non-critical information that is not available in the handover environment. 3.1.4. Missing information in Handover system. Contact information and bed status are captured outside of the handover system on conventional whiteboards, and other specialised information including information about "outliers" is not available via the whiteboard. 3.1.5. Lack of process & attendance There is no formal process and the handover participants change from day to day. 3.1.6. Missing System Integration There were numerous cases where people had to retrieve information from a computer other than the handover system. 3.1.7. Information Presentation There were numerous cases where people had to point at the screen of the handover system. This indicates that the discussed information could not be highlighted. It became obvious that not all handover participants could read the information shown on the screen of the current handover system, which is displayed on a standard PC monitor. 3.1.8. Necessary disruptions The nature of a hospital means that people need to leave the handover in order to attend an emergency. It needs to be addressed how such disruptions are handled.
3.1.9. Unnecessary disruptions The current handover environment has many unnecessary disruptions (printer, other people in the office, phone calls etc.) 3.2. Usability requirements Usability requirements are non-functional requirements
of a software system, that identify what sort of user experience a system may provide . Lausen  Identifies a number of ways of representing such requirements, many of which are based on specific metrics. In our case we would be unlikely to choose extremely quantitative metrics, but see the usability requirements as a set of guidelines that include important heuristics that need to be observed and areas of potential poor usability identified. 4. Discussion This study is not yet complete as the post- intervention work has not yet been undertaken. However an number of questions and requirements have been identified, and a number of actions have taken place as part of the intervention. The handover process has been moved to a dedicated room with a large computer screen available. The SHARED/IBAAAR protocol has been implemented and continuing clinical education is taking place in its use. The whiteboard software has been modified to provide a greater range of information that is required under IBAAAR. One important change has been to identify "outliers" by using additional fields to identify patients that "belong" to women's health when not located within the women's health clinical area. It is common to find that projects do not follow a path that will allow formative usability studies to proceed in an textbook fashion  . In this work we have attempted to use a multidisciplinary approach and take approaches from the usability toolbox and other areas in order to identify problems and solutions to the usability of the handover process. This appears to have been successful, and this approach could be used in different areas. From the viewpoint of the usability assessment groups, one important discovery has been the relatively small amount of domain knowledge that has been required in order to study this process. The usability finding from observation and the usability requirements appear to be easily comprehended by the clinical staff, and they have found this helpful. Initial results appear to show that the CHIPS project is both popular with clinical staff and appears to be assisting in handover quality. Continuing use of non-IT based working surfaces is being encouraged, and systems should be able to support both paper-based and screen-based displays of data. 4.1. Future work There is a need for software to support real-world workflow, and multiple views of clinical data may well need to be added to existing clinical software. There may be value in the use of personalised "to-do" lists being added to users mobile phone
s and the use of novel working surfaces including RFID-enabled patient labels attached to physical objects such as individual notebooks. 5. Acknowledgments Thanks must go to the CHIPS team, the clinical staff of the Women's Health Department ACH and David Churchouse from the audio visual unit ACH. 6. References  Australian Commission on Safety and Quality in Health Care. OSSIE Guide to Clinical Handover Improvement. Sydney2010.  McCann L, McHardy K, Child S. Passing the buck: clinical handovers at a New Zealand tertiary hospital. NZ Med J. 2007;120(U2778).  Johnson JK, Barach P. Patient care handovers: what will it take to ensure quality and safety during times of transition? Medical Journal of Australia. 2009;190(11):S110-S2.
 Royal College
of Obstetricians and Gynaecologists. Good Practice No.122010 December 2010.  Lyons MN, Standley TDA, Gupta AK. Quality improvement of doctors' shift-change handover in neuro-critical care. Quality and Safety in Health Care. 2010 December 1, 2010;19(6):1-7.  Jorm CM, White S, Kaneen T. Clinical handover: critical communications. Med J Aust
2009;190((11 Suppl)):S108-9.  Porteous JM, Stewart-Wynne EG, Connolly M, Crommelin PF. iSoBAR-a concept and handover checklist: the National Clinical Handover Initiative. Med J Aust. 2009;190(11 Suppl):S152-6.  Hannan TJ, Bart S, Sharp C, Fassett MJ, Fassett RG. The sustainability of medical morning handover reporting: adherence in a regional hospital. Aust Health Rev 2010;34:325-7.  Piscioneri F, Chong GC. Surgical handover in a tertiary hospital: a working model. Aust Health Rev. 2011;35(1):14-7.  Bhabra G, Mackeith S, Monteiro P, Pothier DD. An experimental comparison of handover methods. Ann R Coll Surg Engl. 2007;89(3):298-300.  Berg M. Patient care information systems and health care work: a sociotechnical approach. International Journal of Medical Informatics. 1999;55(2):87-101.  Kushniruk AW, Patel VL. Cognitive and usability engineering methods for the evaluation of clinical information systems. Journal of Biomedical Informatics. 2004;37(1):56-76.  Tan W-s, Liu D, Bishu R. Web evaluation: Heuristic evaluation vs. user testing. International Journal of Industrial Ergonomics. 2009;39(4):621-7.  Jick TD. Mixing Qualitative and quantitative method
s: Triangulation in Action. Administrative Science Quarterly
. 1979;24(4):602-11.  Lauesen S, Younessi H. Six Styles for Usability Requirements. Proceedings of REFSQ'98,; Universitaires de Namur: Presses Universitaires de Namur,; 1998.  Saul G, Bill B. Usability evaluation considered harmful (some of the time). Proceeding of the twenty-sixth annual SIGCHI conference on Human factors in computing systems; Florence, Italy: ACM; 2008.