Annotated Bilbiography


Perceptions of the effect of information and communication technology on the quality of care delivered in emergency departments – Callen et al.


“Perceptions of the effect of information and communication technology on the quality of care delivered in emergency departments” by Callen et al. is a qualitative study investigating physicians and nurses attitudes on EDIS in four hospitals in Australia. The same EDIS was used at each hospital, and it was integrated with the larger hospital system, so previous patient data was available to the new system.

Close to 100 people participated in the study, using a combination of observation, interviews, and focus groups. Prior to the EDIS implementation, there had been a standalone system, which was not integrated with any other clinical systems. There seemed to be consensus that clinicians were using the system collaboratively, and that it had improved information sharing. Many benefits were noted, and overall the EDIS was seen as a success. The main negative perception was one that is commonly noted in other literature; that data entry and data capture methods need to be improved.

Topic Influence

One of the more interesting findings was that clinicians perceive the electronic record to be less complete. Due to data entry challenges, not as much information is actually being recorded. So while there was acknowledgement that data retrieval and information search added previously unknown efficiencies, some information was being lost due to the inconvenience of documenting it.

Another point raised regarded the difference in how people use individual technology (such as a personal tablet) versus collaborative technology (such as an EDIS). Collaborative technology is seen as successful if it raises an individual’s understanding of how others in their work environment perform that work. The surrounding social system can create incentives to either help or hinder that goal.

In this system, nurses were able to access the same data as physicians, which is not always the case. It was believed that having previous hospital records, combined with the fact that all the providers had access to this data, led to faster and more accurate diagnoses.

Somewhat paradoxically, some believed that the electronic record used more complete sentences. While data entry was a problem, when people did finally enter data, they wrote more prose and used fewer abbreviations. It was not clear how these tradeoffs affected care.

As in other studies, it was noted that paper was still widely being used, creating a hybrid system. There were gaps in the work processes that were not being served electronically. Therefore, clinicians were creating workarounds, which did create redundant data and led to task duplication.

Going paperless at the emergency department – Vezyridis, Timmons, and Wharrad


“Going paperless at the emergency department” by Vezyridis, Timmons, and Wharrad is a qualitative study strictly from a nursing point of view. Since their interaction both with patients and the medical system differs from that of physicians, they provide a different perspective. The study site sees approximately 400 patients a day, and claims to be one of the busiest emergency departments (ED) in the United Kingdom.

The article looked at EDIS implementation from a variety of angles. First, it compared ED operations before and after. Then it looked at how EDIS affected point of care. Third, it raised the concern of overdependence on the system. Finally, it looked at how these clinicians created combined paper/digital systems. Unlike most other research, transitioning to an entirely paperless system was not perceived as an ultimate end goal. It was acknowledged that EDIS was still in its infancy, but the authors also believe paper will always play a role. Perhaps in 20 years there will be some type of electronic paper that combines the best of both mediums.

Topic Influence

The most thought-provoking idea raised in this paper was how nursing safety concerns decreased their use of the system. Since nurses typically spend much more time with patients than doctors do, this was an issue that had not been raised in other literature. Due to room layout, a satisfactory placement for terminals could not be found. One placement decreased eye contact, which was thought to be detrimental to the patient-nurse bond. Another placement additionally put the patient behind the nurse, and this was thought to be a safety hazard, particularly in the high-stress environment of emergency medicine.

Additional safety concerns dealt with privacy and retiring the traditional whiteboard for tracking patients. Prior to the EDIS, a whiteboard allowed anyone in the vicinity to know where patients were located. This would allow anyone wanting access to a patient, such as the perpetrator of the injury that brought the patient to the hospital, to be able to easily obtain this information. Although nurses found it inconvenient to not have the whiteboard available, they conceded that the EDIS protected patient privacy and enhanced safety. This created yet another hybrid system, where nurses would attach printouts of the electronic whiteboard to use on clipboards that they carried with them.

As more research becomes available, it will be easier to assess user attitudes across varying demographics. In this case, the focus was on nurses. However, it was also a study from the United Kingdom, whereas the Callen study took place in Australia. All of these countries are part of the developed world, where the majority of EDIS implementation has occurred. Hopefully, with greater market penetration, systems will become available at a variety of price points, which will allow greater use in the developing world. In any case, successful implementation may hinge on small details that can only be ascertained from comparing cultures and roles within the worldwide healthcare system.

Quality and safety implications of EDIS – Farley et al.


Quality and Safety Implications of Emergency Department Information Systems” by Farley et al. is about the unforeseen effects of EDIS implementation. The ED environment lends itself to a high rate of errors, so it is the area of health information technology (IT) development that would be best served by fixing system problems quickly. As the title states, quality and safety were the two areas of focus. It listed potential pitfalls and benefits, before reviewing several case studies exploring possible problem scenarios and consequences.

The paper goes on to provide specific recommendations including, “dedication of sufficient clinician time; structured risk gathering, analysis, improvement, and monitoring; risk transparency from the vendors and the clients; widespread collaboration for risk analysis; and a call for legal responsibility from the vendors.” It concludes by recommending reading the analysis provided in the 2001 Institute of Medicine analysis titled “Crossing the Quality Chasm.”

Topic Influence

Although not entirely surprising, this article raised the issue that there is no procedure for users to provide feedback on safety concerns, and even went so far as citing a source which claimed some vendors prohibit their customers from sharing such information. While the healthcare industry is heavily regulated, health informatics (HI) is not. There seems to be a “blame the user” culture developing, instead of a closer analysis of how bad design is contributing.

Perhaps because of this, the case studies were fictitious. They focused on four problems: communication failure, poor data display, wrong order/wrong patient errors, and alert fatigue. One interesting suggestion that was raised was about how EDIS could decrease communication rather than the normally assumed increase. They pointed out that systems rely heavily on generic scripted text, which can hinder more than help.

The paper also raised the issue that many clinicians are not going to report errors if any blame could be placed on them, so in the absence of certainty, they may err on the side of caution. This means that not only are safety issues being blocked from being shared by private industry, they are being self-censored by providers. How can you improve a system where lessons learned are being actively ignored?

Another interesting issue was the use of evidence-based medicine in overloading providers with alerts. Clinical decision support is supposed to provide more information than would normally be available at the point of care. However, this information is often shared with warning messages that require the user provide an explanation for why a certain procedure is being overridden. It’s hard to see how this would not interfere with an ED’s normal operational pace.

Implementing electronic health records in the emergency department – Handel and Hackman


Implementing electronic health records in the emergency department” by Handel and Hackman provides lessons learned in EDIS implementation. Unlike other articles, which typically begin with the benefits of implementation, these authors lead with the risks. They break these into nine broad categories, all of which are commonly mentioned in other research. Only then do they outline benefits, and include an estimate of how long it takes to break-even financially.

Following this, they do an analysis of cost, including both initial installation and follow-on maintenance. They suggest a variety of measurements that can be used to assess system success, including charge capture per patient, ambulance diversion times, left-without-being-seen rates, ED length of stay, and relative value units. This last is a formula used by Medicare to reimburse physicians. Clinician engagement in system’s development is raised, as is vendor certification and computerized physician order entry. The authors conclude that workflow is a key determinant in a favorable outcome.

Topic Influence

The authors raised many questions that they felt should be addressed during an implementation. To begin with, troubleshooting and system unavailability need to be addressed in the pre-implementation phase. There is no system that will not have some unforeseen downtime, and planning for this can be the difference between overall success and failure.

In choosing which system to use, it was pointed out that the best system overall is not necessarily the best one for integrating with any legacy systems. This points to the possibility of a “first-mover” advantage, since hospitals may choose a vendor that is already being used in other departments, simply to increase the chances that integration will not become an issue.

The authors did point out that traditional physician narrative structure was a taught process, and while deeply ingrained in medical culture, did not conform to EDIS data entry as it stands. So it seems that there is an opportunity to take what has been learned about human information processing, and integrate it into the information systems we design. However, changing culture is always a significant challenge.

A statistic was cited that claimed reading text from a computer screen was 40% slower than reading it off the page. This is likely to change as research from eReader development is integrated into other types of screens. But it does present a concern in any time-sensitive environment, such as an emergency room. Finally, it was suggested that emergency departments cannot reduce volume the way that other medical providers can, to allow for gradual introduction of an EDIS. This bears further investigation, since phased implementation is likely to lead to better overall outcomes.

Implemention of an emergency department computer system – Batley et al.


After reading several articles written in the developed world, this research was done at American University of Beirut, a facility in the developing world that has U.S. affiliations. Since financial concerns were a challenge here, they designed a basic system in-house. Four groups of users were surveyed: nurses and clerical staff, interns and residents, medical students, and ED physicians. The focus was on how the design features improved implementation.

Since resistance by medical staff is cited as a common source of EDIS failure, the developers in Beirut sought to create a system that would be more welcoming. They did not have formal design training, and used a simply software package to create the system. Although the system was favorably received, they did acknowledge that the previous system was almost entirely paper. They concluded that tiny design considerations made a difference in how readily a system would be accepted, and how much training would be required for it to be utilized.

Topic Influence

Benefits of implementation were calculated in a variety of ways, including time-savings due to design features and general user satisfaction. The authors calculated that color-coding patient locations saved three-person weeks of work annually. Time-savings from integration with the laboratory orders system saved an additional 900 hours annually. Time is a particularly precious commodity in the ED, so these types of measures should be used in conjunction with the more traditional financial return on investment.

This paper noted that training costs and time are usually a significant component of EDIS. The system in Beirut required no training. While it did much less than those usually used in the developed world, perhaps there is something to be learned from developing a simpler system that requires no training and only enhances operations. Particularly in teaching hospitals where there is constant turnover, more complex systems with high training requirements may not be appropriate.

Several design features were highlighted. For example, patient location mirrored hospital layout in the real world. Color was used in a variety of ways. Patients in pain were coded with a flashing red, while reviewed lab results changed from red to green. Additionally, radio buttons and checkboxes were used in lieu of dropdown menus to limit accidental selection of the wrong option. Finally, there was a division between common tests ordered and unusual requests. This streamlined the ordering process. Clinicians felt this system improved their workflow, whereas many users in the developed world believe their more complex systems to be more trouble than they are worth.



Batley, N. J., Osman, H. O., Kazzi, A. a, & Musallam, K. M. (2011). Implementation of an emergency department computer system: design features that users value. The Journal of Emergency Medicine, 41(6), 693–700. doi:10.1016/j.jemermed.2010.05.014

Callen, J., Paoloni, R., Li, J., Stewart, M., Gibson, K., Georgiou, A., … Westbrook, J. (2013). Perceptions of the effect of information and communication technology on the quality of care delivered in emergency departments: a cross-site qualitative study. Annals of Emergency Medicine, 61(2), 131–44. doi:10.1016/j.annemergmed.2012.08.032

Farley, H. L., Baumlin, K. M., Hamedani, A. G., Cheung, D. S., Edwards, M. R., Fuller, D. C., … Pines, J. M. (2013). Quality and safety implications of emergency department information systems. Annals of Emergency Medicine, 62(4), 399–407. doi:10.1016/j.annemergmed.2013.05.019

Handel, D. a, & Hackman, J. L. (2010). Implementing electronic health records in the emergency department. The Journal of Emergency Medicine, 38(2), 257–63. doi:10.1016/j.jemermed.2008.01.020

Vezyridis, P., Timmons, S., & Wharrad, H. (2011). Going paperless at the emergency department: a socio-technical study of an information system for patient tracking. International Journal of Medical Informatics, 80(7), 455–65. doi:10.1016/j.ijmedinf.2011.04.001