Implementation Assessment

Implementation Assessment


One definition of emergency department information systems (EDIS) is, “electronic health record systems designed specifically to manage data and workflow in support of {emergency department} ED patient care and operations” (Landman, 2010).  This basic description doesn’t impart much. To examine successful implementation, we must ask what makes the ED different. To begin with, EDs tend to have a high volume of cases with many patients being processed quickly through the system. Doctors are usually not familiar with their charge’s medical histories, and their work is constantly being interrupted (Farley, 2013). There are repeated handoffs, as patients are rapidly moved from one assessment area to another, whether it’s to be imaged, have blood drawn, or being given temporary resuscitative care to stabilize them on the way to the intensive care unit (Callen, 2013). On top of all this, EDs are often overcrowded (Batley, 2011).

As with all information systems (IS), implementation is problematic. Each physical system that an IS is meant to support contains numerous moving parts, where even the best pre-implementation research can miss a crucial detail. Nevertheless, the ED is a particularly difficult venue to provide value to due to the abovementioned issues. Operations in an ED more closely resemble the bridge of an aircraft carrier than the transaction system of a banking center.

When reviewing the literature on EDIS implementation, the framework of thought fell into several distinct categories. There were the possible benefits of implementation, as well as the drawbacks to even pursuing a new system, which generally applied across the board. Then there were simply factors that might or might not help or hinder. These could differ from system to system. There was some general consensus about what any EDIS should do, and lots of ideas about options that would make up the ideal combination of features. The success or failure of implementation could fall at any stage in the process; the system planning, the actual implementation itself, or the post implementation and subsequent system evaluation. Finally, there were a few casually accepted impressions that might bear further investigation.


A useful activity in pre-implementation is visiting other locations where a similar product has been implemented. In lieu of on-site assessments, extensive input should still be solicited from as many sources as possible. This is not simply useful for deciding which system to invest in. Once the system has been decided on, the implementation team still needs to go back for a more detailed look at lessons learned from other implementation projects. These should fall all along the continuum between those that are deemed an unequivocal success and those which were ultimately withdrawn from service. This provides a baseline for knowing exactly what to ask of the vendor, what needs to be included in the contract, and what to avoid. Or in the case of in-house development, allows clinicians and IT staff the opportunity to resolve the details (Conn, 2009).

Lowering obstacles to acceptance

One key to implementation success is lowering the obstacles to acceptance. Cost is always a concern and cost benefit analysis is often challenging. Many of the factors involved in adoption are difficult to quantify. Models can be created that show immediate returns, but more comprehensive models will paint a complicated picture. Most people may feel that EDIS should eventually be paperless, seamless, and integrated. But those same individuals don’t necessarily want the pain of a complete transition to happen on their watch. It might be problematic to sell the system based on anticipated efficiencies, when it is equally likely that closer analysis will find examples of lost productivity. Biomedical researchers may love the search functionality. EMS pilots are not going to want to wait for a terminal to be available so they can log the paper time sheets they are used to. Some equitable division of the increase in inconvenience should be reached.

End user involvement

End users should be involved with every stage of implementation. This includes the first steps of research and the design process itself, all the way through to the post-implementation assessment. As with any project, early buy-in by all the concerned parties leads to greater satisfaction with the end product (Salman, 2012).

No one group should be singled out above any others. The ED is a team environment, consisting of everyone from the first responder, to the triage nurse, to the trauma surgeon. Along the continuum of care a patient may encounter police officers, radiologists, neurologists, and orderlies. Perhaps no other medical domain is more interdisciplinary.

What should all EDIS share?

Each EDIS is different, and should be tailored to the prevailing conditions. However, there are a few factors that all should share, any one of which can frustrate implementation. All must address security and privacy concerns. The recent abundance of security breaches in corporate America illustrates that this issue is not being addressed as fully as it needs to be.

Additionally, they must conform to interoperability guidelines, an area that is evolving more slowly than is necessary. The system should also have strong authentication and authorization procedures, which not only identify users, but also provide appropriate access and functionality (Rothenhaus, 2009).

What should all EDIS not share?

There are some design elements that are guaranteed to make ED operations worse. For example, screen size needs are heavily dependent on whether clinicians are entering information, or simply viewing it. Similarly, information that needs to be accessed rapidly (at-a-glance) will require large screens, whereas more detailed issues like test results may require a smaller screen to facilitate privacy or diagnostic analysis (Pennathur, 2011).

Screens that require scrolling back and forth to assess related information will hinder patient care. In such a distraction prone environment, physicians are operating with minimal amounts of short-term memory. They must be able to see as much information as possible directly in front of them (Farley, 2013).

Narrative Flow

When a doctor is collecting information from a patient, that information collection is usually happening as part of a narrative. There may or may not be a general order to the questions, depending on the circumstances, but the data is being collected in a stream. Then it is up to the physician to parse that data into evidence, which supports a working hypothesis about diagnosis and treatment. In a paper record, this narrative structure was preserved. Although it may have been hard to use the data for further research, or search for particular facts, the information was recorded within the context it was received (Handel, 2014). For example, a patient might describe their symptoms in the order they noticed them developing, or by the severity of the pain the symptoms were causing.

When the doctor needs to translate this information into bytes, the translation may not be effective. The very atomization that makes retrieving results instantaneous requires that data be entered according to how the system was designed. The physician may be able to find a test result, but not know that the patient was complaining about a certain type of pain after that test result was obtained. When a system model calls for a mixture of free form text and structured input, efficiency is lost. But without the blend, important information will often be lost. Striking the right balance is one of the more important machine challenges to successful implementation.

User Friendly

It goes without saying that the system should be user friendly, but far too often this seems not to be the case. The development process is frequently run by IT analysts rather than healthcare providers, despite the fact that it is the providers who know the most about the ED work flow (Pennathur, 2011). IT people spend so much time in front of computers that they may not realize the steep learning curve involved for people who don’t.

There must be obstinacy involved on both sides here, because one senior nurse who refuses to use the system can undue months and years of development work. The literature often talks about a “physician champion,” but if the entire nursing staff follows their senior’s lead, physician resistance might never come into play (Batley, 2011).

ED Culture

One element to examine is the underlying factors embedded in ED culture. Most of these are beyond the control of the implementation team. However, they are still useful to look at when assessing if implementation should even be attempted. In some cases there will be so many handicaps, that moving forward should be suspended.

The most helpful cultural factor is probably when the end users have been working at the location for a long period of time. Low staff turnover lends itself to stability and cohesiveness within the ED team. Additionally, it is better to be working with a team of full-time employees, rather than the system being supplemented with contract workers. The more cohesive the group, the more likely it is that the workflow will be efficient and effective (Conn, 2009).

Full implementation

A sometimes-overlooked key to successful implementation may be as simple as implementing fully. Oftentimes the EDIS will be designed to fulfill a variety of tasks, yet in the hurry of implementation, end users will either not take the time to learn all the features, or will overlook features that are not as vital as the central components. Before the system is fully implemented, it can be difficult to know if there is an easy fix to a given post-implementation headache (Pallin, 2010).

Physical Factors

Of course, physical factors are yet another portion of implementation that must be attended to. Bandwidth must be sufficient to handle rapid acquisition of information, including images. The weight of laptops or the battery capacity of mobile devices will effect how these devices are utilized. The cost savings from purchasing heavier laptops may be outweighed by their lack of use when nurses on their feet all day leave the laptops at the nurse’s station. And if a mobile tablet can’t last through a shift (and go into overtime) without being charged, it essentially becomes useless. Or if that same tablet shatters every time it slips out of someone’s grasp, the ED will quickly run through its devices (Karahoca, 2010).

At all steps in the implementation, the system should mirror the physical layout as closely as possible. This facilitates better understanding for the implementation team, and greater ease of use for the clinicians (Batley, 2011). For example, the interface providing the basic overview of patient statistics could reflect the bed order in which these patients are located (Batley, 2011).

Human-Computer Interaction

At least a basic understanding of human-computer interaction is necessary for effective implementation. Shared computer systems operate differently from those designed for use by individuals. An EDIS should encourage members of the ED team to have a greater understanding of their teammate’s work. While a personal laptop is typically only meant to be accessed by one person at a time, a collective information system allows various different specialists to see what their colleagues are doing. This allows them to judge how they can best contribute to the overall goals of the ED (Callen, 2013).

Rapid antiquation

No one seems to have come up with a solution to rapidly antiquated technology. Computers may continue to drop in price, but EDIS is never cheap. Furthermore, equipment can age at different rates, with the processors needing to be replaced well before the monitors.

It is no longer the 1990’s and integration has continued to improve since those “dark ages.” But forging together so many constituents is still demanding. The new model may already have a release date before the contract for the previous model is signed. Therefore, it is important to have someone on the development team with enough experience to judge the longevity of each component of the proposed system.


Post-installation is one of the hardest parts of implementation. Now end users must learn to live with the system, and although changes can be made, the bulk of the structure is set. Vendor support may or may not be sufficient, depending on how financial incentives were written into the contract.

One of the details that could have been downplayed by those vendors was how to adapt to downtime. After all, no vendor wants to admit that their system will go down. Whether it’s a crash, a bug, or planned maintenance, given the rapid pace of operations in the ED, downtime can have particularly damaging consequences (Vezyridis, 2011).

Focus of implementation

One-way to look at implementation is that there are two layers to it; the system design itself, and the implementation of that system. While the focus can be on one or the other, the two are so intertwined that it becomes difficult to separate out where one layer ends and the other begins. A picture-perfect implementation process cannot save a poorly designed system, and dysfunctional humans can still torpedo an elegant system. Given a choice, perhaps the emphasis should be placed on system design, since the machines are generally there to stay, while people tend to rotate out.

Most people believe that focusing efforts on what you have control over provides better results than trying to impact those things over which you have little influence. The human part of implementation is essentially the project management component. Project management is a wonderful tool, but its practitioners may kid themselves regarding the degree of control they have. People can be difficult, behave badly, and be irrational. So it stands to reason that no matter how skilled a project manager is, all their efforts may be wasted.

On the other hand, there is much more control to be had when designing a machine. If there is time to do the job well, and the resources to create that design, each machine will do the job it was created to do. So while it is important to balance both the needs of the end users, and the time (and talent and materials) it takes to design a good system, more control is likely to result from giving priority to the system itself.

Identify and discuss gaps in the literature

Each specialty in medicine has its own cultural underpinnings, but perhaps no more so than the ED. Trauma surgery is among the most stressful of any profession, in or out of the realm of medicine. Because the culture is distinct, research concerning clinician’s interaction with the technology in mainstream HIS may not be applicable to EDIS. This is not to say that there will not be numerous similarities. But the assumption that findings from one area seamlessly translate is suspect.

For example, physicians will often ignore recommendations from the decision support system (DSS) not to take an action. This happens even when that action has been documented to provide no useful end result. However, they are more likely to accept guidance from the machine when an alternative is provided in addition to the original recommendation (Horasani, 2003). This suggests a bias for action across the profession.

But trauma surgeons have to move faster than the average specialty and make decisions that can have instantaneous mortal consequences. This is an entirely different mindset from a physician who has time to weigh the possibilities and probabilities embedded in both patient care and a litigious society. Therefore, designing an EDIS and enhancing the likelihood of successful implementation should draw on literature specific to the specialized customs of this subspecialty.

Recommended Areas for Further Research

The most significant area for further research should focus on testing hybrid systems. Usually researchers downplay workarounds as a temporary and unfortunate byproduct of an imperfect system. Yet workarounds continue to be tenacious. Some research acknowledges that there may always be a place for paper (Vezyridis, 2011). Others suggest that verbal confirmation be required for the most serious test results (Farley, 2013). It may not be popular in IT circles to suggest that the EDIS should be augmented with non-computerized ingredients, and hybrid systems are subject to data variation due to duplication (Callen, 2013). But this could be the reality. And if it is, research should be done on how to optimize those processes. Some adaptations reflect a problem with the system; while others may reflect work processes that are actually better left as they are. The computer age has just begun and the dream of full automation may be neither possible, nor desirable.

Another important area of research should be the pros and cons of “hold harmless” and “learned intermediary” clauses in vendor contracts. Both of these absolve system designers from liability. Healthcare providers should not be held entirely responsible for errors involving EDIS. At the same time, technology law is a new field, and it will be many years before there is a body of rulings robust enough to find the right liability balance. Physician resistance will not be reduced as long as doctors are scapegoated for bad design.

Finally, the assumption that EDs cannot reduce their workload during implementation should be questioned (Handel, 2010). Every region of this country has experienced a mass casualty, and inevitably adjustments are made. Patients can be rerouted, temporary triage facilities can be implemented, and extra supplies can be prestaged. Implementation is most successful when clinicians are given proper training, and vendors given time to adjust the system during the go-live phase. Installing a system in a haphazard fashion almost guarantees failure. Therefore, installing a system must include time for personnel to learn and adjust.


How many of these are problems could essentially be solved by providers getting enough sleep? Or by not overworking doctors or not having a strained ED system in the first place? The entering argument in many ED assessments is that most departments will continue to be operating close to or over capacity. Are information systems becoming the smokescreen that covers the natural result of cost cutting? Is it easier to budget for fun, cutting-edge technology than it is to address the cultural issues that encourage poor working conditions? If that is the case, EDIS is turning into an extremely expensive band-aid. Which is not to say that all of the likely benefits aren’t there, or that the potential downside can’t be mitigated. But the argument could be made that money and manpower is being spent on the wrong priorities.

One point that often comes up in the literature is that a good EDIS can’t fix a poorly run ED. This usually refers to work flow issues, or problems specific to a particular hospital. However, if the medical culture has foundational dysfunctions, this needs to be taken into account when accessing the cost benefit of any system analysis. If medical student fatigue accounts for a certain percentage of errors, it probably makes more sense to focus our limited resources on negotiating pay structures that don’t exploit that segment of the work force, to the detriment of everyone they treat.




Batley, N. J., Osman, H. O., Kazzi, A. a, & Musallam, K. M. (2011). Implementation of an emergency department computer system: design features that users value. The Journal of Emergency Medicine, 41(6), 693–700. doi:10.1016/j.jemermed.2010.05.014

Baumlin, K. M., Shapiro, J. S., Weiner, C., Gottlieb, B., Chawla, N., & Richardson, L. D. (2010). Clinical information system and process redesign improves emergency department efficiency. Joint Commission Journal on Quality and Patient Safety / Joint Commission Resources, 36(4), 179–85. Retrieved from

Callen, J., Paoloni, R., Li, J., Stewart, M., Gibson, K., Georgiou, A., … Westbrook, J. (2013). Perceptions of the effect of information and communication technology on the quality of care delivered in emergency departments: a cross-site qualitative study. Annals of Emergency Medicine, 61(2), 131–44. doi:10.1016/j.annemergmed.2012.08.032

Conn, S., Shaw, J., Matthews, J., Wier, S., & Coughlin-Shuler, B. (2009). ED Information Systems: a guideline for successful implementation. Journal of Emergency Nursing: JEN: Official Publication of the Emergency Department Nurses Association, 35(3), 237–8. doi:10.1016/j.jen.2009.01.002

Farley, H. L., Baumlin, K. M., Hamedani, A. G., Cheung, D. S., Edwards, M. R., Fuller, D. C., … Pines, J. M. (2013). Quality and safety implications of emergency department information systems. Annals of Emergency Medicine, 62(4), 399–407. doi:10.1016/j.annemergmed.2013.05.019

Handel, D. a, & Hackman, J. L. (2010). Implementing electronic health records in the emergency department. The Journal of Emergency Medicine, 38(2), 257–63. doi:10.1016/j.jemermed.2008.01.020

Herrick, D. B., Nakhasi, a, Nelson, B., Rice, S., Abbott, P. a, Saber Tehrani, a S., … Newman-Toker, D. E. (2013). Usability characteristics of self-administered computer-assisted interviewing in the emergency department: factors affecting ease of use, efficiency, and entry error. Applied Clinical Informatics, 4(2), 276–92. doi:10.4338/ACI-2012-09-RA-0034

Horasani, R. A. K., Anasijevic, M. I. T., Iddleton, B. L. M., & Ms, C. (2003). Ten Commandments for Effective Clinical Decision Support : Making the Practice of Evidence-based Medicine a Reality, 10(6), 523–531. doi:10.1197/jamia.M1370.Although

Inokuchi, R., Sato, H., Nakajima, S., Shinohara, K., Nakamura, K., Gunshin, M., … Yahagi, N. (2013). Development of information systems and clinical decision support systems for emergency departments: a long road ahead for Japan. Emergency Medicine Journal: EMJ, 30(11), 914–7. doi:10.1136/emermed-2012-201869

Karahoca, A., Bayraktar, E., Tatoglu, E., & Karahoca, D. (2010). Information system design for a hospital emergency department: a usability analysis of software prototypes. Journal of Biomedical Informatics, 43(2), 224–32. doi:10.1016/j.jbi.2009.09.002

Landman, A. B., Bernstein, S. L., Hsiao, A. L., & Desai, R. a. (2010). Emergency department information system adoption in the United States. Academic Emergency Medicine: Official Journal of the Society for Academic Emergency Medicine, 17(5), 536–44. doi:10.1111/j.1553-2712.2010.00722.x

Pallin, D. J., Sullivan, A. F., Auerbach, B. S., & Camargo, C. a. (2010). Adoption of information technology in Massachusetts emergency departments. The Journal of Emergency Medicine, 39(2), 240–4. doi:10.1016/j.jemermed.2008.09.030

Pennathur, P. R., Cao, D., Bisantz, A. M., Lin, L., Fairbanks, R. J., Wears, R. L., … Sui, Z. (2011). Emergency department patient-tracking system evaluation. International Journal of Industrial Ergonomics, 41(4), 360–369. doi:10.1016/j.ergon.2011.02.003

Rothenhaus, T., Kamens, D., Keaton, B. F., & Nathanson, L. (2009). Emergency Department Information Systems. In American College of Emergency Physicians Resolution 22(07) Task Force White Paper (Vol. 22). American College of Emergency Physicians.

Salman, Y. B., Cheng, H.-I., & Patterson, P. E. (2012). Icon and user interface design for emergency medical information systems: a case study. International Journal of Medical Informatics, 81(1), 29–35. doi:10.1016/j.ijmedinf.2011.08.005

Vezyridis, P., Timmons, S., & Wharrad, H. (2011). Going paperless at the emergency department: a socio-technical study of an information system for patient tracking. International Journal of Medical Informatics, 80(7), 455–65. doi:10.1016/j.ijmedinf.2011.04.001