Needs Assessment

Assessing Information Needs for Emergency Physicians

User population

The hospital emergency department (ED) is a stressful, time-constrained work environment where healthcare providers are often working with incomplete information. They must deal with the widest variety of patients, including everything from a chest pain scare to a catastrophic accident.  Only 22% of ED admissions are due to injuries (National Hospital Ambulatory Medical Care Survey, 2010). More and more people who simply are trying to manage chronic conditions are finding their way into the ED.  And as healthcare has become more expensive, patients often wait to treat their healthcare problems until they do become emergencies, seeking care only when they have no other option (Messina et al., 2013).

This is the environment emergency physicians operate in. Of all the medical specialties, they must seek information for the widest variety of ailments. Often they require this information immediately, and the consequences of acting on erroneous information can be catastrophic for their patients. Additionally, they must routinely function in a noisy and chaotic environment, juggling multiple cases simultaneously. This is bound to decrease information retention and increase the likelihood of errors. For all these reasons, improving information services for this population would be a particularly challenging and rewarding endeavor.

Literature Review

According to Kannampallil, Bennett, and Reddy, one of the most important issues affecting the information behavior of emergency physicians is their inability to access needed information in a seamless manner. In some hospitals, they are constantly switching back and forth between paper patient records and electronic health records (EHR), most often utilizing paper for the narrative information and electronic for structured information such as test results (Kannampallil, 2013). Even in clinics that have robust EHRs, there is still the division between narrative and atomized information structures. ED physicians must also seek information about a wide variety of health issues and they need to utilize multiple general information sources (Bennett, 2005). Their information needs typically arise when collaborating with the ED team. Attempting to integrate multiple sources of information with multiple individuals leads to greater fragmentation of that information (Reddy, 2008).

Hughes, Davies, Bass, and Lappa all point to the constant time debt ED doctors operate under as a second issue affecting their behavior. Hughes examined the behavior of newly minted physicians just out of medical school across all specialties, and found that time poverty was the main driver leading them to utilize mainstream web sources (such as Wikipedia) for their information needs. This group finds the ease of use outweighed concerns regarding validity (Hughes, 2009.) Other research concluded that lack of time was the main barrier to use for electronic clinical support systems (Davies, 2011.) Information seeking still relies heavily on verbal communication between healthcare practitioners, who do not feel they have the time to engage with an information system (Lappa, 2005).

A third issue is that physicians simply do not trust their information systems. Physicians perceived that their judgments regarding high acuity patients in the ED more often relied on personal clinical decision making rather than information seeking from evidence-based decision support systems (DSS). Physicians may be resistant to DSS because they believe that the information in these systems is of limited accuracy (Calder, 2013). In this case, their personal judgment about who was likely to suffer an adverse event was not accurate. Also, physicians in acute care were observed not utilizing personal medical records for information. The perception was that these notes were primarily formatted for billing purposes and did not provide the information needed to treat the patient (Bass, 2012).

A final issue raised in the literature was that like intelligence analysts, physicians sometimes do not know what they do not know. And because of the aforementioned time scarcity, they do not have the time to make a comprehensive search on many topics when there is too much information available.

Questions Explored

Many different information needs and sources are being observed in the ED, but verbal communication is the most commonly used source and patient information is not surprisingly considered the most important need (Ayatollahi, 2013). Bass found that the majority of questions directed to the residents on overnight shifts in the ED were regarding the plan of care. This amounted to 39.8% of inquiries. 13.4% concerned medical knowledge,
 12.1% addressed simply task work and only 11.1% concerned the current condition of patients (Bass, 2012). Finally, almost one third (26.1%) of information seeking in the ED primarily dealt with organizational and coordination issues rather than medical care (Reddy, 2008).

We know what emergency physicians need to know, but we don’t necessarily know the best way to augment their information seeking to overcome the challenges highlighted in the literature. Some of these concerns can be addressed at the information systems level, while others will have deeper underlying problems that can only be improved at the organizational level. For example, time scarcity may be a result of poor economic conditions and the overuse of the ED in America as an impromptu triage station for the greater healthcare system.

One of the key questions that underlie all information system development is when the machine is the best answer and when man is still a better choice. A majority of emergency-care physicians preferred to have a clinical librarian (CL) search for evidence-based medical information, rather than completing the research themselves. CLs believed information technology resources and interfaces could change too quickly to be used effectively by clinicians (Lappa, 2005). Certainly, believing this is in their self-interest, but they may not be wrong. A more efficient use of physician time might be for them to do basic searches, and to give the more intensive searches to a medical librarian who has the training to more efficiently gather information (Davies, 2011).

However, the nature of the work may mean the person engaging in the task (patient care) has long-term value added by doing the research themselves. Residents working in critical care used a patient-based information seeking strategy drawing on a variety of sources, whereas the physician assistants and nurse practitioners they worked with used a source-base strategy focusing on lab and test results. The former is less cognitively taxing and results in less switch cost (Kannampallil, 2014). Could perhaps the switch cost cancel out whatever gains were involved with having someone else pursue the research?

Primary Question For Further Study

Of all the questions raised in the literature, the first place to begin could be by asking how prevalent Calder’s findings were on a larger scale (Calder, 2013). If ED physicians believe that their experience-based judgment is better than that embedded within the DSS, they are likely to avoid using the system. This makes it difficult to encourage use of integrated information systems, and will continue to silo development. While separate systems can improve processes, they are typically mimicking the paper schema they replaced. True leaps in productivity only come about through integration, so encouraging the use of (arguably) the most important part of the overall system should help address several of the noted problems, such as information fragmentation, time scarcity, and lack of trust.

Research Design

Observing what has been done in other fields saves some reinvention of the wheel. Calder’s findings regarding the judgment of acute care physicians have been seen mirrored in judges in the State of New Jersey (Milgram, 2013). The New Jersey Department of Justice (NJDOJ) found that judges believed they were releasing people who were less likely to offend again, but in fact their decisions were not reflected in the data on recidivism. Judges trusted their own judgment even when the data did not support this evaluation. To address the problem, NJDOJ collected 1.5 million cases nationwide at every level of the judicial system from local through federal. They then analyzed that data set and found 900 risk factors. From these, they chose nine that were the most highly predictive of risk. They then designed a risk analysis template for use by judges using those nine characteristics.

Copying this process in EDs would allow for the creation of a field experiment to compare doctors who use data-driven risk assessment tools combined with their professional instinct, versus doctors who continue to rely on instinct alone. Compiling a similar data set across a range of EDs (large, medium, small, urban, rural, etc.) would provide the ability to discover what risk factors were embedded in those cases. From there, those that are most highly predictive of risk could be used to create a risk assessment tool that could make the same changes in medicine that have been seen in criminal justice, professional sports, the financial markets, and other industries that are using big data analytics (Baker, 2014).

Data Analysis

While some believe that the era of big data is overhyped, we are now coming to the point in the evolution of data management where our ability to manipulate “all” the data is giving us insights not possible with traditional sampling techniques (Mayer-Schonberger, 2014). One of the keys to effectively analyzing the data is examining each correlation for unintended blowback. Simply because a risk factor has significant correlation does not necessarily mean that, that particular factor should be used in a risk assessment tool. For example, in the case of American criminal justice some of the leading risk factors are racial (Pishko, 2014). Yet we did not want to penalize individuals for something they have no control over. When translated into healthcare, the measures are different because being “penalized” means you are more likely to be brought back into the system for care. Each system and each metric must be examined individually to ensure that “success” does not lead to further problems.

Improvement in Understanding

Analyzing the difference in outcomes from an emergency physician’s professional judgment, versus that judgment augmented by data analytics, would allow us to ascertain how effective the information seeking behavior of this population is. Depending on the outcome, modifications to the existing habits and training of physicians could be instituted. On the one hand, if the physician’s judgment alone leads to similar success rates as judgment enhanced by known risk factor analysis, then it is likely that his information needs are being effectively met. However, if data analytics show a serious disconnect between judgment and adverse events, the root of this gap should be studied.

Professional judgment is an amalgamation of a lifetime of information seeking. While information for information’s sake is a noble ideal, in the time and resource constrained world in which we live, information needs should be met in such a way as to have the most effective alignment with desired outcomes. Each year brings more data, information, and hopefully knowledge. What people are now referring to as information overload is likely to become a deluge. Perhaps it already is. Understanding information behavior will only increase in importance.

Improvement in Systems

If doctors can be shown that the part of the emergency room information system most likely to prevent adverse outcomes can provide them accurate guidance, their other apprehensions may be eased. Trust in an information system is a complex process and parsing the details is an ongoing challenge. As in a human relationship, trust may be difficult to build and too easy to lose. Of all the things that physicians fear, adverse events are at the top of the list, particularly in our litigious society. Most doctors certainly don’t want to harm their patients to begin with, but the threat of litigation adds an additional layer of stress. If an information system can be shown to use data analytics that significantly mitigates this risk, trust in the overall system is likely to increase.

This analysis has broad applicability across a variety of professions. In any field with the enormous lifelong information seeking required by occupations such as law, engineering, or intelligence analysis, comparing professional judgment with data supplemented judgment can help to realign professional training. The era of big data will most likely change many facets of life in the future, and can help us navigate through the ever-increasing information sea in which we tread.

 

Bibliography

Ayatollahi, H., Bath, P. a., & Goodacre, S. (2013). Information needs of clinicians and non-clinicians in the Emergency Department: A qualitative study. Health Information and Libraries Journal, 30, 191–200. doi:10.1111/hir.12019

Baker, W., Kiewell, D. & Winkler, G. (2014, June). Using big data to make better pricing decisions. Retrieved from http://www.mckinsey.com/insights/marketing_sales/using_big_data_to_make_better_pricing_decisions

Bass, E. J., Devoge, J. M., Waggoner-Fountain, L. A., & Borowitz, S. M. (2012). Resident physicians as human information systems: sources yet seekers. Journal of the American Medical Informatics Association: JAMIA, 1–7. doi:10.1136/amiajnl-2012-001112

Bennett, N. L., Casebeer, L. L., Kristofco, R., & Collins, B. C. (2005). Family physicians’ information seeking behaviors: a survey comparison with other specialties. BMC Medical Informatics and Decision Making, 5, 9. doi:10.1186/1472-6947-5-9

Braun, L., Weisman, F., Herik, J., Hasman, A. & Korsten, E. (2005). Towards Automatic Formulation of a Physician’s Information Needs. Journal of Digital Information Management, 3-1.

Calder, L. A., Arnason, T., Vaillancourt, C., Perry, J. J., Stiell, I. G., & Forster, A. J. (2013). How do emergency physicians make discharge decisions? Emergency Medicine Journal: EMJ, 9–14. doi:10.1136/emermed-2013-202421

Case, D. (2012). Looking for information: A survey of research on information seeking, needs, and behavior. Bingley: Emerald Group Publishing Limited.

Davies, K. (2011). Information Needs and Barriers to Accessing Electronic Information: Hospital-Based Physicians Compared to Primary Care Physicians. Journal of Hospital Librarianship, 11(March 2015), 249–260. doi:10.1080/15323269.2011.587103

Hughes, B., Joshi, I., Lemonde, H., & Wareham, J. (2009). Junior physician’s use of Web 2.0 for information seeking and medical education: A qualitative study. International Journal of Medical Informatics, 78, 645–655. doi:10.1016/j.ijmedinf.2009.04.008

Kannampallil, T. G., Franklin, A., Mishra, R., Almoosa, K. F., Cohen, T., & Patel, V. L. (2013). Understanding the nature of information seeking behavior in critical care: Implications for the design of health information technology. Artificial Intelligence in Medicine, 57(1), 21–29. doi:10.1016/j.artmed.2012.10.002

Kannampallil, T. G., Jones, L. K., Patel, V. L., Buchman, T. G., & Franklin, A. (2014). Comparing the information seeking strategies of residents, nurse practitioners, and physician assistants in critical care settings. Journal of the American Medical Informatics Association: JAMIA, 1–8. doi:10.1136/amiajnl-2013-002615

Lappa, E. (2005). Undertaking an information-needs analysis of the emergency-care physician to inform the role of the clinical librarian: a Greek perspective. Health Information and Libraries Journal, 22, 124–132. doi:10.1111/j.1471-1842.2005.00563.x

Mayer-Schonberger, V. & Cukier, K. (2013) Big data: A revolution that will transform how we live, work, and think. New York: Houghton Mifflin Harcourt

Milgram, A. (2013, October). Why smart statistics are the key to fighting crime [Video file]. Retrieved from https://www.ted.com/talks/anne_milgram_why_smart_statistics_are_the_key_to_fighting_crime

National Hospital Ambulatory Medical Care Survey: 2010 Emergency Department Summary Tables. (2010). Retrieved February 15, 2015, from http://www.cdc.gov/nchs/data/ahcd/nhamcs_emergency/2010_ed_web_tables.pdf

Pishko, J. (2014, August). Punished for being poor: The problem with using big data in the justice system. Retrieved from http://www.psmag.com/politics-and-law/punished-poor-problem-using-big-data-justice-system-88651

Reddy, M. C., & Spence, P. R. (2008). Collaborative information seeking: A field study of a multidisciplinary patient care team. Information Processing and Management, 44, 242–255. doi:10.1016/j.ipm.2006.12.003