(This Chapter comes from the 2003 2nd Edition of the book. All the chapters in the section on decision support systems have been radically updated and expanded in the 3rd edition, with new sections on data analytics, machine learning, and a completely updated discussion of the role and impact of decision support systems)

It also seems that very early on, scientists and doctors alike were captivated by the potential such a technology might have in medicine (e.g. Ledley and Lusted, 1959). With intelligent computers able to store and process vast stores of knowledge, the hope was that they would become perfect ‘doctors in a box’, assisting or surpassing clinicians with tasks like diagnosis.

With such motivations, a small but talented community of computer scientists and healthcare professionals set about shaping a research program for a new discipline called Artificial Intelligence in Medicine (AIM). These researchers had a bold vision of the way AIM would revolutionise medicine, and push forward the frontiers of technology.

AI in medicine at that time was a largely US-based research community. Work originated out of a number of campuses, including MIT-Tufts, Pittsburgh, Stanford and Rutgers (e.g. Szolovits, 1982; Clancey and Shortliffe, 1984; Miller, 1988). The field attracted many of the best computer scientists and by any measure their output in the first decade of the field remains a remarkable achievement.

In reviewing this new field in 1984, Clancey and Shortliffe provided the following definition:

‘Medical artificial intelligence is primarily concerned with the construction of AI programs that perform diagnosis and make therapy recommendations. Unlike medical applications based on other programming methods, such as purely statistical and probabilistic methods, medical AI programs are based on symbolic models of disease entities and their relationship to patient factors and clinical manifestations.’

Much has changed since then, and today the importance of diagnosis as a task requiring computer support in routine clinical situations receives much less emphasis (Durinck et al., 1994). The strict focus on the medical setting has now broadened across the healthcare spectrum, and instead of AIM systems, it is more typical to describe them as computer decision support systems (CDSS). Intelligent systems today are thus found supporting medication prescribing, in clinical laboratories and educational settings, for clinical surveillance, or in data-rich areas like the intensive care setting.

While there certainly have been ongoing challenges in developing such systems, they actually have proven their reliability and accuracy on repeated occasions (Shortliffe, 1987). Much of the difficulty experienced in introducing them has been associated with the poor way in which they have fitted into clinical practice, either solving problems that were not perceived to be an issue, or imposing changes in the way clinicians worked. What is now being realised is that when they fill an appropriately role, intelligent programmes do indeed offer significant benefits. One of the most important tasks now facing developers of AI-based systems is to characterise accurately those aspects of clinical practice that are best suited to the introduction of artificial intelligence systems.

In the remainder of this chapter, the initial focus will thus remain on the different roles CDSS can play in clinical practice, looking particularly to see where clear successes can be identified, as well as looking to the future. Much of the material presumes familiarity with Chapters two and eight. The next chapter will take a more technological focus, and look at the way CDSS are built. A variety of technologies including expert systems and neural networks will be discussed. The final chapters in this section look at several specialised topics where intelligent decision support is an essential component. We will look at the way CDSS can support the interpretation of patient signals that come off clinical monitoring devices, how it can assist in the surveillance for infectious diseases and public health challenges like bioterrorism, and how genome science is supported through bioinformatics.

25.1    AI can support both the creation and the use of clinical knowledge

Proponents of so-called ‘strong’ AI are interested in creating computer systems whose behaviour is at some level indistinguishable from humans (see Box 25.1). Success in strong AI would result in computer minds that might reside in autonomous physical beings like robots, or perhaps live in ‘virtual’ worlds like the information space created by something like the Internet.

An alternative approach to strong AI is to look at human cognition and decide how it can be supported in complex or difficult situations. For example, a fighter pilot may need the help of intelligent systems to assist in flying an aircraft that is too complex for a human to operate on their own. These ‘weak’ AI systems are not intended to have an independent existence, but are a form of ‘cognitive prosthesis’ that supports a human in a variety of tasks.

CDSS are by and large intended to support healthcare workers in the normal course of their duties, assisting with tasks that rely on the manipulation of data and knowledge. An AI system could be running within an electronic patient record system, for example, and alert a clinician when it detects a contraindication to a planned treatment. It could also alert the clinician when it detected patterns in clinical data that suggested significant changes in a patient’s condition.

Along with tasks that require reasoning with clinical knowledge, AI systems also have a very different role to play in the process of scientific research. In particular, AI systems have the capacity to learn, leading to the discovery of new phenomena and the creation of clinical knowledge. For example, a computer system can be used to analyse large amounts of data, looking for complex patterns within it that suggest previously unexpected associations. Equally, with enough of a model of existing knowledge, an AI system can be used to show how a new set of experimental observations conflict with the existing theories. We shall now examine such capabilities in more detail.

25.2    Reasoning with clinical knowledge

Knowledge-based systems are the commonest type of CDSS technology in routine clinical use. Also known as expert systems, they contain clinical knowledge, usually about a very specifically defined task, and are able to reason with data from individual patients to come up with reasoned conclusions. Although there are many variations, the knowledge within an expert system is typically represented in the form of a set of rules.

There are many different types of clinical task to which expert systems can be applied.

Alerts and reminders. In real-time situations, an expert system attached to a patient monitoring device like an ECG or pulse oximeter can warn of changes in a patient’s condition. In less acute circumstances, it might scan laboratory test results, drug or test order, or the EMR and then send reminders or warnings, either via immediate on-screen feedback or through a messaging system like e-mail. Reminder systems are used to notify clinicians of important tasks that need to be done before an event occurs. For example, an outpatient clinic reminder system may generate a list of immunizations that each patient on the daily schedule requires (Randolph et al., 1999).

Diagnostic assistance. When a patient’s case is complex, rare or the person making the diagnosis is simply inexperienced, an expert system can help in the formulation of likely diagnoses based on patient data presented to it, and the systems understanding of illness, stored in its knowledge base. Diagnostic assistance is often needed with complex data, such as the ECG, where most clinicians can make straightforward diagnoses, but may miss rare presentations of common illnesses like myocardial infarction, or may struggle with formulating diagnoses, which typically require specialised expertise.

Therapy critiquing and planning. Critiquing systems can look for inconsistencies, errors and omissions in an existing treatment plan, but do not assist in the generation of the plan.Critiquing systems can applied to physician order entry. For example, on entering an order for a blood transfusion a clinician may receive a message stating that the patient’s haemoglobin level is above the transfusion threshold, and the clinician must justify the order by stating an indication, such as active bleeding (Randolph et al., 1999). Planning systems on the other hand have more knowledge about the structure of treatment protocols and can be used to formulate a treatment based upon a data on patient’s specific condition from the EMR and accepted treatment guidelines.

Prescribing decision support systems. One of the commonest clinical tasks is the prescription of medications, and PDSS can assist by checking for drug-drug interactions, dosage errors, and if connected to an EMR, for other prescribing contraindications such as allergy. PDSS are usually well received because they support a pre-existing routine task, and as well as improving the quality of the clinical decision, usually offer other benefits like automated script generation and sometimes electronic transmission of the script to a pharmacy.

Information retrieval. Finding evidence in support of clinical cases is still difficult on the Web, and intelligent information retrieval systems can assist in formulating appropriately specific and accurate clinical questions, they can act as information filters, by reducing the number of documents found in response to a query to a Web search engine, and they can assist in identifying the most appropriate sources of evidence appropriate to a clinical question. More complex software ‘agents’ can be sent to search for and retrieve information to answer clinical questions, for example on the Internet. The agent may contain knowledge about its user’s preferences and needs, and may also have some clinical knowledge to assist it in assessing the importance and utility of what it finds.

Image recognition and interpretation. Many clinical images can now be automatically interpreted, from plane X-rays through to more complex images like angiograms, CT and MRI scans. This is of value in mass-screenings, for example, when the system can flag potentially abnormal images for detailed human attention.

There are numerous reasons why more CDSS are not in routine use (Coiera, 1994). Some require the existence of an electronic patient record system to supply their data, and most institutions and practices do not yet have all their working data available electronically. Others suffer from poor human interface design and so do not get used even if they are of benefit.

Much of the initial reluctance to use CDSS simply arose because they did not fit naturally into the process of care, and as a result using them required additional effort from already busy individuals. It is also true, but perhaps dangerous, to ascribe some of the reluctance to use early systems upon the technophobia or computer illiteracy of healthcare workers. If a system is perceived by those using it to be beneficial, then it will be used. If not, independent of its true value, it will probably be rejected.

Happily, there are today very many systems that have made it into clinical use (Table 25.1). Many of these are small, but nevertheless make positive contributions to care. Others, like prescribing decision support systems, are in widespread use and for many clinicians form a routine part of their everyday practice.

Diagnostic and educational systems

In the first decade of AIM, most research systems were developed to assist clinicians in the process of diagnosis, typically with the intention that it would be used during a clinical encounter with a patient. Most of these early systems did not develop further than the research laboratory, partly because they did not gain sufficient support from clinicians to permit their routine introduction.

DXplain is an example of one of these clinical decision support systems, developed at the Massachusetts General Hospital (Barnett et al., 1987). It is used to assist in the process of diagnosis, taking a set of clinical findings including signs, symptoms, laboratory data and then produces a ranked list of diagnoses. It provides justification for each of differential diagnosis, and suggests further investigations. The system contains a database of crude probabilities for over 4,500 clinical manifestations that are associated with over 2,000 different diseases.

DXplain is in routine use at a number of hospitals and medical schools, mostly for clinical education purposes, but is also available for clinical consultation. It also has a role as an electronic medical textbook. It is able to provide a description of over 2,000 different diseases, emphasising the signs and symptoms that occur in each disease and provides recent references appropriate for each specific disease.

Decision support systems need not be ‘stand alone’ but can be deeply integrated into an electronic patient record system. Indeed, such integration reduces the barriers to using such a system, by crafting them more closely into clinical working processes, rather than expecting workers to create new processes to use them.

The HELP system is an example of this type of knowledge-based hospital information system, which began operation in 1980 (Kuperman et al., 1990; Kuperman et al., 1991). It not only supports the routine applications of a hospital information system (HIS) including management of admissions and discharges and order entry, but also provides a decision support function. The decision support system has been actively incorporated into the functions of the routine HIS applications. Decision support provides clinicians with alerts and reminders, data interpretation and patient diagnosis facilities, patient management suggestions and clinical protocols. Activation of the decision support is provided within the applications but can also be triggered automatically as clinical data is entered into the patient’s computerised record.

Expert laboratory information systems

One of the most successful areas in which expert systems are applied is in the clinical laboratory. Practitioners may be unaware that while a pathologist checked the printed report they receive from a laboratory, the whole report may now have been generated by a computer system that has automatically interpreted the test results. Examples of such systems include the following.

• The PUFF system for automatic interpretation of pulmonary function tests has been sold in its commercial form to hundreds of sites world-wide (Snow et al., 1988). PUFF went into production at Pacific Presbyterian Medical Centre in San Francisco in 1977, making it one of the very earliest medical expert systems in use. Many thousands of cases later, it is still in routine use.
• A more general example of this type of system is PEIRS (Pathology Expert Interpretative Reporting System) (Edwards et al., 1993). During it period of operation, PEIRS interpreted about 80-100 reports a day with a diagnostic accuracy of about 95%. It accounted for about which 20% of all the reports generated by the hospital’s Chemical Pathology Department. PEIRS reported on thyroid function tests, arterial blood gases, urine and plasma catecholamines, hCG (human chorionic gonadotrophin) and AFP (alpha fetoprotein), glucose tolerance tests, cortisol, gastrin, cholinesterase phenotypes and parathyroid hormone related peptide (PTH-RP).

Laboratory expert systems usually do not intrude into clinical practice. Rather, they are embedded within the process of care, and with the exception of laboratory staff, clinicians working with patients do not need to interact with them. For the ordering clinician, the system prints a report with a diagnostic hypothesis for consideration, but does not remove responsibility for information gathering, examination, assessment and treatment. For the pathologist, the system cuts down the workload of generating reports, without removing the need to check and correct reports.

25.3    Machine learning systems can create new clinical knowledge

Learning is seen to be the quintessential characteristic of an intelligent being. Consequently, one of the driving ambitions of AI has been to develop computers that can learn from experience. The resulting developments in the AI sub-field of machine learning have resulted in a set of techniques that have the potential to alter the way in which knowledge is created.

All scientists are familiar with the statistical approach to data analysis. Given a particular hypothesis, statistical tests are applied to data to see if any relationships can be found between different parameters. Machine learning systems can go much further. They look at raw data and then attempt to hypothesise relationships within the data, and newer learning systems are able to produce quite complex characterisations of those relationships. In other words they attempt to discover humanly understandable concepts.

Learning techniques include neural networks, but encompass a large variety of other methods as well, each with their own particular characteristic benefits and difficulties. For example, some systems are able to learn decision trees from examples taken from data (Quinlan, 1986). These trees look much like the decision tress discussed in Chapter eight, and can be used to help in diagnosis.

Healthcare has formed a rich test-bed for machine learning experiments in the past, allowing scientists to develop complex and powerful learning systems. While there has been much practical use of expert systems in routine clinical settings, at present machine learning systems still seem to be used in a more experimental way. There are, however, many situations in which they can make a significant contribution.

• Machine learning systems can be used to develop the knowledge bases used by expert systems. Given a set of clinical cases that act as examples, a machine learning system can produce a systematic description of those clinical features that uniquely characterise the clinical conditions. This knowledge can be expressed in the form of simple rules, or often as a decision tree. A classic example of this type of system is KARDIO, which was developed to interpret ECGs (Bratko et al., 1989).
• This approach can be extended to explore poorly understood areas of healthcare, and people now talk of the process of ‘data mining’ and of ‘knowledge discovery’ systems. For example, it is possible, using patient data, to automatically construct pathophysiological models that describe the functional relationships between the various measurements. For example, Hau and Coiera (1997) describe a learning system that takes real-time patient data obtained during cardiac bypass surgery, and then creates models of normal and abnormal cardiac physiology. These models might be used to look for changes in a patient’s condition if used at the time they are created. Alternatively, if used in a research setting, these models can serve as initial hypotheses that can drive further experimentation.
• One particularly exciting development has been the use of learning systems to discover new drugs. The learning system is given examples of one or more drugs that weakly exhibit a particular activity, and based upon a description of the chemical structure of those compounds, the learning system suggests which of the chemical attributes are necessary for that pharmacological activity. Based upon the new characterisation of chemical structure produced by the learning system, drug designers can try to design a new compound that has those characteristics. Currently, drug designers synthesis a number of analogues of the drug they wish to improve upon, and experiment with these to determine which exhibits the desired activity. By boot-strapping the process using the machine learning approach, the development of new drugs can be speeded up, and the costs significantly reduced. At present statistical analyses of activity are used to assist with analogue development, and machine learning techniques have been shown to at least equal if not outperform them, as well as having the benefit of generating knowledge in a form that is more easily understood by chemists (King et al., 1992). Since such learning experiments are still in their infancy, significant developments can be expected here in the next few years.
• Machine learning has a potential role to play in the development of clinical guidelines. It is often the case that there are several alternate treatments for a given condition, with slightly different outcomes. It may not be clear however, what features of one particular treatment method are responsible for the better results. If databases are kept of the outcomes of competing treatments, then machine learning systems can be used to identify features that are responsible for different outcomes

25.4   Clinical Decision Support Systems have repeatedly demonstrated their worth when evaluated

Many potential benefits from CDSS have been widely reported in the literature (Johnson & Feldman, 1995; Evans, 1996). The claims made fall into 3 broad categories (Sintchenko et al., 2002):

1.     Improved patient safety e.g. through reduced medication errors and adverse events and improved medication and test ordering;

2.     Improved quality of care e.g. by increasing clinicians’ available time for direct patient care, increased application of clinical pathways and guidelines, facilitating the use of up-to-date clinical evidence, improved clinical documentation and patient satisfaction; and

3.     Improved efficiency in health care delivery e.g. by reducing costs through faster order processing, reductions in test duplication, decreased adverse events, and changed patterns of drug prescribing favouring cheaper but equally effective generic brands.

The evaluation of CDSS are often poorly conceptualised and implemented (Cushman, 1997; Heathfield et al, 1998). In a systematic review of 55 CDSS evaluations, Sintchenko et al. (2003) found that less than a quarter involved a randomised controlled trial (Table 25.2).

Evaluation Methodology %

Table 25.3: Limitations of evaluation components of CDSS studies (Sintchenko et al., 2003).

• A focus on post-system implementation evaluation of users’ perceptions of systems.
• A reliance upon retrospective designs which are limited in their ability to determine the extent to which improvements in outcome and process indicators may be causally linked to the CDSS.
• Rare adoption of a comprehensive approach to evaluation where a multi-method design is used to capture the impact of CDSS on multiple dimensions.
• Concentration on assessment of technical and functionality issues, which are    estimated to explain less than 20% of IT failures. Such evaluations have also failed to determine why useful and useable systems are often unsuccessful.
• Expectations that improvements will be immediate.  In the short term there is likely to be a decrease in productivity.  Implementing information systems takes time and measuring its impact is complex thus a long-term evaluation strategy is required but rarely implemented.
• Almost none use naturalistic design in routine clinical settings with real patients and most studies involved doctors and excluded other clinical or managerial staff.

Evaluation of CDSS is complex, and there are many challenges in appropriately structuring such studies (Randolph et al., 1999). Consequently many studies fall into traps such as overemphasising user satisfaction as a measure of system success. Some of the most frequent limitations of CDSS studies are listed in Table 25.3. While CDSS are often justified on the basis of clinical benefit, evaluation often focuses on technical issues or on clinical processes.  Measurement of clinical outcomes is still sadly rare amongst evaluation studies, and most studies that do attempt to measure clinical impact do so through process variables (Table 25.4).

Nevertheless, the growing pool of evidence on the impact of CDSS in delivering improvements in the quality, safety and efficiency of health is promising, mainly in relation to alerts and reminders, and PDSSs. The following sections demonstrate not only the value of decision support systems in clinical practice, but also the complexity of the evaluation task, the ongoing gaps in our knowledge about their effectiveness, and the richness and variety of form of decision support.

Improvement in Patient Safety

Overall, there is now a body of research which provides good evidence of the effectiveness of CDSS, specifically computerised medication order entry systems, in increasing the safety of patients by reducing errors, adverse events and by increasing the proportion of appropriate and safe prescribing decisions.

Reduction in Medication Errors and Adverse Drug Events

The interest in electronic prescribing systems has been generated by growing evidence of their effectiveness in reducing medication errors, which are one of the most significant causes of iatrogenic injury, death and costs in hospitals (Institute of Medicine, 2000; Thomas et al, 1999). In the US it is estimated that over 770,000 people are injured or die each year in hospitals as a result of adverse drug events (ADEs) (Kaushal & Bates, 2001). The greatest proportion (56%) of preventable ADEs occur at the drug ordering stage, and only 4% at dispensing (Bates et al, 1995).  Errors occurring at the earlier stages are more likely to be intercepted (48%) compared to those occurring at the administration stage (0%).

Physician order entry (POE) systems are used to automate the process of prescription, and assist in the generating and communicating of computerised scripts. POE typically include a prescribing decision support module to check the prescription produced.

Computer-assisted prescriptions have been shown in one study to contain more than three times fewer errors than handwritten prescriptions and to be five times less likely to require pharmacist clarification (Bizovi et al., 2002). The first convincing evidence that such performance translates into a clinically significant reduction in medication errors and adverse drug events comes from two seminal studies undertaken at the Brigham and Women’s Hospital in Boston, Massachusetts (Bates et al, 1998; Bates et al, 1999). The PDSS was developed specifically by the hospital and offered clinical decision support such as allergy alerts and suggested drug doses.

The first study (Bates et al, 1998) demonstrated a 55% reduction in potential adverse drug events (i.e. errors which did not result in injury but had the potential to do so) following system implementation.  The rate of adverse drug events fell from 10.7/1000 patient days to 4.9 per 1000 patient days. The second study (Bates et al, 1999) demonstrated an 86% reduction in potential adverse drug events 4 years post-implementation. Unexpectedly, the number of life-threatening potential adverse drug events increased from 11% pre-implementation to 95% after five months.  The majority of these related to orders regarding potassium chloride because the PDSS system made it easy to order large doses of intravenous potassium without specifying that it be given in divided doses.  Once changes were made to the order screen for this drug, the number of life-threatening intercepted potential adverse events fell to zero. If these systems were implemented nationwide in the USA, it has been calculated that they would prevent approximately 522,000 adverse drug events each year and if only 0.1% of such errors were fatal, over 500 deaths would be avoided (Birkmeyer et al, 2000)

In a broad systematic review of the impact of CDSS on the practice of doctors and patient outcomes, Hunt et al (1996) identified 68 studies, 15 of which involved CDSS to assist dosing of potentially toxic drugs. Of these, 8 focused on dosing of intravenous drugs and 6 found improvements following the use of the CDSS.  Four of the 8 studies examined the impact on patient outcomes, yet only 1 found a significant benefit (Mungall et al., 1994). The authors stated that the consistent pattern of evidence suggests that intravenously medications can be titrated more effectively using a CDSS than without one, if the pharmacokinetics are well understood.

Clinicians are often unaware of the mistakes made when they prescribe. In one study, doctors stated that they were unaware of the potential clinical situation leading to 44% of the clinically significant ADEs that were identified by an alert system (Raschke et al, 1998). However alerting a clinician to an ADE may not result in changes to clinical behaviour, possibly because in many cases no change is necessary. A US ambulatory setting study of 2.3 million people’s prescriptions identified 43,000 alerts for 23,697 people. In 15% of instances when the physician was contacted the alerts resulted in an immediate change in drug management and in a further 9% there was agreement to review management at the next patient visit (Monane et al, 1998).

Enhancing Prescribing Behaviour

PDSS systems have the potential to change prescribing patterns resulting in more cost-effective drug selection. Teich et al (2000) examined four specific drug decision support interventions:

1.   Prompt to use a cheaper generic drug when a more expensive drug was initially ordered;

2.   Presentation of a list of suggested drug doses for each medication ordered;

3.   A highlighted recommended frequency of dose for specific intravenous drugs;

4.   A prompt to suggest the order of a blood thinning drug for patients prescribed bed rest.

For all drug ordering interventions an immediate positive impact was found with greater compliance with the recommended drug orders. Compliance with the generic drug choice changed from around 14% pre implementation to over 80% 2 months post system implementation and to 97% compliance at 1 and 2 year follow-up. The proportion of drug orders that exceeded the recommended maximum dose dropped from 2% in the pre-implementation period to less than 0.3% 2 years post-implementation.

Change in the frequency of dose to reflect the recommended frequency occurred, with 6% compliance pre-implementation moving to 94% compliance at 1, 2 and 3-year follow-up. Orders for anticoagulants for patients prescribed bed rest increased from 24% pre-implementation to 54% at 2-year follow-up.

Improved Quality of Care

Multiple measures have been used to attempt to assess the impact of CDSS on quality of care.  Most have examined changes in work practices with the expectation that improvements in patient outcomes will follow. Examples of process measures include increased compliance with practice guidelines or clinical pathways and increases in the proportion of time available for direct patient care.

Direct measurement of health outcomes is rare and this is largely attributable to the difficulty of undertaking such studies. One of the few such studies undertaken was in a general medical clinic where clinicians were randomised to a reminder system over a two-year period (McDonald et al, 1984). Patients eligible for a flu or pneumococcal vaccine and who attended one of the clinicians who received the reminder system, experienced significantly fewer winter hospitalisations and emergency room visits than patients in the control group.

Improved Compliance with Clinical Pathways and Guidelines

Order entry and results reporting systems with embedded decision support can increase compliance with recommended clinical pathways and guidelines and have been shown to reduce rates of inappropriate diagnostic tests (Harpole et al, 1997).

In a randomised trial of antibiotic guidelines, a proportion of clinicians were shown Vancomycin prescribing guidelines when they attempted to order this antibiotic using an electronic prescribing system (Shojania et al., 1998). The use of Vancomycin dropped by 30% with the guidelines, and when it was prescribed, the medication was given for a significantly shorter duration compared to the control group. The paediatricians in the control group also increased their use of the recommended dose, but by a smaller proportion. This change was attributed to a diffusion of the evidence from the members of the control group who were working in the same clinic.

Alerts have been demonstrated to result in faster treatment by highlighting patients whose medication needed review. For example, Rind et al (1994) assessed the impact of an alert system that notified doctors of patients experiencing rising creatinine levels and receiving nephrotoxic medications.  They found that the use of the alerts was associated with a significantly faster average response to change patients’ medications. While the absolute risk of renal impairment in these patients was low, the alert system reduced this by 55%. However other studies have found alerts to be of little effect in changing clinician behaviour. One study of reminders printed on daily hospital ward round reports were found to have little impact on physicians’ implementation of preventative care (Overhage et al, 1996).

Time Released for Patient Care

One potential benefit of the implementation of order entry systems is that they may allow clinicians to spend more time in direct patient care. Shu et al (2001) undertook a study of the impact of POE on time spent ordering, and available time for other tasks at Massachusetts General Hospital in the USA. Post-implementation interns spent 3% more of their time with patients (from 13-16%), 6% more of their time alone (32% versus 38%) and less time with other physicians (47% versus 41%).

Tierney et al (1993) conducted a time and motion study to determine time spent by clinicians in the order entry process. Use of the POE actually resulted in increased time writing orders compared to paper orders (59 minutes versus 26 minutes) but six minutes less per day was spent on recording routine patient record data, and admitting medication orders on average were filled 63 minutes faster and 34 minutes faster for daily drug orders. These results highlight that single indicators of improved efficiency may be of limited use and that the extent to which such systems release clinicians from administrative tasks will vary depending upon multiple factors such as type of system used, professional group, tasks to be undertaken, and experience with the system.

Improved Efficiency of Health Care Delivery Processes

Studies relating to the impact of CDSS on the efficiency of health care delivery have focused on reductions in costs due to fewer medication errors and adverse drug events, increased efficiency in the execution of patient care, particularly in relation to tests and drug orders; and increased use of generic drug brands.

Several estimates of the money saved as a result of computerised physician order entry systems have been made. At Brigham and Women’s hospital in the US order entry systems are estimated to save about $5-10 million per year largely due to the increased use of less expensive tests and drugs. In the US one adverse drug event is estimated to add an average of $2000 – $6000 to a patient admission and thus in total are estimated to cost over $2 billion each year in US hospitals alone (Raschke et al, 1998).

In a randomised controlled trial, Tierney et al (1993) demonstrated that patients treated by physicians who used a POE containing decision support, which included costs of specific drugs and diagnostic tests, had less expensive hospital stays.  Extrapolation of the cost savings due to reduced costs per admission was in the order of $3 million for the teaching hospital. However, no adjustments were made to account for fixed or marginal costs.

The central economic evaluation question is, “What is the cost of delivery of the IT system and how does this relate to the benefits or value that have resulted?” (Bannister et al, 2001). One of the greatest challenges with economic evaluation is that in addition to cost data, benefits need to be measured and assigned a monetary value, yet most studies ignore benefits for which a dollar value may not be assignable.  True costing of the CDSS investment side is also complex and there is great debate about what should be included or excluded from such calculations. One approach that is advocated is the balanced score card (Kaplan & Norton, 1992) which measures performance indicators for four systems – financial, effectiveness and efficiency of internal processes, the customer perspective and organizational learning and innovation (Van Grembergen & Van Bruggen, 1997; Protti, 2002).

25.5    Summary

The available evidence on the impact of CDSS to deliver improvements in the quality, safety and efficiency of health is promising.  Reductions in medication and adverse drug events alone appear impressive. To date the majority of studies have been undertaken in the United States and importantly, most have involved CDSS systems that have been internally developed and customized for individual health care organizations. The question as to whether the impressive results from systems that are custom built will be more widely reproducible is yet to be answered (Poikonen & Leventhal, 1999).

Discussion Points:

1.     Which clinical tasks are worth automating?

2.     Who is responsible if a CDSS makes a recommendation to a clinician that results in patient harm?

3.     Should clinicians be able to use a CDSS without any specific training, or should we consider a CDSS as a specialised piece of clinical equipment that requires the user to first pass a certification course?

Chapter summary

1.   Artificial intelligence (AI) systems are intended to support healthcare workers with tasks that rely on the manipulation of data and knowledge.

2.   Expert systems are the commonest type of CDSS in routine clinical use. They contain medical knowledge about a very specifically defined task. Their uses include:

• alerts and reminders,
• diagnostic assistance,
• therapy critiquing and planning,
• prescribing decision support
• information retrieval,
• image recognition and interpretation.

3.   Reasons for the failure of many expert systems to be used clinically include dependence on an electronic medical record system to supply their data, poor human interface design, failure to fit naturally into the routine process of care,  and reluctance or computer illiteracy of some healthcare workers.

4.   Many expert systems are now in routine use in acute care settings, clinical laboratories, educational institutions, and incorporated into electronic medical record systems.

5.   Some CDSS systems have the capacity to learn, leading to the discovery of new phenomena and the creation of medical knowledge. These machine learning systems can be used to:

• develop the knowledge bases used by expert systems,
• assist in the design of new drugs,
• advance research in the development of pathophysiological models from experimental data.

6.     Benefits from CDSS include improved patient safety, improved quality of care, and improved efficiency in health care delivery.

  © Enrico Coiera 1997-2014