Are standards necessary?
November 1, 2013 § 10 Comments
A common strategy for structuring complex human systems is to demand that everything be standards-based. The standards movement has taken hold in education and healthcare, and technical standards are seen as a prerequisite for information technology.
In healthcare, standards are visible in three critical areas, typical of many sectors: 1/ Evidence-based practice, where synthesis of the latest research generates best-practice recommendations; 2/ Safety, where performance indicators flag when processes are sub-optimal; and 3/ Technical standards, especially in information systems, which are designed to ensure different technical systems can interoperate with each other, or comply with minimum standards required for safe operation. There is a belief that ‘standardisation’ will be a forcing function, with compliance ensuring the “system” moves to the desired goal – whether that be safe care, appropriate adoption of recommended practices, or technology that actually works once implemented.
In the world of healthcare information systems, the mantra of standards and intra-operability is near a religion. Standards bodies proclaim them, governments mandate them, and as much as they can without being noticed, industry pays lip service to them, satisficing wherever they can. For such a pervasive technology, and we should see technical standards as exactly that – another technical artifact – it is surprising that there appears to be no evidence base that supports the case for their use. There seem to be no scientific trials to show that working with standards is better than not. Commonsense, communities of practice, vested interests and sunk costs, all along with the weight of belief, sustain the standards enterprise.
For those who advocate standards as a solution to system change, I believe the growing challenge of systems inertia has one a disturbing consequence. The inevitable result of an ever-growing supply of standards meeting scarce human attention and resource should from first principles reasoning lead to a new ‘Malthus’ law of standards – that the fraction of standards produced that are actually complied with, will with time asymptote toward zero[1]. To paraphrase Nobelist Herb Simon’s famous quip on information and attention, a wealth of standards leads to a poverty of their implementation[1].
It should come as no surprise then that standardisation is widely resisted, except perhaps by standards makers. Even then they tend to aggregate in competing tribes pushing one version of a standard over another. Unsurprisingly, safety goals remain elusive and evidence-based practice to many clinicians seems an academic fantasy. Given that clinical standards are often not evidence-based, such resistance may not be inappropriate[2 3].
In IT, standards committees sit for years arguing over what the ‘right’ standard is, only to find that once published, there are competing standards in the marketplace, and that technology vendors resist because of the cost of upgrading their systems to meet the new standard. Pragmatic experience in healthcare indicates standards can stifle local innovation and expertise[4]. In resource-constrained settings, trying to become standards compliant simply moves crucial resources away from front-line service provision.
There is a growing recognition that standards are a worthy and critical research topic[5]. Most standards research is empirical and case based. An important but small literature examines the ‘standardisation problem’[6] – the decision to choose amongst a set of standards. Economists have used agent-based modelling in a limited way to study the rate and extent of standards adoption[7]. Crucially, standards adoption is seen as an end in itself with current research, and there seems little work examining the effect of standardisation on system behaviour. Are standards always a good thing? There seems to be no work on the core questions of when to standardise, what to standardise, and how much of any standard one should comply with.
Clearly, some standardisation may be needed to allow the different elements of a complex human system to work together, but it is not clear how much ‘standard’ is enough, or what goes into such a standard. My theoretical work on the continuum between information and communication system design provides some guidance on when formalisation of information processes makes sense, and when things are best left fluid[8]. That framework showed that in dynamic settings where there is task uncertainty, standardisation is not a great idea. Further information system design can be shaped by understanding the dynamics of the ‘conversation’ between IT system and user, and by the task specific costs and benefits associated with technology choice[9 10].
It is remarkable that these questions are not being asked more widely. What is now needed is a rigorous analysis of how system behaviour is shaped and constrained by the act of standardisation, and whether we can develop more adaptive, dynamic approaches to standardisation that avoid system inertia and deliver flexible and sustainable human systems.
This blog is excerpted from my paper “Stasis and Adaptation“, which I gave in Copenhagen earlier this year, to open the Context-Sensitive Healthcare Conference. For an even more polemic paper from the same conference, check out Lars Botin’s paper How Standards will Degrade the Concepts of the Art of Medicine.
1. Coiera E. Why system inertia makes health reform so hard. British Medical Journal 2011;343:27-29 doi: doi:10.1136/bmj.d3693[published Online First: Epub Date]|.
2. Lee DH, Vielemeyer O. Analysis of Overall Level of Evidence Behind Infectious Diseases Society of America Practice Guidelines. Arch Intern Med 2011;171:18-22
3. Tricoci P, Allen JM, Kramer JM, et al. (2009) Scientific Evidence Underlying the ACC/AHA Clinical Practice Guidelines. JAMA 301: 831-841. JAMA 2009;301:831-41
4. Coiera E. Building a National Health IT System from the Middle Out. J Am Med Inform Assoc 2009;16(3):271-73 doi: 10.1197/jamia.M3183[published Online First: Epub Date]|.
5. Lyytinen K, King JL. Standard making: A critical research frontier for information systems research. MIS Quarterly 2006;30:405-11
6. The Standardisation problem – an economic analysis of standards in information systems. Proceedings of the 1st IEEE Conference on standardization and innovation in information technology SIIT ´99 1999.
7. Weitzel T, Beimborn D, Konig W. A unified economic model of standard diffusion: the impact of standardisation cost, network effects and network topology. MIS Quarterly 2006;30:489-514
8. Coiera E. When conversation is better than computation. Journal of the American Medical Informatics Association 2000;7(3):277-86
9. Coiera E. Mediated agent interaction. In: Quaglini BaA, ed. 8th Conference on Artificial Intelligence in Medicine. Berlin: Springer Lecture Notes in Artificial Intelligence No. 2101, 2001:1-15.
10. Coiera E. Interaction design theory. International Journal of Medical Informatics 2003;69:205-22
The Guide to Health Informatics 3rd Edition 
Enrico, you seem to be arguing against some standards, and for others, but I’m not sure which ones. Presumably you are in favour of the standards on which you can build useful systems (such as blogs!) but against those that dictate clinical practice.
On the subject of enabling behaviour, is there really no research into the outcomes of sharing information?
From where I sit – authoring the technical standards to enable the exchange that most people think the future is made out of – the technical standards would be much cheaper to use if purchasers and clinical users all agreed what they wanted. But these are exactly the kind of standards you don’t think should exist. So… they shouldn’t exist yet? Or never?
Why is health different from other domains then? The others standardise the business practices, and IT follows, instead of standardising the IT. (Well, that’s a gross generalisation, so you can give a grossly generalised answer ;-))
I am not against standards but the untested belief that standards are a natural good in all cases.
I am for studying standards, from design to use, to measured outcome, to discover what works and what doesn’t.
It appears to me absurd that we are in a situation where the most pervasive of all technologies, standards and standards processes, appear above question and study – that’s all.
Yet is obvious to everyone that standards are problematic in some settings and terrific in others. I’m sure learned folks like yourself already have a good feel for what makes a ‘good’ standard or standards process, compared to a ‘bad’ one – and we need to move from such empirical insights into something more evidence based and robust.
We also need to remember that the standards process does not end there. Post promulgation, there is the thorny issue of adoption, and implementation, and measured impact.
So, if you read carefully, you’ll see I’m not anti standards, I’m pro the ‘science of standards’.
Indeed, this is really a call to arms – let’s make standards and standards setting, and standards implementation and adoption, evidence based and scientifically rigourous.
well, evidence is always good, and I’m not against it, but I am inclined, after my little experience in standards, to think that it’s a lottery – the effect you think you’ll have is not the one you’ll get. It might be better or worse. The essence of research is good controls, with well controlled variables, and I just don’t believe in that. I’m afraid 😦
Now that’s too bleak a view, even for me!
If you are saying it is a lottery when standards work or don’t, then you are performing no better than chance …… which means no effect.
What you really are saying I think is that in a complex (or complicated) live system, that attributing cause and effect is hard. We are however not stuck only with classical controlled experiments to help in untangling causality.
This is a standard problem for ascribing benefit to most any health systems intervention. There are plenty of study design and methods available to tease things apart. Remember, there has never been a controlled trial to *prove* smoking causes lung cancer … just many lines of evidence drawn together.
So I content that standards are no more difficult or special than any other organisational level intervention and are a natural and worthy (and indeed very interesting) objects of study.
Well, ok, that’s true. I’ll try not to be so bleak. But no better than chance isn’t no effect – you’re being too bleak there 😉
Still, can we get enough data to tease things apart?
See top links on http://wolandscat.net/health-informatics/ for many reasons for the failure.
And arguably http://wolandscat.net/2013/09/29/the-real-reason-most-software-fails/
Thanks – this link reiterates the fundamental premise on the role of models in the design of any technology made in Chapter 1 of my textbook. See https://coiera.com/textbook-resources/chapter-1-models/
We need a more an agile approach to standards in health informatics. I have a blog in progress on what they look like but see Standards are a barrier to innovation? http://www.woodcote-consulting.com/in/
[…] Coiera puts it like this in his excellent post Are standards necessary? (my […]