Analysis of CDA R2 testing tools - most requirements are neither tested nor respected.
Publication date: Feb 13, 2015
During the recent IHIC conference IHE Europe published an analysis that shows serious weaknesses in the current CDA validation tools.
When it comes to the validation of CDA documents one can distinguish a number of steps which should form the basis for a full validation of a CDA document:
any CDA document has to be
- well-formed XML,
- valid with respect to the published CDA schema,
- conform with all requirements of the CDA abstract HL7v3 model - inclusive of all of the CDA requirements that aren't expressed as part of the schema, because the schema
language is simply not expressive enough to deal with all of the requirements as expressed in the abstract model,
- conform to the requirements as expressed in a CDA Implementation Guides - more often than not a process where multiple templates
(as defined by the implementation guide) have to be validated.
The object of the analysis by IHE Europe (Abderrazek Boufahja, Eric Poiseau, Guillaume Thomazon and Anne-Gaëlle Bergé) was to examine
to which degree the existing CDA Tools as well as existing national/regional CDA projects meet the requirements of the 3rd validation step.
The authors identified 160 requirements which should be tested by any CDA validation tool.
Once they had a list of requirements they tested existing CDA validation tools to see whether those would detect the issues in question, and they
tested example messages defined by national/regional projects to see whether those projects used a strong enough validation process.
Boufahja presenting during the IHIC 2015
Even if the list of 160 requirements should be incomplete, or if some of those requirements are erroneous, the results of the analysis is a clear indication
that more work is needed by all parties concerned:
- The CDA tools that were tested are: Trifolia, MDHT, Eclipse Instance Editor, Art-Decor, the NIST validation tool, and the IHE Gazelle ObjectsChecker tool.
If we exclude the new IHE Gazelle ObjectsChecker tool (which does catch almost all of the 160 requirements) the old
Eclipse Instance Editor (no updates since 2011 - no longer available for download) performed best, the remaining tools have a worse performance.
Now some will argue (as I do) that the aim of some of these tools (e.g. Trifolia, Art-Decor) has never been to deal with the 3rd validation step,
but only to focus on the 4th validation step (templates). Still, the validation strength of MDHT and the NIST validation Tool should be improved upon.
Note that the Everest toolkit hasn't been tested - the tool was unknown to IHE Europe at the time of the analysis.
- They also tested message examples from 10 national/regional projects, the average number of errors found in the example CDA documents ranged between 2 and 44.
So there are projects out there where (on average) a CDA document has 44 known issues. It seems the various CDA projects also have to beef up their testing process.
The list of most frequent errors can be found in the paper, they are more or less the same as the ones we found in
an earlier similar study in 2008 -
which just goes to show that projects should move away from XML based validation and should instead focus on model based validation.
The actual paper (published under the CC-BY-NC-ND license) can be found on the
EJBI website (special 2015 issue related to IHIC),
a scanned copy (PDF) can be found on the
In my opinion this was the most interesting paper presented at the IHIC meeting in Prague earlier this week - toolkit developers as well as national/regional CDA projects should
learn its lessons and beef up their support for stricter CDA validation.
PermaLink to this page: http://www.ringholm.com/column/HL7_CDA_Conformance_testing_tools_analysis.htm
Index of columns:
- News from the FHIR DevDays in Amsterdam (Dec 15, 2016)
- Next XDS Release (Oct 27, 2016)
- Five years of FHIR (Aug 11, 2016)
- Interoperability projects in Ireland - FHIReland (Mar 14, 2016)
- 2016 FHIR Jedi Calendar (Jan 06, 2016)
- Top 10 HL7 videos watched in 2015 (Dec 24, 2015)
- Update from the trenches on CDA R2.1/R3 and HL7v2. (Oct 15, 2015)
- FHIR DevDays - UK GP System APIs (Sep 16, 2015)
- IHE XDS - testing and implementation tools (Aug 25, 2015)
- Most often implemented IHE Profiles (Jun 08, 2015)
- Why we sponsor the HL7 WGM (May 10, 2015)
- FHIR in Paris (Apr 21, 2015)
- Mapping HL7v2 messages to FHIR. (Apr 13, 2015)
- Analysis of CDA R2 testing tools - most requirements are neither tested nor respected. (Feb 13, 2015)
- HL7 and IHE in Sweden (Feb 08, 2015)
- 2015 FHIR Chiefs Calendar (Jan 07, 2015)
- The Merry FHIR Choir caroling the 12 Days of Christmas (Dec 09, 2014)
- Chicago FHIR Update (Oct 13, 2014)
- Internationalization of HL7 (Sep 25, 2014)
- New XDS Advanced training course on offer by IHE Services and IHE Academy (Jul 14, 2014)
- Recent and Future developments of the DICOM standard (Mar 06, 2014)
- Top 10 HL7 videos watched in 2013 (Jan 02, 2014)
- Report from the HL7 WGM in Cambridge (Oct 16, 2013)
- Documenting the history of HL7 (Sep 03, 2013)
- Histology Lab Device Automation using HL7 version 2 (Jul 23, 2013)
- HL7 FHIR Elevator Pitch (Jul 15, 2013)
- Interoperability Standards - the no-sales pitch (Jul 09, 2013)
- HL7 UK - new landscape, new opportunities (Jun 26, 2013)
- Validation and error correction at the IHE Connectathon (Apr 25, 2013)
- CDA Implementation Guides - (not) invented here (Apr 17, 2013)
- Usage of IHE Profiles (Feb 25, 2013)
- 10 year anniversary - Dutch Ringholm HL7 v2 training courses. (Feb 19, 2013)
- About IHE Academy and new IHE training courses (Jan 12, 2013)
- CDA implementation experiences in the UK (Dec 04, 2012)
- Musings on free HL7 IP (Oct 01, 2012)
- HL7 Connectathons (Sep 09, 2012)
- Renovate HL7 version 3 (Aug 03, 2012)
- Frequency of use of HL7 message types (Jul 24, 2012)
- Lighting the FHIR, HL7s new major interoperability standard (Jun 15, 2012)
- Reflections on the HL7 membership model - the affiliate life cycle (Dec 28, 2011)
- Thinking like an OWL reasoner (Sep 17, 2011)
- RFH (Resources for Health): HL7 version 3 taken to the next step (Aug 18, 2011)
- What's so great about the HL7 organization? (Aug 04, 2011)
- Kerndossier: een Nederlandse versie van CCD (Dutch, May 03, 2011)
- A HL7 RIMBAA update (Apr 21, 2011)
- Timezone Hotel (Mar 29, 2011)
- HL7 and openEHR are cooperating (finally) (Jan 21, 2011)
- Increasing demand for IHE training courses (Dec 18, 2010)
- Context issues with the IHE QED profile (Dec 15, 2010)
- The changing role of HL7 country organizations (Jul 16, 2010)
- Implementing HL7 version 3 - the book (May 06, 2010)
- Adding openness to a closed world (Feb 09, 2010)
- How to lower the hurdle for HL7 v3 implementers (Jan 21, 2010)
- HL7 v3 deployment statistics (Dec 17, 2009)
- There's Trouble in Paradigm (Sep 25, 2009)
- Internationalization of HL7 (Sep 24, 2009)
- HL7 UK signs deal with Ringholm to deliver HL7 v2/v3 training courses in London (Sep 17, 2009)
- The use of HL7 in South Africa (Aug 20, 2009)
- The Next Web Conference in Amsterdam (Apr 17, 2009)
- The HL7 UK AGM and RIMBAA (Apr 16, 2009)
- The HL7 Wiki reaches 2000 pages (Mar 02, 2009)
- The HL7 roadmap for CDA R3 and the CCD (Jan 17, 2009)
- HL7 Affiliates Meeting in Orlando (Jan 11, 2009)
- Swiss and Dutch HL7 News (Dec 31, 2008)
- Devices and Prizes (Nov 22, 2008)
- HL7 in Norway: a situation report (Sep 02, 2008)
- Russian whitepaper (Jul 09, 2008)
- The HL7 Interoperability Conference - IHIC 2008 (May 30, 2008)
- HL7 creates a RIM Based Application Architecture (RIMBAA) group (May 18, 2008)
- Notes from the HL7 WGM in Phoenix (May 08, 2008)
- Germany embraces CDA eReferral document specification (May 02, 2008)
- HL7 v3 RIM based applications: an unintended side effect (Jan 19, 2008)
- Collaborative Tools (Jun 21, 2007)
- HL7 ist Pflicht in der deutschen Telematikinfrastruktur (German, Mar 16, 2007)
- HL7 based Tree inventory system (Jan 30, 2007)
- The link between HL7 and Open Source Software (Jan 06, 2007)
- Workflow Bribery (Sep 15, 2006)
- Timezones in HL7 (Jan 23, 2004)
- Controlled vocabularies: "@*%!!!" ? (Sep 01, 2003)
- Trusting the other Party (Nov 01, 2002)
About Ringholm bv
Ringholm bv is a group of European experts in the field of messaging standards and systems integration in healthcare IT.
We provide the industry's most advanced training courses and consulting on healthcare information exchange standards.
See http://www.ringholm.com or call +31 33 7 630 636 for additional information.
Rene is the Tutor-in-chief of Ringholm.