Pipes, Hats … and OpenClinica: Digesting HL7 in OpenClinica

If you’re an OpenClinica administrator somewhere, the chances are good somebody has asked you: “Can OpenClinica handle HL7 messaging?”

“No, it doesn’t,” you’ve said.

You probably said that with a sigh of relief because HL7 is a byzantine data exchange standard whose complexity keeps an army of consultants employed and drives neophytes like myself to madness.  The HL7 2.X specification uses eye-fatiguing pipes (“|”) and hats (“^”) as delimiters and has been referred to by experts as the “non-standard standard” (see this). Unfortunately, it is also the lingua franca of health-care messaging currently, and will likely continue to be for a long time to come.

So, it is with a heavy sense of resignation that we here at Geneuity are taking up the challenge to make OpenClinica fluent in HL7. As a contract clinical laboratory, we are particularly interested in having OpenClinica able to digest HL7 ORU messages that convey lab results.  This article details our first pass at the problem.

Our approach is shown in Figure 1.  It makes use of Mirth and a new web service in OpenClinica developed by Geneuity called EventDataInsert.  As shown, an HL7 message containing a lab result is sent by TCP to a Mirth channel which is configured to transform it into a SOAP message palatable to EventDataInsert.  EventDataInsert reads the message and then sees if the specimen has already been accessioned into OpenClinica.  If so, it inserts the data into the underlying database and signals a successful entry.  If not, it does nothing and signals a rejection.  These signals are transmitted back to Mirth which issues a standard HL7 acknowledgment (ACK) message coded with either ‘AA’ for ‘Application Accept’ or ‘AR’ for ‘Application Reject’.  It is the responsibility of whoever (or whatever) sent the HL7 message in the first place to follow up when a lab result is rejected.

To develop this strategy, we used several tools.  To generate HL7 test messages, we utilized the HL7 generator (freely available here) made by the people responsible for the ELINCS initiative.  To send and receive HL7 messages to and from Mirth via TCP, we used Netcat, another freely available utility.

And there you have it!  Of course, the HL7 standard covers much more than the delivery of lab results, but this exercise is most relevant to our concerns and represents an important first step in making OpenClinica talk the talk when it comes to HL7.

HL7 Strategy for OpenClinica

Figure 1:  HL7 Strategy for OpenClinica

First, a HL7 message conveying lab results is sent to a Mirth channel listening for TCP requests.  Mirth parses the message and transforms it into a SOAP message which it then hands off to the EventDataInsert webservice listening within OpenClinica.  EventDataInsert looks to see if the specimen to which the lab result pertains has been accessioned into OpenClinica’s underlying database.  If so, it inserts the results and signals back to Mirth that fact.  If not, it enters nothing and signals back to Mirth that it did nothing.  Mirth digests these signals and sends back to the sender an appropriately configured ACK message via TCP.

Facilitated Data Entry of Lab Results Using OpenClinica’s New Web Services Feature

As mentioned previously, we at Geneuity Clinical Research Services are big fans of OpenClinica and are even more so now with the upcoming release of version 3.0 with its new web services capability.  This article describes how we exploit this new feature to help automate entry of lab results, a particularly important topic given that we do lots of batch testing of specimens and oftentimes test the same specimen for many different analytes.

Prior to 3.0, you had three options when it came to CRF data entry.  The first was to log into OpenClinica’s web interface and manually enter your data.  This was no problem so long as you didn’t have lots and lots of data.  But we did.

Alternatively, you could upload a flat file of your data as long as it was formatted in XML and associated with the appropriate subject id’s and visit descriptions.  Assembling this file wasn’t trivial though and manually looking up each specimen’s subject and event nearly defeated the purpose of the procedure, which was to save time and effort.

Finally, you could do what we did: write custom code to automate the job.  Lab data is amenable to this sort of approach because it is always tagged with something called an accession number that uniquely identifies it.  When designing CRF’s, we always make sure to include a field for the event’s accession number, and when a specimen first arrives through our door the first thing we do is to log into OpenClinica and enter the specimen’s accession number in the appropriate event’s CRF.  Because the number is unique to the study, this entry effectively tags the event and provides a ‘hook’ inside the database so that the event_crf_id of any data item subsequently  annotated with the accession number can be easily looked up using a database query like so: ‘SELECT event_crf_id FROM item_data WHERE value = ‘<accession_number>’.  This, in turn, gives you the requisite information to insert the lab data thusly: ‘INSERT INTO item_data VALUES (‘event_crf_id’, ‘value’ …’ provided you also know the item_id.

To implement this strategy, we wrote custom servlets that operated within the context of our OpenClinica installations.  More recently, we configured MirthConnect channels to do the same.   They worked well and data entry was greatly expedited, but the coding was complex and had to be refactored over and over again for each study and for every CRF change.  While helpful, this strategy wasn’t sustainable in the long run.

Luckily, the latest version of OpenClinica provides a way out.  It incorporates the Spring WS Framework which allows programmers to write something called a ‘web service.’  A web service digests and acts upon XML data sent to it on an on-demand basis over a network.  The source need not be a human being uploading data on a web form, but, more usefully, it can be, say, a clinical testing platform automatically spitting out HL7 messages.  This, of course, is ideal in our case.  So we wrote a web service called ‘EventDataInsert’ that parses XML containing lab data values annotated with accession numbers and item names, looks up the corresponding event_crf_id’s and item_id’s, and inserts the data into item_data accordingly.  The service is generic enough so that it doesn’t have to be refactored for each and every study, but it does make some critical assumptions.  Namely, it assumes that both accession numbers and item names are unique.  So care has to be taken to ensure both these preconditions are met.

The power of EventDataInsert doesn’t just lie in the fact that it handles inserts on an unattended basis, but also in that, like most web services, it requires only simple XML as input.  The latter makes the source of the data irrelevant as long as it can be correctly mapped and transformed into XML.  We often use MirthConnect to do this, using it’s easy-to-use graphical interface to configure channels between incoming raw data and OpenClinica’s web-service interfaces.

The figure below shows a typical deployment of OpenClinica at Geneuity.  MirthConnect is used not only to get data into OpenClinica but also to generate canned PDF reports of the results.  This scenario works for us and gets easier and easier to maintain as OpenClinica evolves new electronic data capture features and makes old ones ever more robust.

Diagram of OpenClinica at Genuity Clinical Research Services
Diagram of OpenClinica at Genuity Clinical Research Services