OpenClinica 3.1 (project “Amethyst”) Preview

The next major milestone for OpenClinica (project code-named “Amethyst”) will be OpenClinica version 3.1. While this release contains roughly 85 tweaks, fixes, and enhancements, this post describes some of the more significant enhancements that will be included (check out the Amethyst project roadmap page for a full list—note: login required).

New features in OpenClinica 3.1 will include:

  • Dynamic Items in CRFs
  • Audit Log and Discrepancy Note Data with ODM Exports
  • Assignment of Failed Validation Check Discrepancy Notes
  • Relative Paths for Item OIDs in Rules
  • Discrepancy Note Flag Colors in the CRF
  • Modularization
  • Multiple Site Level Users Allowed Access to an Event CRF
  • Simple Conditions

Dynamic Items in CRFs

OpenClinica 3.1 will support Showing and Hiding of CRF Items using both the CRF template and a Rules file. When a user enters data and saves a CRF section, data fields may be shown or hidden based on value(s) that user provided in one or more other fields on that form. This capability is also commonly referred to as “skip patterns.” A simple example would be a CRF question asking if the patient is male or female. If the value provided is female, a pregnancy related question could then be displayed to the user entering data and all questions associated with males could be hidden from view.

Audit Log and Discrepancy Note Data with ODM Exports

Audit Log data and Discrepancy Note data for a subject will be available in the ODM data extract format with OpenClinica extensions. This will allow the user to have all of the clinical data, audit log data, and discrepancy note data in a single data file.

Assignment of Failed Validation Check Discrepancy Notes

Currently, OpenClinica supports assignment and workflows around only the Query type of Discrepancy Note. OpenClinica’s “assignment” capability will be expanded to also include the Failed Validation Check type of Discrepancy Note. For Failed Validation Checks, the first note in the Discrepancy Note thread will not be assigned, but Data Managers, Study Directors and Monitors will be allowed to assign the thread to a user for review/resolution. OpenClinica will also restrict the Clinical Research Coordinators and Investigators (both site-level users) from setting the status of Failed Validation Checks to Closed.

Relative Paths for Item OIDs in Rules

A new enhancement to Rules authoring will allow the Rule creator to write one Rule Assignment for a particular CRF Item, and have the Rule execute wherever that Item’s OID is used throughout all of the study events. This increases the “portability” of rules, allowing the user to write one Rule, and have it apply multiple times rather than having to author multiple Rules and multiple Rule Assignments.

As an example, a Rule Assignment would need to contain the following path in OpenClinica 3.0.x:


If the CRF was used in multiple events, the creator of the Rule file would have to specify the path to the other event as well.

In 3.1, all the user has to do is write:


and the Rules will be executed wherever that Item shows up in the study.

Discrepancy Note Flag Colors in the CRF

OpenClinica 3.1 will include additional Discrepancy Note flag colors that correspond to the various statuses of a particular thread. Currently in OpenClinica, if a Discrepancy Note thread exists, the flag will always display in a red color regardless of the Discrepancy Note status. In 3.1, the color of the flag will be reflect the “most severe” status of any thread that is on a particular item (more than one thread may exist for any item). For example, if there is a Closed thread and an Updated thread on one item, the color of the flag will be yellow representing the Updated status. If there is just a Closed thread, the color of the flag will be black. To support people who are color blind or shade blind (like myself) there will be a roll over when you put your mouse on the flag, showing you the number of threads and each of their statuses when Viewing a CRF.

Modularization & New Web Services

Modularization is defined as a software design technique that increases the extent to which software is composed from separate parts, called modules. Conceptually, modules represent a separation of concerns, and improve maintainability by enforcing logical boundaries between components.

What this means for OpenClinica is we have started to separate the application into multiple pieces. In version 3.1, we have modularized the web application from the web services functionality. This will allow new web services to be developed on separate release timelines from the main web application, facilitating the system’s extensibility.

In addition, we are release some web services with 3.1, (these were contributed by Colton Smith of Geneuity and openXdata):

  • CRF Data Import
  • Retrieve a list of studies
  • Retrieve a list of events and their CRFs
  • Retrieve a list of subjects and their events

Multiple Site Level Users Accessing an Event CRF

OpenClinica 3.1 will allow different site level users access to an Event CRF, even if they are not the conceptual “owner” of that CRF. In prior versions of OpenClinica, once a user began data entry in a CRF, the system prevented other users from adding information or data to the CRF until it had been marked complete. The new feature will allow a second user to continue entering data before the CRF is marked complete.

This change to OpenClinica will also help facilitate the ease of recording adverse events in a separate CRF. A user will not have to mark it complete in order for another user to provide additional adverse events that have occurred for a particular subject. In addition, this new functionality will prevent users from accessing an Event CRF if another user already has the form open. In this case, the second user will receive a message saying that the form is currently being accessed by another user.

Simple Conditions

In addition to the Dynamics capabilities that will be part of 3.1, we have added a feature called Simple Conditions. This feature is similar to Dynamics in many ways, but can be implemented through the CRF Template directly rather than writing a separately Rules XML document.

With Simple Conditions, a person creating an OpenClinica CRF will have the ability to designate Items as “hidden” from the data entry person’s view until a particular option is chosen for another CRF Item. The data entry person will not have to click “save” on the form–instead, as soon as the option is selected, this hidden field will be shown in real time.An example of the type of use case this feature targets, is a CRF question with two fields, one for “race” and the other for “comments” (which is hidden). If the data entry person selects the value of “other” for the race field, the hidden comments field will be display underneath.

Akaza Research is excited about bringing OpenClinica 3.1 to the community! Your comments and feedback are appreciated. Please check back in next week or so for an update on our timelines for Alphas, Betas and a Production release.

XForms and OpenClinica

Here at Geneuity, we do a lot of contract research work for sponsors of clinical trials.  That sometimes means developing and validating custom assays that aren’t available off the shelf. And this, in turn, often means developing special interfaces for data capture that can’t be configured inside OpenClinica (as yet).

Does that mean we jettison OpenClinica?

No, absolutely not!

Since OpenClinica is open source and dedicated to interoperability, it can be easily extended.  This article describes the specifics of one such case involving XForms.

A while back, Geneuity was developing an ELISA to measure the abundance of a specific protein in plasma.  We needed a single web form with a spreadsheet-like feel and functionality in which lab technicians could enter data, interpolate unknowns, send the resulting interpolated values for insertion into OpenClinica and then perform source data verification.  Additionally, the form needed to be pre-populated with the accession numbers of specimens requiring testing.  The form had to work on any browser and not be dependent on any browser plugins.  And we had to have it fast.

Impossible? No, not with XForms and the open source XForms renderer betterFORM.

XForms have several advantages.  First of all, you author the forms in relatively simple XML while the complicated AJAX that makes the spreadsheet-like feel and functionality is rendered by the renderer and presented to the client browser for you.  Secondly, XForms allows you to easily call upon webservices to populate data items in the form.  In our case, we developed three such services: one to return a list of accession numbers corresponding to specimens yet to be tested along with corresponding hyperlinks directed inside OpenClinica for rapid source data verification (SDV); another to calculate and graph a standard curve and to interpolate the specimen unknowns; and still another to insert the interpolated values into OpenClinica.  The data flow is diagrammed in Figure 1.  Although a great deal is going on behind the scenes, the user experience is seamless as a single form is being used while data retrieval and refreshment are being done without interrupting the display (thanks to AJAX as automatically implemented by betterFORM). The form itself is pictured in Figure 2.

And there you have it.  The bottom-line: no matter how complicated your data capture requirements are, you can count of the interoperability of OpenClinica.

Figure 1. First, the lab tech initially summons the XForm which calls upon OpenClinica via a webservice for the list of specimens for which there are no test results. The tech enters the data and clicks on the appropriate button to call for data interpolation and calculation. After review of the results, the tech then submits the data for insertion into OpenClinica (for more on this see here). Finally, the tech performs source data verification using the hyperlinks populated in the XForm as the result of step 1.
Figure 2. This is a snapshot of the XForm as it exists after interpolation and calculation. Only two specimens are shown, but usually there are many more in a batch.

Pipes, Hats … and OpenClinica: Digesting HL7 in OpenClinica

If you’re an OpenClinica administrator somewhere, the chances are good somebody has asked you: “Can OpenClinica handle HL7 messaging?”

“No, it doesn’t,” you’ve said.

You probably said that with a sigh of relief because HL7 is a byzantine data exchange standard whose complexity keeps an army of consultants employed and drives neophytes like myself to madness.  The HL7 2.X specification uses eye-fatiguing pipes (“|”) and hats (“^”) as delimiters and has been referred to by experts as the “non-standard standard” (see this). Unfortunately, it is also the lingua franca of health-care messaging currently, and will likely continue to be for a long time to come.

So, it is with a heavy sense of resignation that we here at Geneuity are taking up the challenge to make OpenClinica fluent in HL7. As a contract clinical laboratory, we are particularly interested in having OpenClinica able to digest HL7 ORU messages that convey lab results.  This article details our first pass at the problem.

Our approach is shown in Figure 1.  It makes use of Mirth and a new web service in OpenClinica developed by Geneuity called EventDataInsert.  As shown, an HL7 message containing a lab result is sent by TCP to a Mirth channel which is configured to transform it into a SOAP message palatable to EventDataInsert.  EventDataInsert reads the message and then sees if the specimen has already been accessioned into OpenClinica.  If so, it inserts the data into the underlying database and signals a successful entry.  If not, it does nothing and signals a rejection.  These signals are transmitted back to Mirth which issues a standard HL7 acknowledgment (ACK) message coded with either ‘AA’ for ‘Application Accept’ or ‘AR’ for ‘Application Reject’.  It is the responsibility of whoever (or whatever) sent the HL7 message in the first place to follow up when a lab result is rejected.

To develop this strategy, we used several tools.  To generate HL7 test messages, we utilized the HL7 generator (freely available here) made by the people responsible for the ELINCS initiative.  To send and receive HL7 messages to and from Mirth via TCP, we used Netcat, another freely available utility.

And there you have it!  Of course, the HL7 standard covers much more than the delivery of lab results, but this exercise is most relevant to our concerns and represents an important first step in making OpenClinica talk the talk when it comes to HL7.

HL7 Strategy for OpenClinica

Figure 1:  HL7 Strategy for OpenClinica

First, a HL7 message conveying lab results is sent to a Mirth channel listening for TCP requests.  Mirth parses the message and transforms it into a SOAP message which it then hands off to the EventDataInsert webservice listening within OpenClinica.  EventDataInsert looks to see if the specimen to which the lab result pertains has been accessioned into OpenClinica’s underlying database.  If so, it inserts the results and signals back to Mirth that fact.  If not, it enters nothing and signals back to Mirth that it did nothing.  Mirth digests these signals and sends back to the sender an appropriately configured ACK message via TCP.

Rapid Deployment of New Functionality in OpenClinica Using MirthConnect

In a previous article, we describe how we at Geneuity Clinical Research Services exploit OpenClinica’s new web services feature to automate the entry of lab data keyed to accession numbers.  Here, we describe more fully how and why we use MirthConnect.

Started in 2006, MirthConnect is an open source project sponsored by the Mirth Corporation of Irvine, CA.  It is middleware designed to transform, route and deliver data.  It supports HL7, X12, XML, DICOM, EDI, NCPDP and plain old delimited text.  It can route via MLLP, TCP/IP, HTTP, files, databases, S/FTP, Email, JMS, Web Services, PDF/RTF Documents and custom Java/JavaScript.  MirthConnect has been likened to a Swiss army knife and justifiably so.

Channels are the heart and soul of a MirthConnect installation.  A channel is user defined and has a source and a destination.  A source may be a flat file residing on a remote server or a web service call or a database query or even another channel—whatever you like, it doesn’t matter. A destination may be to write a PDF document, email somebody an attachment or enter data into a database.  Again, whatever!

To illustrate, say you want to poll a database and generate a weekly report.  No sweat! Using MirthConnect’s easy-to-use drag-and-drop template-based editor, define a channel with a database reader as a source, and a document writer as a destination, fill in details like user names, passwords and machine names, define which database fields you want to retrieve and how you want to display the data, and you’re done!  MirthConnect’s daemon handles the rest based on your channel’s configuration.

Once defined, a channel can be exported as XML for later import into another MirthConnect installation.  This is all done with the point and a click of a mouse.

At Geneuity, we use MirthConnect to get data in and out of OpenClinica.  Originally, we used custom JAVA code to do this.  But once we found MirthConnect, we quickly realized we were reinventing the wheel.  Why do that?

Here’s a concrete example.  Consider the very simple CRF from a mock OpenClinica installation shown in Figure 1.  It has three groups of items: accessioning, results and reportage.  When a specimen arrives at Geneuity, the lab tech looks up the patient and event pairing in the subject matrix as specified by the requisition and types into the CRF the accession number, the receipt date and any shipping deviations.  This is done by hand and is indicated as step 1 of Figure 2.

Then, as shown in step 2 of Figure 2, the tech tests the specimen at the testing platform.  In step 3, the platform spits out the data whereupon a collection of MirthConnect channels operating in tandem parses the results, transforms them into SOAP messages and sends them to the EventDataInsertEndpoint web service feature of OpenClinica for upload into the CRF fields designated ‘Assay date’ and ‘Analyte concentration’.

After the tech reviews the data and marks it complete, another collection of channels polls the database for results newly marked complete, generates and delivers PDF reports of the corresponding data (step 4) and then reports back to OpenClinica (step 5) via EventDataInsert the details of the reportage, including status, time and any errors (see the third and last item grouping labeled ‘REPORTAGE’ in Figure 1).

The scenario outlined above requires NO CUSTOM CODE beyond the channel configurations and these are encapsulated and standardized by design.  As such, you don’t need an army of coders on staff to develop and maintain them.

Both OpenClinica and MirthConnect are great as standalone products.  Linked together, however, they really sizzle.

Figure 1: A simple CRF from a mock OpenClinica installation
Figure 1: A simple CRF from a mock OpenClinica installation
Figure 2: This shows how the different item groupings in the CRF depicted in Figure 1 are populated.  Values for items under ACCESSIONING are entered manually by the lab tech.  Values for items under RESULTS are populated by the Mirth channels continuously listening for in-coming data from the clinical testing platform.  Values for items under REPORTAGE are populated by a distinct set of  Mirth channels responsible for polling and reporting newly completed results.
Figure 2: This shows how the different item groupings in the CRF depicted in Figure 1 are populated. Values for items under ACCESSIONING are entered manually by the lab tech. Values for items under RESULTS are populated by the Mirth channels continuously listening for in-coming data from the clinical testing platform. Values for items under REPORTAGE are populated by a distinct set of Mirth channels responsible for polling and reporting newly completed results.

OpenClinica and the Paper Chase: A Case Study

The developers of OpenClinica like to say they are ‘powering the electronic data capture (EDC) revolution’ in clinical data management and indeed that’s the case.  However, sometimes there’s no substitute for paper, especially when it comes to tracking and handling laboratory specimens.  We at Geneuity should know: testing lab specimens for clinical trials is our business.  This article shows how OpenClinica and paper records can augment one another through the use of URL-encoding barcodes.

Readers of this blog probably already know the drill:  a specimen is collected, a paper requisition is attached and the whole kit and kaboodle is mailed to a laboratory for testing.  When a specimen arrives at Geneuity, we log into OpenClinica’s web interface, look up the patient and event pairing in the subject matrix as specified by the requisition and then finally type in the values for items like accession number, receipt date, shipping deviations, freezer location and the like.  When one or two specimens arrive, this process (called ‘accessioning’) is no problem; when 20 to 60 arrive, it rapidly becomes tedious.

In a more perfect world, things would be easier and more automated.  Here’s one potential scenario to help achieve this.  In addition to all the necessary information in human-readable form, also print on the requisition a QR barcode that encodes the URL corresponding to the appropriate event case report form (CRF) in the study’s OpenClinica installation.  For instance, consider the example shown in Figure 1.

It encodes: “” (for illustration only, not intended to link anywhere for real).  Upon being scanned with an appropriate hand-held reader (or smart-phone even), this would automatically direct the lab technician’s browser to the correct screen to input the attendant accessioning information for study event number one.  This would eliminate the manual look-up step described above and would reduce tedium and errors as a consequence.

Implementing this idea would require enabling OpenClinica to print out requisitions.  While challenging, this would not be impossible.  Indeed, middle-ware like MirthConnect may make it relatively easy.

Here’s another example of paper and OpenClinica working together, one that Geneuity actually implements currently to facilitate quality control.  Prior to testing a batch of specimens, lab technicians here use a custom Mirth channel to print out details of the samples to be tested.  A dummy example is shown in Figure 2.  As shown, it includes a simple linear barcode of the accession number, the principle means of tracking specimens in a clinical lab.  Scanning this saves the technician from having to type it into the testing platform during set-up.  It also contains a QR barcode specifying the event-specific URL that directs the technician where to go to review the data upon automatic upload into OpenClinica.  Having this URL handy means the technician doesn’t have to look it up in the subject matrix table which oftentimes has hundreds or even thousands of entries.

URL-encoding barcodes are a cheap and reliable way of linking physical objects with on-line databases, and nowhere is this link more critical than when tracking clinical specimens.  Thanks to Akaza’s commitment to open source, it’s easy to incorporate this technology into OpenClinica and to realize the myriad of benefits.

Figure 1: a QR barcode encoding an event-specific URL in a mock OpenClinica installation
Figure 1: a QR barcode encoding an event-specific URL in a mock OpenClinica installation
Figure 2: A list of specimens to be tested reported by a Mirth channel extracting from a mock OpenClinica installation.  The linear barcode helps with data entry at the testing platform while the 2D QR barcode is used to direct the lab tech's browser to the appropriate URL in OpenClinica for subsequent data review after testing is complete.
Figure 2: A list of specimens to be tested reported by a Mirth channel extracting from a mock OpenClinica installation. The linear barcode helps with data entry at the testing platform while the 2D QR barcode is used to direct the lab tech's browser to the appropriate URL in OpenClinica for subsequent data review after testing is complete.

OpenClinica 3.0 Features Preview: Part III

Welcome to the 3rd and final installment of the OpenClinica 3.0 features preview!  This post covers the new Web Services interface that is part of 3.0 and the job scheduler that can be used to automate Data Import and Data Export jobs.

OpenClinica 3.0 allows for programmatic interaction with external applications to reduce manual data entry and facilitate real-time data interchange with other systems.  The OpenClinica web services interface uses a SOAP-based API to allow the registering of a subject and scheduling of an event for a study subject.

OpenClinica provides a WSDL (Web Service Definition Language) that defines a structured format which allows OpenClinica to accept “messages” from an external system. For example, an EHR system could register subjects for a study in OpenClinica without direct human intervention. At the same time, the EHR could also be programmatically scheduling study events for these subjects. More information about the OpenClinica API can be found on the OpenClinica developer wiki.

An early reference implementation conducted by clinical lab Geneuity used the API to create a web service which inserts data programmatically into OpenClinica CRFs directly from laboratory devices. See the post by Geneuity’s Colton Smith below.

Another major productivity tool in 3.0 is the introduction of a Job Scheduler for automating bulk data import and export.  With this feature users can define a job that will generate an export at a specified time interval.  The Jobs Scheduler can also be configured to regularly scan a specific location for CDISC ODM files and run data imports when a new file is available. This feature can be particularly helpful in automating routing functions, such as the incorporation of lab data into OpenClinica from an external system.  The lab data does need to be in a valid CDISC ODM format (this can be accomplished via another great open source tool called Mirth), but it does save a person from entering data in two applications separately.

At time of this post, OpenClinica 3.0 is currently released as a beta3, but the production ready application is soon to follow. The application is passing through the highly rigorous strictures of our quality system (think Navy Seals training for software) and the output will be fully validated and ready of use in roughly a month. Needless to say, I, and everyone else here at Akaza is very excited to be so close to releasing 3.0. It is already quite clear that this release will have a momentous, positive impact on the community.

How a Busy Research Clinical Laboratory Deploys OpenClinica as a Laboratory Information System (LIS)

Here at Geneuity Research Services, we do laboratory tests for clients
conducting clinical trials. We are a one-­stop shop, handling everything
from routine clinical assays to esoteric molecular analyses. We use
OpenClinica as an in-­house LIS to keep track of work­flow and to help
with administrative tasks like billing and specimen archiving. This isn’t
exactly what the developers of OpenClinica had in mind originally. The
application was designed from the point of view of trial sponsors, not their
subcontractors. But it works very well for us nonetheless. In this article,
we briefly describe how we do it.

As everybody who uses OpenClinica knows, defining your event x
CRF matrix is the most important step in configuring a trial. When a
client comes to us with a new trial, we do just that. We are careful
to reproduce the anticipated flow of events and patient groupings
just as designated by the trial’s specifications. “Why bother?”
you’re probably asking. “You’re just a subcontractor. That’s not your
concern.” But it is. Invariably, specimens get mislabeled and
inappropriate tests get ordered, and we’re able to detect many such
mistakes by doing a quick visual inspection of OpenClinica’s patient
x event dashboard with its easy-­to-­take-­in iconification. We feel
that this double-checking is part of our service to the sponsor.

CRF design is our next consideration. In this case, we don’t
try to reproduce everything required from the sponsor’s point of
view. Instead, we design a set of CRF’s that reflects just our role
as a contract clinical laboratory. This means that our CRF’s are far
more narrowly focused and less complicated. But it also means we
include some unique types of fields for pricing, specimen archiving
and the like, items that a trial sponsor usually doesn’t need to
address in their CRF’s but which are very helpful to us.

We are very careful how we name our items when crafting our
CRF’s. Specifically, we use the same terminology across our trials
whenever possible. For instance, the field for a specimen’s accession
number is always named ‘accession_number‘. Likewise, we consistently
use the names ‘freezer‘, ‘received_date‘, ‘price‘, ‘shipping_deviation
and ‘%assay_date‘ (where % is the wildcard string).

Using such a controlled vocabulary is vital in our case because
we use a separate installation of OpenClinica for each study and
employ the postgresql contrib module ‘dblink’ to federate their
attendant databases for querying. A helter­skeleter vocabulary would
render the latter problematic, to say the very least.

For example, we wrote a federated database procedure we call
accessions(). When executed, it consults the accession_number field
in all our installations and returns the number of accessioned
specimens broken down by trial name. This table lists some of the
federated procedures we’ve developed and their functionality.
Collectively, they provide us with a real­time, global snap­shot of
our work­flow and freezer contents.

Because OpenClinica is web­-based and has finely grained user
roles, our clients can remotely log into their respective
installations at our site with read­-only privileges and see the
status of their specimens whenever they want. Conversely, when we
think a specimen has been mislabeled or have some other issue, we can
call a client and direct them to a particular URL displaying the
pertinent information and seek clarification. OpenClinica thus
becomes a shareable laboratory notebook between us and our clients.

The bottom­line: OpenClinica works for Geneuity. Because
OpenClinica is open source, elegantly designed and well documented,
we are able to tailor the application to our needs. In the spirit of
mutual collaboration and aid fostered by OpenClinica’s lead
developers at Akaza Research, we’re making our federated procedures
freely available. They can be downloaded here.