Validation Approach for OpenClinica

Lately there has been quite a bit of discussion in the OpenClinica community about validation. The following paragraphs provide a basic overview of the key pricipals and components of a validation approach for OpenClinica.

In 21 CFR Part 11, the FDA requires validation of all systems that store or manipulate data that will be part of a regulatory submission. However, the agency provides few hard-and-fast rules on what constitutes acceptable validation. Coming up with a validation plan without outside help can be a painful and inefficient process. Akaza Research facilitates the validation of the OpenClinica electronic data capture software by providing standardized documents that make the process of validating OpenClinica more efficient for our customers. Here’s a summary of what’s involved in validation OpenClinica, or any other computer system used in clinical trials, for that matter.

Because they are unsure what the FDA requires, sponsors tend to look for validation approaches that have been used successfully by others. Common validation practices in the industry are heavily influenced by a framework called GAMP, defined by the International Society of Pharmaceutical Engineers. (GAMP originally stood for Good Automated Manufacturing Processes, but has come to be used outside the realm of manufacturing equipment.)

A typical structure for validation under GAMP is to start with User Requirements Specifications, which drive Functional Specifications, which in turn inform the Design Specifications.  The vendor’s work has to follow the specifications and the vendor’s Systems Development Lifecycle (SDLC).

Appropriate testing must follow test scripts that map back to the requirements. The individual piece of equipment (such as an OpenClinica server) is tested with Installation Qualification (IQ) by the vendor or the customer. This is essentially acceptance testing of the hardware. The next test is Operational Qualification (OQ) of the system, carried out by the vendor or customer. Finally, the customer carries out Performance Qualification (PQ). PQ has to be related to the User Requirements Specifications.

When Akaza implements its OpenClinica Enterprise solution for a customer, we carry out the IQ and OQ testing, and provide the signed test scripts together with a detailed report on the setup and configuration of the software.

It is generally not practical for the users of off-the-shelf software to produce User Requirements Specifications and the PQ scripts themselves. For this reason, it is common for customers base their specifications and PQ scripts on documents they obtain from the vendor. Akaza’s enterprise validation package includes electronic copies of these documents, as well as a traceability matrix that ties the PQ scripts back to the requirements. The customer can modify them as needed.

The FDA expects the customer to run the PQ tests. You can probably also hire a third party to do this for you. While an experienced user can run through the OpenClinica PQ scripts in two or three days, it is reasonable to expect a novice to spend a week or so on it.

The other principal element of validation is to make sure that the software development process followed a defined SDLC (Systems Development Lifecycle). Akaza’s customers do this by auditing our procedures. At a minimum, they look at our Standard Operating Procedures, and our training records. They look at records of our internal test procedures. In addition they will sometimes look at our software defect tracking systems and source code configuration management systems.

Validation of electronic record keeping systems is a labor intensive process, but is an essential element of any submission to a regulatory body. Software vendors can make the process much more efficient for their customers. At Akaza Research, that’s one of the key things we do for users of OpenClinica to ensure they are compliant with regulations such as 21 CFR Part 11.

Envisioning a better way to manage user requirements

The prevailing methods of managing software user requirements in the Life Sciences industry are far too primitive for the task at hand. This is not a small problem: meticulously documented requirements are critical for systems validation, especially for demonstrating 21 CFR Part 11 compliance. In hopes that someone will develop an open source tool to make the task easier and more effective, here’s a vision for how it would work.

Today, systems analysts write requirements one line at a time in Microsoft Word tables stretching on for many pages. This format wastes so much space on the page that the documents are needlessly difficult to read. The resulting requirements documents are so tiresome that even stakeholders tend to avoid reading them, or pay less attention than they should. Later in the lifecycle, software engineers and testers add additional redundancy and potential for error as they draft their design documents and test cases. They tie these documents back to the requirements using a requirements traceability matrix, yet another Microsoft Word table generated entirely by hand.

Here’s what I think a user requirements system should do:

  • Make it easy to enter requirements, with auto-numbering; search for duplicate entries; spell-checking; and easy linking to related requirements.
  • Hide complexity, showing only enough information to answer the question at hand. For example, “Which modules and top-level features are we planning to add in the next version of the system? How much time do we estimate they will take to implement? What are the highest priority items that didn’t make the cut?”
  • Track the requirements in logical groups (Maybe a hierarchy.)
  • Generate documents that are intelligently formatted for readability.
  • Manage the relationships between user requirements, technical specifications, detailed design documents, and test cases.
  • Generate a requirements traceability matrix as a report.

Of course, to manage the requirements in a 21 CFR Part 11 compliant fashion, the system itself might have to be validated. That implies a need to rigorously document the requirements. That in turn calls for a requirements management tool…