Data managers invest a lot of time and attention documenting lab processes, and for good reasons. Regulatory compliance demands it. Also, ensuring the validity and clinical significance of lab results is critical to assessing safety and efficacy. But while necessary, this process is often inefficient and error-prone.
In an ideal clinical study, every lab sample would, within minutes of collection, find its way to a central lab whose equipment was forever up-to-date, whose validations were always fresh, and whose inner workings were transparent to the data manager. But clinical trials aren’t conducted in an ideal world. More often than not, data managers and local lab managers share an ongoing responsibility to document equipment features and report on results collected on a variety of instruments, all calibrated differently. The challenges associated with this process are familiar. Equipment changes. Validations expire. And one lab’s “normal” may be another lab’s “low.”
The task of keeping labs up to date for many data managers is akin to keeping dozens of centrifuges spinning at the same rate, all at the same time. Collecting lab reference ranges from one lab for one analyte may be straightforward, but when the process is multiplied across dozens of analytes and sometimes hundreds of sites, your study can be exposed to significant time delays and human error. Success in this task, like most, hinges on clear expectations and guidance. Here is where good data managers shine. By providing sites with explicit instructions, a deadline, and tools to boost completeness and accuracy, data managers can make the collection of reference ranges a lot less painful and time-consuming.
Anatomy of a Lab Reference Range
Ranges are always defined by either:
a standard applied to all labs contributing data to a study (“textbook ranges”), or
the individual lab
Often, the difference between the two is minimal, so adopting the textbook range can save time and administrative burden. For measures that are critical to analysis, though, using a textbook range may not be suitable. In that case, each local lab manager (or the site coordinator representing that lab) must communicate to the study’s data manager their “in house” range for all analytes measured in the study. In both cases, a range is not complete unless it specifies
the name of analyte
the unit of measure
the lower limit of normal, by gender and age
the upper limit of normal, by gender and age
Even for one analyte, the normal range for a 25-year-old female may differ from that of a 50-year-old female, or a 25-year-old male. Consequently, specifying a range for an analyte often means specifying a number of “sub-ranges” that, taken together, associate a normal range for every possible patient. For example:
In the course of providing comprehensive ranges for dozens of analytes, it’s easy for a lab representative to overlook (or duplicate) an age or gender inadvertently. A well designed, dynamic form for capturing these requirements can help ensure exactly one range is provided for any given individual.
Anatomy of a Lab Reference Range Collection Form
Just as a value without a unit of measure is meaningless, so too is a local lab range that is not tied to a particular lab. Along with their ranges for each study analyte, labs should also provide a set of identifying information. The data manager, as part of her request to provide the ranges and lab information, should also specify the study for which the ranges are being collected. A complete lab reference range collection form includes all of these components.
Specified by the data manager
the name of the sponsor and study (avoid abbreviations or paraphrases)
which analytes are included in the study, and therefore require ranges from the lab
where the lab representative must send the completed file
a deadline for completing the file
Entered by the lab representative
the name, address, and applicable ID numbers (e.g. CLF, or core laboratory facility, number) of their lab
the name of the Principal Investigator for the site and study it serves
the effective date of the ranges to be provided
the LLN and ULN for each analyte, by gender and age
Tools You Can Use
For users of OpenClinica, we’ve designed a form template that can be used as a reference range collection form, which includes the components listed above. Try it here! Would you like to use a customized version of this form in your study? Contact your client support lead. For those not using OpenClinica, we’ve built an Excel workbook. Download it for free here.
Click either image above to test this form.
Click either image above to download the Excel version.
Regardless of how labs communicate their reference ranges, it’s essential that the communication is ongoing. Changes in equipment or clinical guidelines often occasion changes to upper and lower limits of normal. That’s why an effective date must be documented for all ranges. Good data managers encourage sites to communicate any such changes promptly. Great data managers give them the reminders, and tools, to do so.
Talks from drug development luminaries. Exhibits that combine “luxury apartment” with “miniature theme park.” And a city that offers some of the world’s best modern architecture.
Those descriptions don’t do justice to what DIA 2017 has in store, but they do fit the experience. As any veteran attendee can attest, there’s an outsized splendor to the conference. But it isn’t splendor for splendor’s sake. Half of it is celebration for the advances the industry has made in bringing life-changing therapies to market. The other half is a rallying cry to bring even more.
OpenClinica will be there, to join the celebration and the rallying cry. Attendees can find us at booth 1748, near the central break lounge. And we plan on using our patch of exhibit space to the fullest.
But the most valuable offering comes from the attendees. We get to hear directly from them on what’s working, what’s not, and what needs changing in the world of eclinical. We’re confident we’ve addressed many of those needs with our upcoming release, but as company devoted to continuous innovation, we’re never finished learning and iterating on our successes.
“Three days to enter data, five days to answer queries.”
The rule couldn’t be any clearer. You’ve told your sites at the IM and reminded them in each newsletter. You know you won’t get 100% compliance, and that’s fine. You’re reasonable.
But this is getting out of control.
As a data manager, you’ll always live with missing forms, blank fields and open queries. It’s a chronic condition that gives rise to acute episodes around interim and final locks. You’ve learned to manage it, even thrive with it, but you know there’s got to be a more effective treatment regimen.
Good news. While there’s no panacea, I’d like to offer a tool you can begin using today, regardless of your systems or processes, to spur your sites onto improved data entry, query resolution, and even enrollment. But as with any treatment, we need to consider directions, precautions and potential side effects.
First, though, some background. If you use EDC and IxRS to facilitate data collection and enrollment, you’ve probably made it a habit to pull their stable of available reports at some regular interval. (If not — if you’re relying solely on the summary statistics and visualizations available on these systems’ dashboards — consider getting acquainted with the detailed exports. This post will explain why.) These reports are almost always available in some Excel-readable format. Chances are you’ve become practiced at applying some formulas to the data inside. (If not, here’s a tutorial on getting started.) The calculations you make are vital in assessing which sites are leading the pack in subject recruitment and data management tasks, including the timely entry of data or resolution of queries. You and your fellow study leaders depend on this information to refine projections, meet lock deadlines, and offer assistance to those sites behind the curve on key operational metrics. But do you share this information with sites?
Yes! As interim locks approach, I always email out the number of total open queries and missing forms, along with encouragement to tidy these issues up. If that’s your response, you’ve already adopted a best practice. But there’s more you can do.
Provided you do so with the right context and tone, you can and in many cases should communicate to each site exactly how they compare to their peers on several key metrics, from average open query age to subjects screened per month. When you supply this information, you recognize the site’s invaluable contributions, feed their natural and justified curiosity, and tap their desire to maximize their performance.
This practice involves three major challenges. The first challenge is calculating useful, “apples-to-apples,” site metrics from the raw data found in your EDC and IxRS reports. The second is distributing this information to each site in a systematic way. The third is couching this information in a message that conveys gratitude and support. But each can be met.
Making the calculations
Here, I can offer some great news for users of OpenClinica, and a valuable tool for everyone. OpenClinica now supports a suite of configurable reporting dashboards, providing data managers and those they authorize (including sites) with clear, real-time visualizations of their study data. If you’re currently using OpenClinica, contact us and we’ll gladly share more details.
Once you’ve created a table of performance metrics by site, you have the beginnings of a “mail merge.” You simply need to add a column specifying the email address of the individual responsible for data entry for each site.
The steps for executing a mail merge differ from email client to email client. However, some starter documentation is available here:
So far, we’ve touched on the technology of quantitative performance reporting. But what about the art? It’s crucial that sites understand that your intent isn’t to chastise, but to inform and encourage. The metrics you calculate are just one piece of a broader discussion, which would include particularities that simply aren’t reflected in a spreadsheet, such as patient availability and staff experience. A site whose “screened per month” measure ranks in the bottom quartile may have had to overcome incredible hurdles to enroll their six or seven subjects. Meanwhile, they may be adding valuable thought leadership.
To establish the right tone, you might consider adopting a message template like this one:
Hello Site <<site_id>>,
The Data Safety Monitoring Committee will meet two weeks from today, so it’s important we enter all data for visits that occurred on or before March 31st by this Friday, and close all queries by next Wednesday. We can’t thank you enough for your diligence in screening qualified patients and entering data. As you well know, your efforts here support not just our study, but the patients themselves.
It’s been an incredibly busy month, and we recognize it’s not always possible to enter data within five days of events. We realize some queries take weeks to close. And we know your first priority remains and should remain your patients, whether they’re participating in this study or not. Your accomplishments are all the more impressive in light of these facts.
We believe you deserve insight into the contributions you’re making to our study. That’s why we’re initiating a weekly, custom report to share your site’s progress with you. We understand you may be curious about how your “numbers” stack up against those of those of other sites, so we’ve included some comparative measures in this report. Also, to help you navigate data management, we’ve listed out your missing forms and open queries as of the report date shown. (Please note that you may have closed one or more queries or submitted one or more forms in time between report generation and your receipt of this email. The numbers below are not real-time.)
Thank you again for all you do in service to our study and your patients!
Site <<<site_id>>> By the Numbers Report date: <<<date>>> Screened : << screened>> Failed : << failed>> Randomized : << rand>> SF Rate (Failed / Failed + Randomized) : << sfrate>> Months Activated : << mons>> Screened/Month : << srate>> Screen Rate Country Rank : << srankc>> Screen Rate Global Rank : << srankg>> Randomized/Month : << rrate>> Randomization Rate Country Rank : << rrankc>> Randomization Rate Global Rank : << rrankg>> Days Since Last Screening : << dsls>> Days Since Last Randomization : << dslr>> Open Queries : << oq>> Queries Per Subj Screened : << qrate>> Queries/Subject Country Rank : << qrankc>> Queries/Subject Global Rank : << qrankg>> Average Age of Open Queries : << avgqage>> Age of Oldest Query (Days) : << oldestq>> Query List : << qlist>> Missing Pages : << mpgs>> Missing Pages Per Subject Screened : << mrate>> Missing Pgs Per Subject Country Rank : << mrankc>> Missing Pgs Per Subject Global Rank : << mrankg>> Average Age of Missing Pages (Days) : << avgmpgage>> Age of Oldest Missing Page : << oldestmpg>> Missing Page List : << mpgslist>>
Some final precautions
How often you provide a report like the one above, and what you include in it, are at your discretion. Fast-moving infectious disease trials may warrant a weekly report. Large, endpoint-drive cardiac studies may benefit from just one report per month. Also, carefully consider the cultural differences that exist among sites in various countries. There may be no acceptable way of communicating comparative metrics in some.
There’s power in your metadata. You should consult it frequently on your own, weekly if not daily. You can use the workbook above to do that and nothing more. But we have an obligation to patients worldwide to conduct trials in the most efficient manner compatible with the highest data quality. Bringing some gentle pressure to bear on sites is one method of achieving that. If you adopt some version of the practice described in this post, please let us know your experience with a comment or email.
Big data has been a recurring topic in medical research news for years now. It’s a topic that deserves our attention. Big data’s potential to revolutionize fields like genomics and to advance precision medicine generally is stunning. Today, though, a lot of the press is speculation. Robustly effective designer drugs for cancer, based on the patient’s genetic markers, remain an ideal that is likely decades away.
But if we adopt a broader conception of big data–one that includes the massive infrastructure supporting social media, the Internet of Things, and (potentially) interoperable health record platforms–real world applications are not hard to find.
One facet of these examples stands out. For all their diversity, projects that rely on big data rely just as much on collaboration. Moving from genomes to biomarkers to disease risk models and personalized treatment requires more than one big dataset: it requires the integration of data from multiple systems that are secure, geographically separated, and disparately schematized.
Ten years ago, the ability to handle this task might have been seen as a leading-edge, if not commonly leveraged, feature of clinical technology. Today, software that cannot facilitate integration is doomed to obsolescence.
eCitizen of the Data World
What does this requirement mean for EDC? Simply put, those of us building data capture solutions need to look far beyond the “coordinator keying in vitals” use case. (Our solution for that use case had better already be rapid, reliable and easier to execute than ever, considering the burdens placed on trial sites in 2017.) With “insight by integration” at the forefront of research strategies, we technologists had better think of our system as a world traveler: one familiar with the laws in multiple countries, authorized to enter and leave those countries, and fully knowledgeable of their languages and customs. In the world of data management, this means the ability to pass authentication to enter a source database, map the data to a target, and leave the source while maintaining data provenance.
As a long-standing promoter of open, standards-based interoperability, OpenClinica represents this “world traveler.” The native language of OpenClinica’s EDC is the Clinical Data Interchange Standards Consortium Operational Data Model (CDISC ODM). This fact alone makes the OpenClinica data model an ideal cosmopolitan, instantly conversant with research peers around the globe. But holding fast to one standard is not sufficient. We need to be willing to learn new languages. By offering a well-documented web services API, OpenClinica makes it easy for its users to leverage RESTful web services, together with OAuth protocol version 2.0, to systematically:
extract data from almost any third-party source (e.g. labs and imaging centers),
associate each element of that data to the relevant Case Report Form (CRF) field.
APIs and authentication protocols offer the most direct route to turnkey integration. But it’s not enough to be powerful in the pursuit of data integration. A system has to be flexible, too, when tapping data sources that aren’t available to an API. For OpenClinica, this means providing a host of configurable tools to data managers and data entry personnel.
OpenClinica’s Jobs feature allows for custom imports from local files. A Job may be scheduled to run at any frequency, so that users responsible for data entry based on a regularly updated flat file (e.g. a CSV on their hard drive) may provide that data without keying in each element. A Job well-defined and set up just once improves accuracy and saves hours of research time.
An Import Data feature makes ad hoc batch uploads easy, as well. Users simply generate a XML file based on OpenClinica-supplied Object Identifiers (OIDs) to map data from the import file to the EDC.
OpenClinica supports a variety of Single Sign On (SSO) protocols, reducing repetitive authorization while maintaining security. OpenClinica is also an early and already experienced adopter of SMART on FHIR, a set of open specifications to integrate its core EDC with Electronic Medical Records (ER) and other health IT systems.
A Look at Our Passport
So far, I’ve outlined a set of capabilities required of any EDC in 2017, and claimed that OpenClinica meets them all. But where’s the evidence? In the second half of this post, I’m going put three of our partners in the spotlight. For each, OpenClinica was able to play a pivotal role in bringing together multiple data sources.
“links to other data storage and analysis tools within the TraIT platform, allowing researchers to integrate and analyse case report data, imaging data, experimental data and bio banking information,” and…
the “possibility to integrate with Trusted Third Party which handles proper (de-)identification of participant data within OpenClinica and other tools/services used in TraIT.”
The 100,000 Genomes Project, led and funded by Genomics England, is another example of a large-scale effort to combine clinical and genomic data. The 100,000 Genomes Project is sequencing 100,000 genomes in order to:
better diagnose rare disease,
understand its causes, and
set a direction for research on effective treatment
Whole genome sequencing (WGS) offers the best hope for determining which genetic mutations give rise to particular phenotypes, including disease states. WGS yields the syntactical equivalent of the three billion nucleotide base pairs that make up just one strand of one individual’s DNA, so a research program involving even one such sequencing has already entered the territory of “big data.” While highly specialized systems are responsible for sequencing itself, and yet others for the analysis of the output, an equally essential tool for this research is a system that can manage the clinical data and biospecimen tracking of subjects visiting one of several geographically dispersed clinical centers. Here, too, OpenClinica serves as the hub. Researchers at 13 NHS Genomic Medical Centers are using OpenClinica to register participants, capture clinical information, and ensure that blood samples stay matched with their de-identified contributors.
Project leaders have made public a 10-page guide to researchers on this process, one whose brevity and clarity speaks to how easy OpenClinica makes it. Due the dedication of the researchers, collaboration of participants and the fitness of the technology, the project is on track for completion in 2017.
PECARN, the Pediatric Emergency Care Applied Research Network, is the first federally-funded pediatric emergency medicine research network in the United States. To date, PECARN has conducted 24 studies that have already changed how clinicians are preventing and managing acute illness and injury in children.
As part of their mission to advance clinical practice, PECARN has taken a lead role in the implementation and study of clinical decision support tools. For all the potential benefit offered by these tools, questions remain about their adoption and effectiveness. Do physicians and nurses generally follow evidence-based recommendations for treatment or diagnostic procedures? When they do, are outcomes improved?
To help answer these questions, PECARN study leaders conducted a nonrandomized trial with concurrent controls at thirteen emergency departments between November 2011 and June 2014. These thirteen departments were consolidated into ten research sites. At eight of these sites, clinicians creating an EHR record for any patient <18 years old with minor blunt head trauma were automatically presented with a custom template. This template solicited additional data about the injury before providing recommendations on CT use and risk estimates of clinically important traumatic brain injuries. (CT imaging of the brain is associated with a non-negligible risk of tumor formation in those who undergo the procedure, especially children. At the same time, early detection of ciTBI–i.e. injuries leading to death or requiring neurosurgery, intubation for more than 24 hours, or hospital admission for two or more nights–is critical for effective intervention. The recommendations provided by the EHR template were intended to limit CT use to those patients who met established predictive criteria for significant ciTBI risk.)
The clinicians work in their EHR, together with subsequent cranial imaging and TBI-related outcomes, all generated data that would require aggregation to determine (1) how frequently care providers heeded recommendations surrounding CT use, and (2) whether the predictive rules for ciTBI risk were valid. That aggregation fell to OpenClinica. By accepting reports generated by each site’s EHR to automatically create study subjects, and by integrating with the source of imaging data at each site, OpenClinica enabled a true e-source study that left clinical workflows unaffected. Not one of the 28,669 subjects created in the study database required manual data entry.
The moral? Big data isn’t just found: it’s made, through the coordinated efforts of both people and systems that travel light and fast. You’re contributing to big data during more and more of your waking hours these days. If you want to help shape it through technology, get ready to cooperate… and pack your digital bags.
With patient-centricity claiming more and more of the spotlight in both research and care (rightfully so), patient-reported outcomes will only play a large roles in clinical trials. But there are significant obstacles to getting quality data from PRO measures. Patients, especially those who are very sick, don’t want to hand-write dry medical diary entries. They don’t want to learn yet another electronic device, download and manage an app, or have to recall yet one more password. And who can blame them? Trial participants are the heroes of the research story, and when it comes to the collaborative process of data gathering, they deserve a hero’s welcome.
That’s why we developed OpenClinica Participate. We’re gratified by the success our clients have found leveraging this innovative ePRO solution, but we’re not surprised. When you prioritize a trial subject’s convenience and obsess over making things simple, you simply get better results. Here’s an example. Let’s call it:
Out with the old, in with the new
OpenClinica teamed with Danish CRO, Signifikans, to implement OpenClinica Participate for a leading Denmark-based bioscience technology company developing an innovative treatment to alleviate colorectal disorders, a common side effect of numerous medicines affecting millions of people at any time. The study’s objective was to investigate exercise induced intestinal permeability, immune markers, and bowel habits in 18-40 year old healthy volunteers. Participants were given two strains of bifidobacterium, an anaerobic bacteria that resides in the intestinal tract. The study involved 48 participants throughout Sweden, and each patient was required to provide 65 daily diaries in addition to 5 in-person visits over the course of the study.
The Old Way
In a similar prior study, the sponsor collected paper diaries from 700 participants. Each participant provided their (hopefully) completed and accurate paper diary to their site coordinator during the in-person study visits. The site coordinators then delivered the completed paper diaries to a data coordinating center. Coordinating center staff then scanned the diaries into a document management system and an overseas data entry vendor used a double data entry workflow to populate a database. Completed diaries were scanned and uploaded in batches for data entry. Phew!
On average, four months elapsed between the point of data capture and the first day of availability of that data to the sponsor. Monitoring participant compliance was also a challenge in this study, as it was impossible to discern when each patient actually completed their daily diary. The expenditures associated with this process for data entry tallied over $213 per patient diary, or $4.97 per diary page.
Overall, this process was cumbersome, expensive, and logistically complex. The sponsor was planning a similar new study, and this time around was determined to find a way to
get faster access to the study data,
improve data reliability, and
The New Way
The sponsor enlisted local specialty CRO, Signifikans, to help it identify and implement a better approach. Signifikans recognized that with OpenClinica Participate, the sponsor could have immediate access to patient data, and that this data would automatically sit right alongside data captured from other sources during the study. No EDC integration was necessary.
Signifikans also took the lead on configuring the study in OpenClinica. (Our “make it simple” credo guides how we design tools for data managers, as well. That’s why we have invested so much in our forms engine, a topic for another post.) While building the study, Signifikans was able to easily demo prototypes to the sponsor along the way, iterating rapidly through edits and changes. Data capture forms were developed in the Swedish language, and the study was configured to send email reminders to patients to help ensure diaries were completed on time. The reminders contained a secure, uniquely-identified link the participant could click to go right to their diary, eliminating the need for participants to remember usernames and passwords.
As soon as the study went live, the sponsor was able to monitor precisely when data were captured; something that was not possible with the old paper-based method. They observed, for example, that all five participants enrolled in the study’s first week each completed their diary card daily, per the protocol. The sponsor’s confidence in patient compliance and data quality surged; so much so, that they implemented an increase in the amount of data being collected this way. Scaling that quickly would have been impossible with paper diaries and slow transcription processes.
“Participate was very low friction: set-up was quick and efficient, and patients really seemed to embrace the technology.”
– Andreas Habicht, CEO, Signifikans Aps
The OpenClinica solution delivered a unified study database out-of-the-box, with patient-reported data sitting alongside clinician-reported data and accessible via the same interface. Having everything in one audit log made it easy to follow the patient’s trajectory through the study. Signifikans was able to use the same tools to configure and manage both ePRO and non-ePRO aspects of the study, resulting in a faster time-to-launch, and facilitating mid-study changes.
In addition to enhanced data quality and faster access to data, the cost of data capture per diary with OpenClinica Participate resulted in cost savings of over 80%.
Keep an eye out for more ePRO success stories on this blog. Our next post will delve into a different topic, but, as with this one, you can be sure it will feature better results through a better eClinical experience.
What does the term “source” bring to mind for you? Paper files? A clinic’s EMR? Fair enough. Those are the typical formats of source data in clinical trials. But when you think about it, those records are a few removes from the true source: the patient.
Granted, most trials depend on a host of instruments and analyzers. No patient can self-report their own hemoglobin levels. But there are measures, such as quality of life, whose source really is the patient’s own experience. And for an industry whose raison d’être is enhancing (and sometimes saving) lives, we have a hard time obtaining those measures.
That’s not to say that patient-reported outcomes are a rarity in clinical trials. In fact they’re common. But so are obstacles to collecting them. Even today, with several ePRO solutions on offer, too many trials rely on patients to complete paper diaries. But paper records are prone loss or damage. Also, it’s virtually impossible to tell whether a patient made daily diary entries as instructed, or retrospectively wrote responses just prior to a study visit, raising data quality concerns. (Here’s a great analysis of “parking lot syndrome”.)
ePRO is a big improvement on paper, but it doesn’t integrate seamlessly into a patient’s day to day life. Nearly all trials today using an electronic system for PROs provision dedicated devices to patients for this purpose. Patients need to be trained on how to use this device, recall that training at the relevant time, and make a habit of keeping the device on their person to stay current with reporting tasks. These systems also require patients to remember passwords and other access credentials. A step as apparently simple as downloading an app can be fraught with challenges: the patient must remember their app store login, locate the download area, have sufficient memory on their phone to house and run the app, ensure the app installs successfully, and ensure it remains installed, running, and updated.
All of the above examples are potential points of failure that can compromise the quality of real world data and increase trial costs. These myriad tasks and responsibilities unrelated to the data itself can place a significant burden on a volunteer patient, particularly one who is very sick.
providing an system that can be used on the patient’s own device(s),
eliminating the need to patient’s to remember login credentials, and
removing the complexities associated with apps.
This helps place the patient at the center of the research. And perhaps best of all, OpenClinica Participate places the patient reported data into the master EDC database in real-time with zero additional effort.
The advantages of this approach aren’t just theoretical. OpenClinica and its partners have realized their benefits in more than 20 studies since the launch of OpenClinica Participate about a year and a half ago. In this post and future ones in this series, I’d like to share some short and informal case studies. I call this one:
On time, nearly every time
S-cubed, an innovative pharmaceutical consultancy and contract service company, based in the UK and Denmark, recently supported a study assessing a medication to protect against a virus associated with the common cold. In the study, subjects were asked to self-report symptoms at multiple time points per day over several days. On most days, self-reports were to be submitted in the morning, in the afternoon, and at night. To help drive compliance with this rigorous self-reporting schedule, study managers wished to engage their subjects and ensure the data capture method was convenient. To do so, they tapped into the ability of OpenClinica Participate to present clear, attractive eCRFs to subjects on their own mobile devices (smartphone, tablet, or laptop). S-cubed data managers established rules that fixed the date of all future study events based on the protocol events schedule and each subject’s date of enrollment. OpenClinica Participate automatically sent text messages and emails to each subject prior to a scheduled event (and at the end of the time window if the diary was not completed), directing them to the relevant data capture form.
The data collection process involved no offline component. Subjects always provided their reports on a connected device, without regard to the specificity of device or location, etc. Consequently, fine-grained metadata regarding the date and time of eCRF completion was available. This metadata provided compelling evidence that the experience afforded to patients by OpenClinica Participate drives widespread data entry and timelines.
Of the 92 subjects who completed at least one of the ePRO forms associated with the study, thus signaling intent to participate, 89 (97%) completed all forms required in the morning, afternoon and evening over multiple consecutive days, when timely reporting was most critical, Reasons for non-completion were sometimes outside of OpenClinica Participate’s control.
Of the all “3x daily” forms completed, the vast majority were completed in real-time, with approximately 93% being completed within 1- 2 hours of the scheduled timepoints.
In the previous post, I described the difference between efficacy and effectiveness, an increasingly important concept in clinical research and healthcare. After stressing the importance of effectiveness research to health policy planning and patient decision-making, I summarized seven criteria for identifying effectiveness studies. Finally, I asked whether these criteria could be re-purposed beyond a medical intervention to inform how we measure the effectiveness of software systems used to conduct clinical trials.
Is it possible to assess clinical trial software through the lens of effectiveness, as opposed to just efficacy?
I believe that it’s not only possible, but crucial. Why? We all want to reduce the time and cost it takes to deliver safe, effective drugs to those that need them. But if we don’t scrutinize our tools for doing so, we risk letting the status quo impede our progress. When lives are on the line, we can’t afford to let any inefficiency stand.
In this post, I adapt the criteria for effectiveness studies in clinical research into a methodology for evaluating the effectiveness of clinical research software. I limit the scope of adaptation to electronic data capture (EDC) systems, but I suspect that a similar methodology could be developed for CTMS, IVR, eTMF and other complementary technologies. If I open a field of inquiry, or even just broaden one that exists, I’ll consider it time well spent.
For pure pathogen-killing power, it’s hard to beat a surgeon’s hand scrub. Ask any clinician, and she’ll tell you how thoroughly chlorhexidine disinfects skin. If she’s a microbiologist, she’ll even explain to you the biocide’s mechanism of action–provided you’re still listening. But how would the practice fare, say, as a method of cold and flu prevention on a college campus? Your skepticism here would seem justified. After all, it’s hard to sterilize a cough in the dining hall.
Efficacy and effectiveness. It’s unfortunate their phonetics are so close, because while the terms do refer to relative locations along a continuum, they’re the furthest thing from synonyms, as the ever accumulating literature on the topic will attest.
In this post and the one that follows, I’d like to offer some clarity on efficacy vs. effectiveness and illustrate the value that each type of analysis offers. If nothing else, what emerges should provide an introduction to the concepts for those new to clinical research. But I have a more speculative aim, too. I’d like propose standards for assessing trial technology through each of these lenses. Why? Because while we’ve been asking whether a particular technology does what it’s explicitly designed to do, as we should and must, we may have forgotten to ask a critical follow-up question: Does it improve the pace and reliability of our research?