Calculating ROI for ePRO

I recently delivered a webinar titled “Getting Started with eCOA/ePRO,” in which roughly a third of attendees polled cited expense as the number one reason that has prevented them from adopting an ePRO solution. So what does ePRO really cost? Is it worth it? Here I strive to provide a basic, high-level framework for thinking about the return on investment ROI of eCOA vs. paper.

Let’s start by taking a look at the costs that are unique to each approach.

Paper

In a traditional paper based model, you are incurring costs that stem from printing, mailing, data entry, and data cleaning. These are all expenses than can be estimated with a fair degree of accuracy, with the cost of data entry being the most significant of these. To estimate the cost of data entry, see how long it takes to key in a subject completed paper casebook, multiply this by your cost of labor (don’t forget to include overhead!).

ePRO

The cost side for ePRO is similarly straightforward, but the expense elements are different. You’re either building an ePRO system (which will almost carry a highly unpredictable cost) of buying one (much more predictable cost). Assuming you’re buying, here are the costs you may expect to incur:

· License
· Hosting
· Training and support
· Professional services (e.g. study configuration)
· Devices

You should evaluate whether your study and selected ePRO system will allow for patients to use their own devices, or if you will need to provision devices (or some mix thereof). The cost of provisioning devices, especially for a global study can be significant—in addition to the costs of the devices themselves, you will need to consider the costs of data plans, and logistics associated with supporting the devices. I’m a big fan of BYOD (bring your own device) but, depending on the study, it may not be feasible to utilize while maintaining scientific validity of data collected.

Once you’ve mapped out your costs of each route, you can begin to weigh these against the benefits of going eCOA.

cropped-istock_000037068804.jpg

Paper vs. eCOA

When you boil it down, people employ ePRO/eCOA to maximize data quality, increase productivity, and/or enable new capabilities that help answer their research questions. ePRO is e-source, so you don’t have worry about administering a paper data entry process. Depending on the study, the cost savings from this alone might justify ePRO.

There are some additional benefits ePRO offers over paper that may be harder to quantify, but nonetheless  very real. For example, there are clear data quality benefits to ePRO. The electronic system can ensure a minimum standard of data quality through edit checks and enforced data structures. ePRO data will always be cleaner than the same data captured on paper.

Benefits and Motivations for eCOA

 

The use of an ePRO system also allows you to know for sure when the data were recorded. For instance, patients can be reminded automatically when their diaries are overdue, and you now only have much stronger assurances that data were collected at the appropriate time (vs paper), you can also more easily monitor the study progress.

Bypassing manual data entry and having the system provide notifications to subjects to ensure data are captured in a timely way might allow for faster and better in-study decision making and even may accelerate study closeout. Also, an increasing amount of evidence exists that mobile-based messaging and communication strategies help increase patient engagement and treatment adherence. And of course, not having to deal with a stack of paper during a site visit might allow the clinician’s interaction with the patient to be higher quality.

Quantifying the benefits of all of these things can be tough, but start with those which are most quantifiable and see if those items alone these alone provide a compelling ROI (from my experience they often do).  Then the less tangible benefits become gravy to the ROI argument.  When modeling costs over time and a pay-back period, keep in mind that ePRO will typically carry a higher upfront cost than paper, with the cost saving benefits realized downstream over time. With today’s technologies, even most smaller studies should be able to realize a positive payback.

Naturally, there may be additional ROI factors to consider which are specific to your situation. If you have particular thoughts, questions, or experiences on this topic I encourage you to add a comment to this post.

Engage. Learn. Repeat.

At OpenClinica we are driven to reduce obstacles to the advancement of medical research. The OpenClinica open source project started because EDC was too complex, too inaccessible, and too expensive. Not to mention far too difficult to evaluate and improve. So we built an EDC / CDMS platform and released it under an open source license. It is now the world’s most widely used open source EDC system and has an active, growing user community.

 

As the user base grew, we listened to users and understood that integration and interoperability were another major obstacle. While we don’t claim to have fully cracked that nut yet, OpenClinica’s CDISC ODM-based APIs have been pretty widely adopted and helped to drive some significant innovations. These APIs have been improved upon by a large number of developers in the few years they have been part of the codebase.

 

As we continue to improve the clinical and researcher experience, our attention has more recently been directed to the experience of trial participants. The difficulty of meaningful, timely engagement with these volunteers also strikes us as an obstacle to successful research. We live in a world where 90% of American adults have mobile phones, 81% text, and 63% use their phone to go online (Pew), and even older age groups are adopting smartphones at a rapid pace [1]. Because of this, we think that mobile technology could be a pretty effective means to help more meaningfully engage participants research.

 

Why is this important? Treating research volunteers as participants, as opposed to subjects, can lead to concrete benefits – improving participation, motivation, and adherence. Increasing your ability to meet recruitment goals, budget, and completion timelines. Getting more complete, timely data. Even enabling new protocol designs that better target populations and/or more closely align with real-world use. But most of all, it just seems like the right thing to do. As one HIV trial participant put it, “I’d initially had this nagging fear in my head, that, once recruited, I would cease to be nothing more than a patient number – a series of digits, test results and charts in a file – which is quite a daunting prospect when you’re not entirely sure how your body is going to respond to the vaccine. This could not have been further from the reality of the trial. I felt safe, informed and valued at every stage of the trial.”

 

The great (and often unrecognized) news is that so many of the people involved in research and care already do an unbelievable job creating this type of engagement – making participants feel safe, informed, and valued. But it takes a lot of work. With a mobile-enabled, real-time solution like OpenClinica Participate, you can provide an engagement channel and data capture experience that is simple, elegant, and easy to use on any device. Because it is fully integrated with OpenClinica and captures data in a regulatory-compliant manner, you can reduce time and headache for your research team from, for instance, merging disparate sources of data and keying in paper reports. Leaving you more time to focus on the kinds of human to human engagement that technology cannot do.
Participate Webinar
[1]  For the over 55 age group, most likely to participate in many types of trials, the picture is a bit different. As of 2013, around 80% have mobile phone but only 37% are smartphones. However over-55s are the fastest growing smartphone adopters, expected by Deloitte to soon reach 50% and reach parity with other age groups by 2020.
See http://www2.deloitte.com/content/dam/Deloitte/global/Documents/Technology-Media-Telecommunications/gx-tmt-2014prediction-smartphone.pdf. Outside of the developed world, the picture is different, though the opposite of what you might expect. According to Donna Malvey, PhD, co-author of mHealth: Transforming Healthcare, cell phones are even more pervasive, and mHealth “apps are the difference between life and death. If you’re in Africa and you have a sick baby, mHealth apps enable you to get healthcare you would normally not have access to… In China and India, in particular, mobile apps can bring healthcare to rural areas. “

 

Introducing OpenClinica Participate

In clinical research, we all work towards better evidence-based, patient-centered health interventions. We all understand the importance of evidence. But how about patient-centered? We hear this phrase perhaps too often nowadays, but it’s more than just a buzzword. A patient-centered approach directs research toward questions that are important to patients so they can make more informed healthcare decisions. It measures the outcomes that are noticeable and important to patients, and produces results that help them weigh the value of healthcare.

At OpenClinica, we think increasing the patient-centeredness of research is vital. Industry, NIH, FDA, and the general public seem to agree, and furthermore share our view that technology can increase patient engagement. This can happen by designing highly accessible, mobile technology to:

  • Improve patient participation, motivation, and adherence
  • Increase ability to meet recruitment goals, budget, and completion timelines
  • Enable new designs that better target populations and/or more closely align with real-world use

HTCPhoneI’d like to introduce our upcoming product, OpenClinica Participate, a tool tightly integrated with OpenClinica for engaging patients and collecting data directly from study participants.

If you took our recent survey, you got a glimpse of what your forms for patient-reported outcomes could look like in OpenClinica Participate. Driven by a powerful forms engine based on proven open-source technology, the participant forms are simple, dynamic, mobile-focused, and platform-independent.

But it’s about a lot more than mobile-friendly forms. Patient expectations are rising while trial participation is shrinking. Clinical trials need to engage ‘Subjects’ as ‘Participants’ — by recruiting and retaining through real-time engagement and meeting the ‘anytime, anywhere’ expectations of a mobile, smartphone enabled world. It starts with an intuitive, action-oriented dashboard. Participants are greeted with a simple interface that motivates and focuses them on what they need to do today. The layout is responsive and knows whether you are using your phone, a tablet, or a traditional browser. Communication with the participant can occur through multiple channels — via the dashboard, through SMS, or via email, with options to help you make sure the right balance is struck between security and accessibility.

Participate_OCUILast, and perhaps most important, Participate is tightly integrated with the OpenClinica EDC platform. The Participate solution allows you to design your study and forms using the OpenClinica study build tools you already know, while seamlessly capturing all your clinical and participant-reported outcomes in a single database. You can build skip patterns, repeats, and other logic into your participant forms just as you do with traditional OpenClinica CRFs, using the same rules engine. You can use scheduling rules to identify the next time participant feedback is required. As data is submitted by study participants, their activities become part of the same audit trail that tracks what your clinical users do. The data they submit can immediately be reviewed and extracted along with CRF data from other sources.

Participate is extremely easy to adopt. As a modular add-on to your hosted OpenClinica Enterprise instance, it can be activated for any new study from within your study build screen. Forms are powered by the widely used open source enketo-express library and all editions of OpenClinica will support the widely-used OpenRosa API to let you run Enketo, ODK Collect, or any of a number of OpenRosa-compliant data capture clients. Participate utilizes a cloud-based Software-as-a-Service (SaaS) delivery model, so there is no costly and delay-inducing software deployment to worry about.

 

Visit the OpenClinica Participate Website