How’s your health? Introducing a mobile-friendly SF-12

We’ve blogged about ePRO form design and the power of real-time calculation previously. Today we invite you to experience both on one of the most thoroughly validated self-reported health instruments of the last 30 years: the SF-12.

Take it for a spin one your smartphone, tablet, or laptop. We’ve paginated this form to minimize the chance of missing any item and all but eliminate scrolling. Scoring algorithms built into the form deliver immediate results. Yet the simplicity and familiarity of the paper form remain.

Instruments like these will only grow in importance, as regulatory bodies and payers continue to call for more real-world evidence. These same stakeholders are also embracing digital: unlike paper, electronic forms capture the date and time of entry (helping to avert “parking lot syndrome“), and can even prompt a participant to revisit skipped items. The result is a dramatic increase in data quality and response rates, along with a concomitant reduction in delays and transcription costs.

Why wait for data, only to discover how compromised it is? Start building your smart, responsive ePRO forms now!

 

Complete this form on your smartphone! Just go to bit.ly/ocsf-12 on your device’s default browser.

What’s the score? Real-time scoring of the PHQ-9

Data capture screen illustrating scoring on a tablet

Heart disease. Lung cancer. Type II diabetes.

You. Me. The barista at the coffee shop.

For all the differences between the diseases above, each presents, if it does present, with a certain severity.  For all the varied experiences of the people above, each bears some risk of developing the diseases. How do we evaluate that severity? How do measure the risk? The answer is with a score.

What is a score?

A score is a value on an ordinal scale, used to classify the severity of a condition or to predict its future course. (Only a rigorous validation study can establish if and how well the score predicts.)  Instruments for generating scores take more basic measures like weight, blood pressure, or the presence or absence of some biomarker as their inputs, then combine these inputs in mathematically explicit ways. Crucially, a given score is calculated the same way from setting to setting and study to study, thus endowing the score with universal meaning.

Why are scores useful?

Scores characterize, classify, and predict. In cases of trauma or disease, a score (or a stage, or a grade) is what makes prognoses and treatment decisions possible. With scores so essential to clinical practice, it’s hardly surprising we encounter them so frequently in research. Eligibility criteria may set bounds to acceptable scores, to ensure safety or to tailor the investigation to a particular patient profile. A change in score over time may represent a primary outcome, suggesting a therapy’s superiority or inferiority to some comparator in reducing a disease burden or improving quality of life.

Scores are not comprehensive descriptions of a patient’s disease, much less of the patient herself. They are never perfectly predictive.  They are quantitative heuristics whose success in classifying the stage or severity of a disease, or in predicting the risk of its development, has been established through statistical studies.

How do researchers calculate scores?

Most (not all) scores are matters of simple arithmetic. Measure A, B, and C, then add them together. If D is present, add one. If D is absent, subtract one. A researcher mentally calculating a score for one patient in a calm, quiet setting stands a good chance of doing so correctly. But as inputs grow larger (34 x 72, say, instead of 4 + 9), so does the chance of a miscalculation. Asked to perform mental calculations again and again, for dozens of study participants, the researcher is all but guaranteed to make at least one mistake.

How does EDC make working with scores easier?

When it comes to computation, it’s hard to beat, well, a computer. Collecting data electronically facilitates rapid, accurate operations on that data. When data capture is web-based, the calculations may be shuttled between the clinical site and data management office almost instantly. That rapid exchange optimizes every stage of trial conduct. Real-time scoring at screening visits can stratify participants into cohorts. Scores that signal an adverse event can immediately trigger workflows for stabilizing the participant and submitting safety reports. When the time comes to analyze results, a portion of the statistical labor is already done.

Can you give me an example?

I was hoping you’d ask. Below you’ll find two presentations of the Patient Health Questionnaire-9 (PHQ-9), an instrument for screening, diagnosing, monitoring and measuring the severity of depression. The first presentation assumes out-of-clinic, ePRO use, where study managers expect most participants to respond on their own smartphone. The form will render on any web-enabled device, but the pagination is set to display one item at a time, for ease-of-use on smaller screens. The second presentation assumes in-clinic use on a tablet. Depending on the protocol, the researcher may administer the PHQ-9 through an interview with the participant, or the participant may complete the questionnaire on her own.  In both cases, best practices in ePRO form design dictated layout and behavior.

OpenClinica forms support real-time scoring through syntax any data manager can learn quickly. Data managers have complete control over when and if the score is displayed on the form, rather than simply saved to the database. Scores may be built into forms for patient-reported outcomes (ePRO) as easily as they are for clinic visit forms. And conditional logic can trigger additional questions or other workflows based on specific scores.

So give your study’s major eClinical system a chance to do what it does best, and score one for speed and accuracy.

 

Click here to view a tablet-based presentation of the PHQ-9.
Click the image above to view a tablet-based presentation of the PHQ-9.

14 Best Practices for ePRO Form Design

Is there any term in data collection more despairing than “abandonment rate”? That’s the percentage of respondents who start inputting data into a form but don’t complete the task. Sometimes, it’s hard not to take that kind of rejection personally.

Sadly, it’s just one problem among dozens that hinder the collection of timely, clean and complete data directly from study subjects. A participant’s compliance with study procedures (and data entry) is always only voluntary, and apart from the occasional stipend, these participants rarely receive compensation in dollars. That’s not to say that the care provided in the course of a study isn’t of value to patients. But that “quid pro quo” mindset isn’t easy to maintain outside the clinical site. Paper diaries and even electronic forms ask a participant for their time and attention, goods that are in short supply these days. As an industry, we can’t stop looking for ways to minimize the burden on participants. Not if we want to maximize their contribution.

In previous posts, we explored BYOD as a preferred approach to collecting ePRO data. But that’s only half the story. What good are friendly, motivational messages to a participant’s mobile device and computer, if the form to which they’re directed is needlessly long or convoluted? That’s a recipe for heartbreak (form abandonment), or at least a host of trust issues (incomplete or poor quality data).

So, what are the keys to getting your participants to enter all their PRO data, accurately and on time? Not surprisingly, they’re not too different from those we rely on in any budding relationship. Below, I bundle them into four categories.

Make a good first impression

Imagine yourself as a job interviewee. The hiring manager asks you to rattle off the top three responsibilities in every role you’ve ever filled. How coherent will you answer be? A practiced interviewer, interested in getting to know you, would start differently. “Tell me, what’s your top priority in your current role?” She’ll need to gather a lot more information before the end of the interview, but she’s set out a comfortable pace for getting it.

The lesson for ePRO form design is clear.

1. Start with a single (or very small number) of questions that can be answered simply. Present these one to three questions, and no more, on the first page.

This best practice is one example of a broader rule: always proceed from the simple to the complex. Don’t ask the participant to choose one of sixteen options. Rather, ask two yes or or no questions that will reduce the options first to eight and then four, and have the user pick from that list. Yes, the latter scenario involves more clicks, but it involves less cognitive labor and less scrolling.

Dress to impress

The “e” in “ePRO” isn’t usually capitalized, but maybe it ought to be. Leave the paper mindset behind. Spatial and color constraints no longer apply, and you can present information in a way that prevents the participant from looking ahead (and becoming discouraged). In short, you’re free to give respondents a form-reading and form-filling experience they actually enjoy. Here are some pointers on visual cues that work with the eyes and minds of your participants:

2. Black or dark grey text on a white background is always the safest default for instructions and form labels. For section headings and buttons, a vibrant color that contrasts sufficiently with the background can help orient the respondent.

3. If you have a choice of fonts, sans serif is preferable. Why? While studies do not point to a clear winner for readability in all contexts, evidence suggests that lengthy text passages that ask the reader to parse several ideas at once are best served by serif fonts, while short prompts and labels are best conveyed with a sans serif font. (And you already know that short and direct is best, right?) The wide variety of serifs in existence make character-recognition more difficult to non-native readers, while san serifs are more likely to remain readable when scaled down.

4. Place the field label above, and left justified with, any field requiring the respondent to type. This “stacked” format suits the portrait orientation with which most smartphone users hold their device. Placing the field label inside the field for the user to type over may save space, but it also causes your label to disappear once the user interacts with the field.

5. Avoid grids. There are contexts where a grid layout is preferable; for example, when recording multiple instances of body position, systolic pressure and diastolic pressure for a patient undergoing blood pressure tests. But these are almost always in-clinic scenarios. For the collection of ePRO data, stay within a single column layout.

6. Paginate. One screen containing six questions, or two screens containing three each? All things being equal, fewer questions on more pages is preferable. Why? Clicking next is easier than scrolling. Also, breaking up a large set of questions into two smaller ones reduces the intimidation factor.

7. Use images as selection options when text could lead to misinterpretation. Not all form engines support this feature, but the ability to present an image in place of (or with) a checkbox or radio button is more than a “nice to have.” Which of the following is more likely to result in quality, computable data?

 

or…

 

 

I’ve relied on a few best practices in this example, starting with some very simple questions in order to narrow down the remaining ones and present them in a manner that gives us clean, accurate, and computable data. But the images in particular are what rescue the participant from needless typing and potential confusion. By making the regions within the head illustration clickable, I can capture very discrete data without taxing the participant’s time.

Respect their time

“Press C to confirm or X to cancel.” That’s a familiar formula to anyone who’s received a text message before an appointment. These are easy to appreciate. If I feel I owe any response to the sender, I’m more likely to complete this pseudo-form than any other.

Chances are, however, you may need a little more data than this from your participants. Here’s how you can gather it while respecting your participant’s time.

8. Minimize the number of fields. This advice may seem obvious and simple, but it’s neither. So long as a participant’s date of birth has been documented once, having them input age is never necessary. And if your system is capturing metadata precisely (e.g. a date and time stamp for every change to a field’s content), then you don’t need to ask the participant to record this information. In general, before adding a field, it is helpful to ask:

  • Do I really need this information? If yes, then…
  • Can I reliably deduce or calculate it based on prior data? If no, then…
  • Do I need the participant to supply it? If (and only if) yes, then include the field.

9. Use skip logic. The phrase “if applicable” should never appear in a form, especially forms designed for participant use. If you are asking the question, it had better be applicable. You can ensure that it is by using skip logic. Has the participant eaten that day? Only inquire about nutritional content for that day’s meals if she responds with “yes”.

10. Use branching logic. Branching logic can help a participant isolate the response she wishes to provide by a process of elimination. Suppose you need a participant to input a cancer diagnosis she had received. Given the wide variation in health literacy, we can’t assume she recalls the formal diagnosis. It may be more helpful to solicit this information through a series of questions easier to answer. Did the cancer involve a solid mass? Depending on the participant’s response, the following question might pertain to an organ system (if “yes” is selected) or blood cells (if “no” is selected). Just five yes or no questions can narrow a field of 32 options down to one.

Doesn’t the use of branching logic conflict with strategy of minimizing the number of fields? These are guidelines, not rules, so trade-offs are sometimes necessary. A drop-down menu showing 32 options may represent just one field, but scrolling through that many selections (not all of which will be visible at once) places an enormous hurdle in front of the participant. The mental effort and time spent scrolling on that one field far outweighs any time savings that might be secured by eliminating three fields. Meanwhile, you’ll have frustrated the participant.

11. Use autocomplete. There’s another way of solving the history of cancer problem above. As long as the participant can recall any portion of the diagnosis when spelled out, autocomplete can help them retrieve the full term. The best instances of autocomplete reveal all matches as the participant types, so that upon typing “Lym” a choice is immediate presented among:

Acute Lymphoblastic Leukemia (ALL)
Central Nervous System Lymphoma
Hodgkin Lymphoma
etc.

The ubiquity of predictive search among search engines like Google makes autocomplete a familiar experience for your participants.

In an era where summoning a taxi or making a credit card payment is a 20-second task, participants will not (and should not) tolerate inefficiency on the web. You are competing with their other online experiences, not just traditional, paper forms. The good news is that you can delight your participants by showing them that even contributing to medical research can be as easy as navigating their favorite web pages.

Show appreciation

You’ve read 85% of this post! Knowing that doesn’t guarantee you’ll read to the end, but it does make it more likely. Regular, responsive feedback is a powerful spur to action. Here are three ways to interact positively with your participant throughout (and even after) the form-filling process.

12. Convey their progress as the participant completes the form. Reflecting back to the participant the portion of the form they have completed and the portion that they have remaining serves two functions. The first is informative. You’ve anticipated the participant’s question (“how much longer?”) and answered it proactively. The second is motivational. Completing even simple tasks triggers the release of dopamine in our brains. We get a neurochemical rush every time we drop a letter in the mailbox or hit send on an email. 

You can reward your participant throughout the form-filling process by incorporating a dynamic progress bar into your ePRO form. Every page advance is an opportunity to “dose” your participant, visually, with a bit of happiness.

13. Autosave. Batteries die. Smartphones drop to the floor. Thumbs twitch and close screens. None of these scenarios is justification for losing a participant’s work. Your system should capture input on a field-by-field basis; that is, a back-end process should save the participant’s input into a field the moment he or she leaves that field for another. If a participant abandons a form and then returns to it, he or she should be able to resume where they left off. If you can signal back to the participant that their input has been saved with each field transition, all the better, as this leverages the same psychological power as the progress bar.  

14. Show gratitude. Imagine a campaign staffer asking you a series of questions about your views over the phone. You answer the last question and he or she hangs up without so much as a goodbye. Chances are, they’ve lost your vote on account of rudeness alone.

Don’t let this happen to your participants. When they submit a completed form online, they should immediately receive a “thank you” message that is specific to the task they have just completed.

Ensuring the optimal experience for participants supplying ePRO data is more than courtesy: it’s a critical measure for maximizing data quality, quantity and timeliness. So commit to dazzling the people who matter most in your research. Because, as in all relationships, you get back what you give. Click here to learn more about participant friendly ePRO from OpenClinica.

 

Resources

https://www.formassembly.com/blog/web-form-design/
https://www.ventureharbour.com/form-design-best-practices/
https://www.nngroup.com/articles/web-form-design/
https://medium.theuxblog.com/10-best-practices-for-designing-user-friendly-forms-fa0ba7c3e01f

OC Participate Delivers Better ePRO Data, Faster

With patient-centricity claiming more and more of the spotlight in both research and care (rightfully so), patient-reported outcomes will only play a large roles in clinical trials. But there are significant obstacles to getting quality data from PRO measures. Patients, especially those who are very sick, don’t want to hand-write dry medical diary entries. They don’t want to learn yet another electronic device, download and manage an app, or have to recall yet one more password. And who can blame them? Trial participants are the heroes of the research story, and when it comes to the collaborative process of data gathering, they deserve a hero’s welcome.

That’s why we developed OpenClinica Participate. We’re gratified by the success our clients have found leveraging this innovative ePRO solution, but we’re not surprised. When you prioritize a trial subject’s convenience and obsess over making things simple, you simply get better results. Here’s an example. Let’s call it:

Out with the old, in with the new

OpenClinica teamed with Danish CRO, Signifikans, to implement OpenClinica Participate for a leading Denmark-based bioscience technology company developing an innovative treatment to alleviate colorectal disorders, a common side effect of numerous medicines affecting millions of people at any time. The study’s objective was to investigate exercise induced intestinal permeability, immune markers, and bowel habits in 18-40 year old healthy volunteers. Participants were given two strains of bifidobacterium, an anaerobic bacteria that resides in the intestinal tract. The study involved 48 participants throughout Sweden, and each patient was required to provide 65 daily diaries in addition to 5 in-person visits over the course of the study.

The Old Way

In a similar prior study, the sponsor collected paper diaries from 700 participants. Each participant provided their (hopefully) completed and accurate paper diary to their site coordinator during the in-person study visits. The site coordinators then delivered the completed paper diaries to a data coordinating center. Coordinating center staff then scanned the diaries into a document management system and an overseas data entry vendor used a double data entry workflow to populate a database. Completed diaries were scanned and uploaded in batches for data entry. Phew!

On average, four months elapsed between the point of data capture and the first day of availability of that data to the sponsor. Monitoring participant compliance was also a challenge in this study, as it was
impossible to discern when each patient actually completed their daily diary. The expenditures associated with this process for data entry tallied over $213 per patient diary, or $4.97 per diary page.

Overall, this process was cumbersome, expensive, and logistically complex. The sponsor was planning a similar new study, and this time around was determined to find a way to

  • get faster access to the study data,
  • improve data reliability, and
  • reduce costs

The New Way

The sponsor enlisted local specialty CRO, Signifikans, to help it identify and implement a better approach. Signifikans recognized that with OpenClinica Participate, the sponsor could have immediate access to patient data, and that this data would automatically sit right alongside data captured from other sources during the study. No EDC integration was necessary.

Signifikans also took the lead on configuring the study in OpenClinica. (Our “make it simple” credo guides how we design tools for data managers, as well. That’s why we have invested so much in our forms engine, a topic for another post.) While building the study, Signifikans was able to easily demo prototypes to the sponsor along the way, iterating rapidly through edits and changes. Data capture forms were developed in the Swedish language, and the study was configured to send email reminders to patients to help ensure diaries were completed on time. The reminders contained a secure, uniquely-identified link the participant could click to go right to their diary, eliminating the need for participants to remember usernames and passwords.

Results

As soon as the study went live, the sponsor was able to monitor precisely when data were captured; something that was not possible with the old paper-based method. They observed, for example, that all five participants enrolled in the study’s first week each completed their diary card daily, per the protocol. The sponsor’s confidence in patient compliance and data quality surged; so much so, that they implemented an increase in the amount of data being collected this way. Scaling that quickly would have been impossible with paper diaries and slow transcription processes.

“Participate was very low friction: set-up was quick and efficient, and patients really seemed to embrace the technology.”

– Andreas Habicht, CEO, Signifikans Aps

The OpenClinica solution delivered a unified study database out-of-the-box, with patient-reported data sitting alongside clinician-reported data and accessible via the same interface. Having everything in one audit log made it easy to follow the patient’s trajectory through the study. Signifikans was able to use the same tools to configure and manage both ePRO and non-ePRO aspects of the study, resulting in a faster time-to-launch, and facilitating mid-study changes.

In addition to enhanced data quality and faster access to data, the cost of data capture per diary with OpenClinica Participate resulted in cost savings of over 80%.


Comparison: Paper vs. Participate
(click to enlarge)

Keep an eye out for more ePRO success stories on this blog. Our next post will delve into a different topic, but, as with this one, you can be sure it will feature better results through a better eClinical experience.

For Better ePRO Data, Empower Your Participants

What does the term “source” bring to mind for you? Paper files? A clinic’s EMR? Fair enough. Those are the typical formats of source data in clinical trials. But when you think about it, those records are a few removes from the true source: the patient.

Granted, most trials depend on a host of instruments and analyzers. No patient can self-report their own hemoglobin levels. But there are measures, such as quality of life, whose source really is the patient’s own experience. And for an industry whose raison d’être is enhancing (and sometimes saving) lives, we have a hard time obtaining those measures.

That’s not to say that patient-reported outcomes are a rarity in clinical trials. In fact they’re common. But so are obstacles to collecting them. Even today, with several ePRO solutions on offer, too many trials rely on patients to complete paper diaries. But paper records are prone loss or damage. Also, it’s virtually impossible to tell whether a patient made daily diary entries as instructed, or retrospectively wrote responses just prior to a study visit, raising data quality concerns. (Here’s a great analysis of “parking lot syndrome”.)

“Mochaccino with a side of ePRO” has to become the new normal for trials

ePRO is a big improvement on paper, but it doesn’t integrate seamlessly into a patient’s day to day life. Nearly all trials today using an electronic system for PROs provision dedicated devices to patients for this purpose. Patients need to be trained on how to use this device, recall that training at the relevant time, and make a habit of keeping the device on their person to stay current with reporting tasks. These systems also require patients to remember passwords and other access credentials. A step as apparently simple as downloading an app can be fraught with challenges: the patient must remember their app store login, locate the download area, have sufficient memory on their phone to house and run the app, ensure the app installs successfully, and ensure it remains installed, running, and updated.

All of the above examples are potential points of failure that can compromise the quality of real world data and increase trial costs. These myriad tasks and responsibilities unrelated to the data itself can place a significant burden on a volunteer patient, particularly one who is very sick.

OpenClinica Participate eliminates potential points of failure by:

  • providing an system that can be used on the patient’s own device(s),
  • eliminating the need to patient’s to remember login credentials, and
  • removing the complexities associated with apps.

This helps place the patient at the center of the research. And perhaps best of all, OpenClinica Participate places the patient reported data into the master EDC database in real-time with zero additional effort.

The advantages of this approach aren’t just theoretical. OpenClinica and its partners have realized their benefits in more than 20 studies since the launch of OpenClinica Participate about a year and a half ago. In this post and future ones in this series, I’d like to share some short and informal case studies. I call this one:

On time, nearly every time

S-cubed, an innovative pharmaceutical consultancy and contract service company, based in the UK and Denmark, recently supported a study assessing a medication to protect against a virus associated with the common cold. In the study, subjects were asked to self-report symptoms at multiple time points per day over several days. On most days, self-reports were to be submitted in the morning, in the afternoon, and at night. To help drive compliance with this rigorous self-reporting schedule, study managers wished to engage their subjects and ensure the data capture method was convenient. To do so, they tapped into the ability of OpenClinica Participate to present clear, attractive eCRFs to subjects on their own mobile devices (smartphone, tablet, or laptop). S-cubed data managers established rules that fixed the date of all future study events based on the protocol events schedule and each subject’s date of enrollment. OpenClinica Participate automatically sent text messages and emails to each subject prior to a scheduled event (and at the end of the time window if the diary was not completed), directing them to the relevant data capture form.

The data collection process involved no offline component. Subjects always provided their reports on a connected device, without regard to the specificity of device or location, etc. Consequently, fine-grained metadata regarding the date and time of eCRF completion was available. This metadata provided compelling evidence that the experience afforded to patients by OpenClinica Participate drives widespread data entry and timelines.

Of the 92 subjects who completed at least one of the ePRO forms associated with the study, thus signaling intent to participate, 89 (97%) completed all forms required in the morning, afternoon and evening over multiple consecutive days, when timely reporting was most critical, Reasons for non-completion were sometimes outside of OpenClinica Participate’s control.

Of the all “3x daily” forms completed, the vast majority were completed in real-time, with approximately 93% being completed within 1- 2 hours of the scheduled timepoints.

97% form completion rate

93% on-time rate for “3x daily” forms

“Bring your own device” may not be the final frontier in ensuring patient convenience. Passive monitoring by nearly invisible devices is already a reality. But the principle will never change: let technology do the work, so people can live their lives.

 

Is Your Clinical Trial Software Effective, or Just Efficacious? (Part 1 of 2)

Are you measuring all the relevant outcomes of your clinical trial technology?

For pure pathogen-killing power, it’s hard to beat a surgeon’s hand scrub. Ask any clinician, and she’ll tell you how thoroughly chlorhexidine disinfects skin. If she’s a microbiologist, she’ll even explain to you the biocide’s mechanism of action–provided you’re still listening. But how would the practice fare, say, as a method of cold and flu prevention on a college campus? Your skepticism here would seem justified. After all, it’s hard to sterilize a cough in the dining hall.

Efficacy and effectiveness. It’s unfortunate their phonetics are so close, because while the terms do refer to relative locations along a continuum, they’re the furthest thing from synonyms, as the ever accumulating literature on the topic will attest.

In this post and the one that follows, I’d like to offer some clarity on efficacy vs. effectiveness and illustrate the value that each type of analysis offers. If nothing else, what emerges should provide an introduction to the concepts for those new to clinical research. But I have a more speculative aim, too. I’d like propose standards for assessing trial technology through each of these lenses. Why? Because while we’ve been asking whether a particular technology does what it’s explicitly designed to do, as we should and must, we may have forgotten to ask a critical follow-up question: Does it improve the pace and reliability of our research?

Continue reading Is Your Clinical Trial Software Effective, or Just Efficacious? (Part 1 of 2)

Reducing friction in patient engagement: an (unconventional) case study

Participate_SCDM_SurveyOur quest for frictionless, electronic patient reported outcomes (ePRO) data capture has us looking for novel ways to engage patients and streamline process. I’d like to share a fun and interesting example of this work, in which we used Participate (the OpenClinica ePRO solution) to engage study subjects at the recent SCDM annual conference.

Our goal at the SCDM conference was to get as many attendees to try OpenClinica Participate as possible. With the vast array of vendors, eye candy, and giveaways, it’s a big challenge to cut through the noise and offer a simple, fun way to engage an audience. The same holds true when engaging patients. With the enormous number of daily distractions, ensuring that your patients can quickly access, fill out and submit well-constructed, simple forms is key to compliance and ultimately, better data.

I built the form, shown here, in OpenClinica and enabled access to it via a custom URL, a new feature in our latest release.

Attendees filled out the form, sprinkled with fun health habit questions, then captured information to allow us to draw their names to win Fitbits and other giveaways. We were able to use this data and update our graphs to give the participant a view of how they stacked up with their peers—cool!

Imagine if patients could view a visual representation of the study they are enrolled in – see the parallels and possibilities?

Graphs_SCDM

Who says ePRO and patient engagement can’t be fun?

Calculating ROI for ePRO

I recently delivered a webinar titled “Getting Started with eCOA/ePRO,” in which roughly a third of attendees polled cited expense as the number one reason that has prevented them from adopting an ePRO solution. So what does ePRO really cost? Is it worth it? Here I strive to provide a basic, high-level framework for thinking about the return on investment ROI of eCOA vs. paper.

Let’s start by taking a look at the costs that are unique to each approach.

Paper

In a traditional paper based model, you are incurring costs that stem from printing, mailing, data entry, and data cleaning. These are all expenses than can be estimated with a fair degree of accuracy, with the cost of data entry being the most significant of these. To estimate the cost of data entry, see how long it takes to key in a subject completed paper casebook, multiply this by your cost of labor (don’t forget to include overhead!).

ePRO

The cost side for ePRO is similarly straightforward, but the expense elements are different. You’re either building an ePRO system (which will almost carry a highly unpredictable cost) of buying one (much more predictable cost). Assuming you’re buying, here are the costs you may expect to incur:

· License
· Hosting
· Training and support
· Professional services (e.g. study configuration)
· Devices

You should evaluate whether your study and selected ePRO system will allow for patients to use their own devices, or if you will need to provision devices (or some mix thereof). The cost of provisioning devices, especially for a global study can be significant—in addition to the costs of the devices themselves, you will need to consider the costs of data plans, and logistics associated with supporting the devices. I’m a big fan of BYOD (bring your own device) but, depending on the study, it may not be feasible to utilize while maintaining scientific validity of data collected.

Once you’ve mapped out your costs of each route, you can begin to weigh these against the benefits of going eCOA.

cropped-istock_000037068804.jpg

Paper vs. eCOA

When you boil it down, people employ ePRO/eCOA to maximize data quality, increase productivity, and/or enable new capabilities that help answer their research questions. ePRO is e-source, so you don’t have worry about administering a paper data entry process. Depending on the study, the cost savings from this alone might justify ePRO.

There are some additional benefits ePRO offers over paper that may be harder to quantify, but nonetheless  very real. For example, there are clear data quality benefits to ePRO. The electronic system can ensure a minimum standard of data quality through edit checks and enforced data structures. ePRO data will always be cleaner than the same data captured on paper.

Benefits and Motivations for eCOA

 

The use of an ePRO system also allows you to know for sure when the data were recorded. For instance, patients can be reminded automatically when their diaries are overdue, and you now only have much stronger assurances that data were collected at the appropriate time (vs paper), you can also more easily monitor the study progress.

Bypassing manual data entry and having the system provide notifications to subjects to ensure data are captured in a timely way might allow for faster and better in-study decision making and even may accelerate study closeout. Also, an increasing amount of evidence exists that mobile-based messaging and communication strategies help increase patient engagement and treatment adherence. And of course, not having to deal with a stack of paper during a site visit might allow the clinician’s interaction with the patient to be higher quality.

Quantifying the benefits of all of these things can be tough, but start with those which are most quantifiable and see if those items alone these alone provide a compelling ROI (from my experience they often do).  Then the less tangible benefits become gravy to the ROI argument.  When modeling costs over time and a pay-back period, keep in mind that ePRO will typically carry a higher upfront cost than paper, with the cost saving benefits realized downstream over time. With today’s technologies, even most smaller studies should be able to realize a positive payback.

Naturally, there may be additional ROI factors to consider which are specific to your situation. If you have particular thoughts, questions, or experiences on this topic I encourage you to add a comment to this post.

Engage. Learn. Repeat.

At OpenClinica we are driven to reduce obstacles to the advancement of medical research. The OpenClinica open source project started because EDC was too complex, too inaccessible, and too expensive. Not to mention far too difficult to evaluate and improve. So we built an EDC / CDMS platform and released it under an open source license. It is now the world’s most widely used open source EDC system and has an active, growing user community.

 

As the user base grew, we listened to users and understood that integration and interoperability were another major obstacle. While we don’t claim to have fully cracked that nut yet, OpenClinica’s CDISC ODM-based APIs have been pretty widely adopted and helped to drive some significant innovations. These APIs have been improved upon by a large number of developers in the few years they have been part of the codebase.

 

As we continue to improve the clinical and researcher experience, our attention has more recently been directed to the experience of trial participants. The difficulty of meaningful, timely engagement with these volunteers also strikes us as an obstacle to successful research. We live in a world where 90% of American adults have mobile phones, 81% text, and 63% use their phone to go online (Pew), and even older age groups are adopting smartphones at a rapid pace [1]. Because of this, we think that mobile technology could be a pretty effective means to help more meaningfully engage participants research.

 

Why is this important? Treating research volunteers as participants, as opposed to subjects, can lead to concrete benefits – improving participation, motivation, and adherence. Increasing your ability to meet recruitment goals, budget, and completion timelines. Getting more complete, timely data. Even enabling new protocol designs that better target populations and/or more closely align with real-world use. But most of all, it just seems like the right thing to do. As one HIV trial participant put it, “I’d initially had this nagging fear in my head, that, once recruited, I would cease to be nothing more than a patient number – a series of digits, test results and charts in a file – which is quite a daunting prospect when you’re not entirely sure how your body is going to respond to the vaccine. This could not have been further from the reality of the trial. I felt safe, informed and valued at every stage of the trial.”

 

The great (and often unrecognized) news is that so many of the people involved in research and care already do an unbelievable job creating this type of engagement – making participants feel safe, informed, and valued. But it takes a lot of work. With a mobile-enabled, real-time solution like OpenClinica Participate, you can provide an engagement channel and data capture experience that is simple, elegant, and easy to use on any device. Because it is fully integrated with OpenClinica and captures data in a regulatory-compliant manner, you can reduce time and headache for your research team from, for instance, merging disparate sources of data and keying in paper reports. Leaving you more time to focus on the kinds of human to human engagement that technology cannot do.
Participate Webinar
[1]  For the over 55 age group, most likely to participate in many types of trials, the picture is a bit different. As of 2013, around 80% have mobile phone but only 37% are smartphones. However over-55s are the fastest growing smartphone adopters, expected by Deloitte to soon reach 50% and reach parity with other age groups by 2020.
See http://www2.deloitte.com/content/dam/Deloitte/global/Documents/Technology-Media-Telecommunications/gx-tmt-2014prediction-smartphone.pdf. Outside of the developed world, the picture is different, though the opposite of what you might expect. According to Donna Malvey, PhD, co-author of mHealth: Transforming Healthcare, cell phones are even more pervasive, and mHealth “apps are the difference between life and death. If you’re in Africa and you have a sick baby, mHealth apps enable you to get healthcare you would normally not have access to… In China and India, in particular, mobile apps can bring healthcare to rural areas. “

 

Reacting to #ResearchKit

OpenClinica_AppleRK Apple, Inc. has a remarkable ability to capture the world’s attention when announcing “the next big thing.” They have honed their well-known Reality Distortion Field skills for over 30 years. As the largest company in the world, and bellwether of the technology industry, Apple’s announcements are immediately recounted, opined, lionized, and criticized all across the Internet—sometimes with very limited real information on the new product itself. Of course, it helps to have their unmatched track record in actually delivering the next big thing.

ResearchKit has grabbed such attention. Maybe not as much as The Watch, but amongst the minority of us who pay attention to such things. And the reactions have been typically polarized—it’s either an “ethics quagmire” or “Apple fixing the world.”

But reality rarely presents an either-or proposition. I’ve written before on the need to use technology in simple, scalable ways to engage more participants in research and capture more data. Every form of engaging with patients and conducting research is fraught with potential for bias, bad data, and ethical dilemmas. Properly controlling these factors is difficult, and the current handling of these factors lead many to conclude that clinical research is overly “bloated and highly controlled”. There’s truth to that, but the fundamental need for good controls is real. As technology enables us to engage in new ways, how we implement such controls is likely to transform, perhaps unrecognizably so.

I don’t think Apple—or anybody—has these problems fully solved yet. And I expect we’re going to a see a vigorous debate in coming years between #bigdata and #cleandata that I hope will lead us to more of both. But ResearchKit, or at least the announcement thereof, is a game changer. Whether or not ResearchKit in its present form becomes a widely adopted platform, the impact was felt overnight: “11,000 iPhone owners signed up for a heart health study using Apple’s newly-announced ResearchKit in the first 24 hours… To get 10,000 people enrolled in a medical study normally, it would take a year and 50 medical centers around the country”. ResearchKit builds on momentum towards patient-centricity established in the last five years within pharma, NIH, online patient communities, mHealth, and health care, and uses Apple’s consumer clout to bring it to the attention of the average person on the street.

So let’s break down what we know about ResearchKit. Since this is a blog about OpenClinica, we’ll also share early thoughts on how we see OpenClinica, ResearchKit, and OpenClinica Participate fitting together.

It’s Open Source. Great move! We’ll learn more about what this means when the code is released next month.

The technical paper indicates it is a front-end software framework for research, and that they expect it to expand over time as modules are contributed by researchers. Through use of both platforms’ APIs, OpenClinica could serve as a powerful backend and ‘brain’ to ResearchKit.

It’s not clear if data goes through Apple’s servers on its way to a final destination. I also haven’t seen anything from Apple mentioning if it will be portable to other non-iOS platforms (which represent 80% of mobile device market share), though its open source nature would suggest that will be possible.

Surveys. Analogous in many ways to the forms module in OpenClinica Participate, it is a pre-built user interface for question and answer surveys. As somebody who’s worked in this realm for years, I know that this can mean a lot of things. What specific features are supported, how flexible is it, how easy is the build process? Perhaps most important, can it be ported to other mobile app platforms, or to the web?

Informed Consent. The need for fundamental ethical controls for for research conduct and data use are just as important in the virtual world as they are in the brick-and-mortar realm, and Informed Consent is a cornerstone. I’m glad to see ResearchKit taking this on; I don’t expect they have it 100% figured out, but their work with Sage Bionetworks, who has released an open toolkit on Participant Centered Consent, is a great sign.

Active Tasks. Maybe the most exciting component, here’s where ResearchKit takes advantage of the powerful sensors and hardware in the device and provides a way to build interactive tests and activities. In this way, I expect ResearchKit will be a great complementary/alternative frontend to OpenClinica Participate when specialized tests tied to specific, highly-calibrated devices are required.

In general, the promise is big: that technology will lower barriers in a way that leads to fundamental advances in our understanding of human health and breakthrough treatments. That we’ll go from data collected once every three months to once every second, and we’ll encounter–and solve–problems of selection bias, identity management, privacy, and more along the way. And that, according to John Wilbanks at Sage, “there’s coming a day when you’re not going to have an excuse to have a tiny cohort just because you chose not the use digital technologies to engage people.”