Rho site logo

Rho Knows Clinical Research Services

Challenges in Clinical Data Management: Findings from the Tufts CSDD Impact Report

Posted by Brook White on Fri, Feb 09, 2018 @ 12:24 PM

Derek Lawrence, Senior Clinical Data ManagerDerek Lawrence, Senior Clinical Data Manager, has 9 years of data management and analysis experience in the health care/pharmaceutical industry.  Derek serves as Rho's Operational Service Leader in Clinical Data Management, an internal expert responsible for disseminating the application of new technology, best practices, and processes.

The most recent Impact Report from the Tufts Center for the Study of Drug Development presented the results of a study including nearly 260 sponsor and CRO companies into clinical data management practices and experience. A high-level summary of the findings included longer data management cycle times than those observed 10 years ago, delays in building clinical databases, a reported average of six applications to support each clinical study, and a majority of companies reporting technical challenges as it pertained to loading data into their primary electronic data capture (EDC) system.

These findings represent the challenges those of us in clinical data management are struggling with given the current state of the clinical research industry and technological changes. EDC systems are still the primary method of data capture in clinical research with 100% of sponsors and CROs reporting at least some usage. These systems are experiencing difficulties in dealing with the increases in data source diversity. More and more clinical data are being captured by new and novel applications (ePRO, wearable devices, etc.) and there is an increased capacity to work with imaging, genomic, and biomarker data. The increases in data changing EDC paradigmvolume and data velocity have resulted in a disconnect with the EDC paradigm. Data are either too large or are ill-formatted for import into the majority of EDC systems common to the industry. In addition, there are significant pre-study planning and technical support demands when it comes to loading data into these systems. With 77% of sponsors and CROs reporting similar barriers to effective loading, cleaning, and use of external data, the issue is one with which nearly everyone in clinical research is confronted.

EDC integrationRelated to the issues regarding EDC integration are delays in database build. While nearly half of the build delays were attributed to protocol changes, just over 30% resulted from user acceptance testing (UAT) and database design functionality. Delays attributed to database design functionality were associated with a LPLV-to-lock cycle time that was 39% longer than the overall average. While the Tufts study did not address this directly, it would be no great stretch of the imagination to assume that the difficulties related to EDC system integration are a significant contributor to the reported database functionality issues. With there already being delays associated with loading data, standard data cleaning activities that are built into the EDC system and need to be performed before database lock would most certainly be delayed as well.

Clinical data management is clearly experiencing pains adapting to a rapidly-shifting landscape in which a portion of our current practices no longer play together nicely with advances in data-mining.jpgtechnology and data source diversity. All of this begs the question “What can we do to change our processes in order to accommodate these advances?” At Rho, we are confronting these challenges with a variety of approaches, beginning with limiting the impulse to automatically import all data from external vendors into our EDC systems. Configuring and updating EDC systems requires no small amount of effort on the part of database builders, statistical programmers, and other functional areas. Potential negative impacts to existing clinical data are a possibility when these updates are made as part of a database migration. At the end of the day, importing data into an EDC system results in no automatic improvement to data quality and, in some cases, actually hinders our ability to rapidly and efficiently clean the data. In developing standard processes for transforming and cleaning data external to the EDC systems, we increase flexibility in adapting to shifts in incoming data structure or format and mitigate the risk of untoward impacts to the contents of the clinical database by decreasing the prevalence of system updates.

The primary motivation for loading data received from external vendors into the EDC system is to provide a standard method of performing data cleaning activities and cross-checks against the clinical data themselves. To support this, we are developing tools to aggregate that data from a variety of sources and assemble them for data cleaning purposes. Similar to the ways the banking industry uses machine learning to identify ‘normal’ and ‘abnormal’ spending patterns and make real-time decisions to allow or decline purchases, similar algorithms can identify univariate and multivariate clusters of anomalous data for manual review. These continually-learning algorithms will enable a focused review of potentially erroneous data without the development of the traditional EDC infrastructure. This will save time performing data reviews and also identify potential issues which we would normally miss had we relied on the existing EDC model. With the future state resulting in an ever-broadening landscape of data sources and formats, an approach rooted in system agnosticism and sound statistical methodology will ensure we are always able to provide high levels of data quality.

5 Reasons Your CRO is Putting Your Trial at Risk

Posted by Brook White on Mon, Mar 10, 2014 @ 09:44 AM

Information for this article was contributed by Alicia McNeil and Elizabeth Kelchner, Clinical Data Scientists at Rho with significant experience in rescue studies.

warning signs in clinical trialsNo one wants to be here: You went through a lengthy proposal process, endured the bid defense process and thought you selected the best CRO for your study. The contract is signed and the study has started, but things just don’t seem right.

Here are five red flags to watch for if you suspect problems with your CRO, as well as tips for what to do if you decide your study needs to be rescued.

  1. You loved the team you met at the bid defense meeting, but once your study started you were assigned a completely new (and less experienced) team.
    Some new assignments are always a possibility. Maybe one of the team members left the company or a team member was unavailable because a study that they anticipated would be complete ran over. That said, you shouldn’t see substantial turnover on your team, and replacement team members should have similar experience and expertise to those you met at the bid defense.
  2. Project team members don’t return your calls or respond to email in a timely fashion.
    You can’t expect your project team to sit at their desks all day to answer the phone and check their email. After all, they need to be busy working on your project. So what is a reasonable amount of time to wait for a response? Hopefully, they’ve set clear expectations for response times during the kick-off meeting and in the project management plan, but one business day is a typical benchmark for non-urgent communications. There also should be a process and expectations in place to deal with time critical issues.
  3. You haven’t seen and haven’t been asked to sign off on important study documents.
    There are a variety of documents that the CRO should draft during study start-up (project management plan, clinical monitoring plan, data management plan, etc.). You should be given the opportunity to provide input, review, and sign off on these documents as they will set the direction for the execution of the study and ensure expectations are set on all sides.
  4. You aren’t receiving regular status reports.
    Status reports are another topic your project team should have covered at the kick-off meeting and in study documents so that you know what to expect in each area for which the CRO has contracted responsibility. They should be sending you status reports in a consistent format on a regular schedule. The reports should include enough detail that you can track the progress of important activities and gauge any significant risks to the study.
  5. There are signs data isn’t being collected or managed properly.
    In a study that is running smoothly, you should expect to see the following in terms of data collection and management:
    • Data is being collected in a system designed to handle clinical trial data. If data is being collected in spreadsheets or another unorthodox manner, this is a very significant problem. This may seem obvious, but more than once we’ve rescued studies where data was being collected into Excel spreadsheets.
    • You are given the opportunity to participate in User Acceptance Testing (UAT) if your study is using EDC. This is a good way to familiarize yourself with the specifics of how data is collected and what sites will experience, and it demonstrates transparency on the part of the CRO.
    • Queries are being sent and closed on a regular basis. As soon as sites start collecting data, you should be getting updates about queries sent and closed.

Sometimes, despite your best efforts to correct course with your CRO, you may decide that you need to change CROs mid-stream.  Here are some tips for interacting with your existing CRO before and during the transition as well as tips for selecting a CRO to rescue your study:

Tips for working with the incumbent CRO:

  • Keep copies of all documentation (study plans, annotated CRF, build specifications, decision logs, etc.) in case you need to transition to another CRO. Don’t rely on the incumbent CRO to do this. The more historical information available to the new CRO, the better.
  • Get regular data transfers.
  • Request and implement a communication plan. Know how to escalate issues if needed.
  • Have regular meetings with all team members to keep everyone on the same page. If possible, meetings that include both members of the incumbent CRO team and the rescue team can make the transition much smoother.
  • Avoid burning bridges with the incumbent CRO early in the process.
  • Negotiate the financial side of transitions carefully so that the incumbent CRO can work as necessary to complete tasks where practical given timeline constrains as opposed to relying on the new CRO to complete all tasks. This also allows appropriate communication between the incumbent and the new CRO.

rescue trialsTips for Selecting a CRO for the Rescue

  • They should have a thoroughly documented rescue process or rescue project plan template.
  • They’ve successfully implemented rescues between the applicable platforms (i.e., EDC to EDC, paper to EDC, EDC to paper).
  • They understand the need to minimize and manage process change for clinical trial sites.
Click me

Choosing the Right System for your Clinical Trial: Understanding the Differences between EDC and IVR/IWR

Posted by Jamie Hahn on Tue, Apr 09, 2013 @ 09:05 AM

headshot steve palmatierThe following article was contributed by Steve Palmatier, Rho's service leader for Interactive Response Technology (IxR) system configuration and development.

Sometimes it's difficult to determine the best tool for a job, especially when technologies are developed in parallel to handle similar tasks.  Take Interactive Response Technology (IxR) and Electronic Data Capture / Electronic Case Report Forms (EDC), for example.  Both technologies provide a method for electronic entry of important data.  Both can have data verification checks incorporated to minimize the potential for ambiguous or incorrect data entry.  Both commonly incorporate user roles to limit access of individual users to functionality that is appropriate.  So what are the differences that would provide insight on which technology to use when?  Several areas of differentiation are outlined below.

Purpose of the System

EDC - In short, EDC systems’ primary purpose is to electronically collect and validate participant data for eventual use in statistical analyses.  Collecting these data electronically makes them more quickly available to the study team than traditional paper CRFs, and therefore allows more informed and proactive decision making.

IxR – The goal of IxR in clinical trials is to perform specific tasks, such as randomization, study drug dispensation, study drug resupply requests, emergency unmasking, etc.  It is not the goal of IxR in most cases to be the primary place where participant data are entered and stored, though some data are required to perform the aforementioned tasks.

System Interface

EDC – Due to the sheer volume of data to be captured, EDC systems nearly always use a computer-based interface that allows users to easily navigate between forms and between different areas on the same form.  While swift entry of data into EDC systems is often desired so that study teams have accurate enrollment information, it is not usually operationally critical, so it is acceptable for a user to enter data in not-quite-real-time.  Moreover, most clinical sites in developed countries can be expected to have computers, so a computerized interface is acceptable the vast majority of the time.

IxR – IxR has two main interfaces: web and voice (IWR and IVR respectively).  Over the past 10 years or so, the prevalence of IVR systems has decreased significantly due to workstations, laptops, smartphones, and tablets becoming more widely available in the clinical setting.  However, there are still some instances in which the phone interface is beneficial, such as when entry of data for randomization is highly time-sensitive (e.g., in neonatal trials where randomization must occur very shortly after birth), and when the IxR will be used for patient-reported outcomes or diary entry, since study subjects may not have access to a computer at home.

Navigation Paradigm

EDC - Most EDC systems are form-based, and most of the data entry fields on any particular web page are static.  When a participant is enrolled in a trial, a set of forms is made available into which that participant’s data will be entered.  Whether these forms are necessary or not becomes apparent later.  For instance, if a participant withdraws consent early in the study, there may be many forms for visits later in the study that never have data associated with them.  In many cases, the order in which data are entered is not controlled since different data will become available at different times, though sometimes additional forms are generated as they become necessary (e.g., SAE forms).

IxR - IxR systems generally create data entry pages dynamically.  That is, the information and entry fields that appear on-screen or that are prompted over the phone are a result of previous selections and entries made by the user.   This both minimizes data entry by the user and provides a gating mechanism that forces things to happen in the correct order.  For instance, a user cannot skip to kit assignment prior to randomization, or randomization prior to entry of valid stratification data.

User Modification of Previously Entered Data

EDC - EDC forms can usually be revisited multiple times because all of the data that are to be entered on a form may not be available at once (e.g., lab values).  Often, entry of data that seems inaccurate or is in an incorrect format is accepted and stored but fires a query that must be resolved prior to database lock, and the user may return at a later time to correct or confirm the entry.  This is consistent with the primary purpose of EDC, to store data for use in data analysis that will take place at a later date.

IxR - Unlike EDC forms, entry of data and completion of a function in IxR usually triggers an action that is based on the entered data, so it is uncommon for a user to be able to return to the system to make corrections of previously missing or incorrectly entered data without support intervention.     Incorrect entry of stratification data prior to randomization has cascading impacts, so correcting the mistake often involves more than simply updating that one data point.

Validation Burden

EDC – Because there is an opportunity to correct mistakes between the initial entry and database lock, the importance of correct and complete data at the time of entry is not often assessed to be at the highest level.  Also, since the primary purpose of EDC is to store data rather than to perform actions, validation of the system can focus primarily on making sure that edit checks fire correctly and that the data is stored accurately.

IxR - Because IxR performs actions that impact the course of the study, IxR systems generally carry a higher risk than EDC systems.  Not only is it important for validation efforts to ensure that the entered data is correct; but it is also important to validate the logic that is exercised in order to make the decisions and perform the actions that are based on that data – assigning the correct treatment kits, requesting resupply of investigational product when appropriate, enforcing cohort caps, etc.  The result is that IxR systems (especially those that are highly configurable) generally require more extensive validation and a higher percentage of setup time allotted to validation activities.

In the next post in this series, we’ll use these distinctions to help determine the appropriate scope for IxR systems so that the technology can be used most advantageously.

Free Expert Consultation