Rho site logo

Rho Knows Clinical Research Services

The Rise of Electronic Clinical Outcome Assessments (eCOAs) in the Age of Patient Centricity

Posted by Brook White on Tue, Dec 06, 2016 @ 10:36 AM

Lauren Neighbours, Clinical Research ScientistLauren Neighbours is a Research Scientist at Rho. She leads cross-functional project teams for clinical operations and regulatory submission programs and has over ten years of scientific writing and editing experience. Lauren has served as a project manager and lead author for multiple clinical studies across a range of therapeutic areas that use patient- and clinician-reported outcome assessments, and she worked with a company to develop a patient-reported outcome instrument evaluation package for a novel electronic clinical outcome assessment (eCOA).

Jeff Abolafia, Chief Strategist Data StandardsJeff Abolafia is a Chief Strategist for Data Standards at Rho and has been involved in clinical research for over thirty years. He is responsible for setting strategic direction and overseeing data management, data standards, data governance, and data exchange for Rho’s federal and commercial divisions. In this role, Jeff is responsible for data collection systems, data management personnel, developing corporate data standards and governance, and developing systems to ensure that data flows efficiently from study start-up to submission or publication. Jeff has also developed systems for managing, organizing, and integrating both data and metadata for submission to the FDA and other regulatory authorities.

patient centricityWith the industry-wide push towards patient-centricity, electronic clinical outcome assessments (eCOAs) have become a more widely used strategy to streamline patient data collection, provide real-time access to data (for review and monitoring), enhance patient engagement, and improve the integrity and accuracy of clinical studies.  These eCOAs are comprised of a variety of electronically captured assessments, including patient reported outcomes (PROs), clinician-reported and health-care professional assessments (ClinROs), observer reported outcomes (ObsROs), and patient performance outcomes administered by health-care professionals (PerfOs).  The main methods for collection of eCOA data include computers, smartphones, and tablets, as well as telephone systems.  While many companies have chosen to partner with eCOA vendors to provide these electronic devices for use in a clinical study, other sponsors are exploring “bring your own device (BYOD)” strategies to save costs and start-up time.  No matter what strategy is used to implement an eCOA for your clinical study, there are several factors to consider before embarking on this path.  

Designing a Study with eCOAs

The decision to incorporate an eCOA into your clinical study design is multifaceted and includes considerations such as the therapeutic area, the type of data being collected, and study design, but the choice can first be boiled down to 2 distinct concepts: 1) the need for clinical outcome data from an individual, and 2) the need for this data to be collected electronically. Thus, the benefits and challenges to eCOAs can be aligned with either or both of these concepts.

Regarding the first concept, the need for clinical outcome data should be driven by your study objectives and a cost-benefit analysis on the optimal data collection technique. Using eCOAs to collect data is undoubtedly more patient-centric than an objective measure such as body mass index (BMI), as calculated by weight and height measurements. The BMI calculation does not tell you anything about how the patient feels about their body image, or whether the use of a particular product impacts their feelings of self-worth. If the study objective is to understand the subjective impact of a product on the patient or health-care community, a well designed eCOA can be a valuable tool to capture this information. These data can tell you specific information about your product and help inform the labeling language that will be included in the package insert of your marketed product. Additionally, FDA has encouraged the use of PROs to capture certain data endpoints, such as pain intensity, from a patient population who can respond themselves (see eCOA Regulatory Considerations below). Of course, it’s important to note that the inherent subjectivity of eCOAs does come with its own disadvantages. The data is subject to more bias than other objective measures, so it’s critical to take steps to reduce bias as much as possible. Examples of ways to reduce bias include single- or double-blind trial designs, wherein the patient or assessor is not aware of the assigned treatment, and building in a control arm (e.g., placebo or active comparator) to compare eCOA outcome data across treatment groups.

Another important concept is the process for identifying and implementing the electronic modality for eCOA data collection.  Many studies still use paper methods to collect clinical outcome data, and there are cases when it may make more sense to achieve your study objectives through paper rather than electronic methods (e.g., Phase 1 studies with limited subjects).  However, several types of clinical outcome data can be collected more efficiently, at lower cost, and at higher quality with electronic approaches (e.g., diary data or daily pain scores).  From an efficiency standpoint, data can be entered directly into a device and integrated with the electronic data management system being used to maintain data collection balancing time and cost when considering paper or electronic clinical outcomes assessmentsfor the duration of the study.  This saves time (and cost) associated with site personnel printing, reviewing, interpreting, and/or transcribing data collected on paper into the electronic data management system, and it also requires less monitoring time to review and remediate data.  Additionally, paper data is often “dirty” data, with missing or incorrectly recorded data in the paper version, followed by missing or incorrectly recorded data entered into the data management system.  The eCOA allows for an almost instantaneous transfer of data that saves the upfront data entry time but also saves time and cost down the road as it reduces the effort required to address queries associated with the eCOA data.  Aside from efficiencies, eCOA methods allow for more effective patient compliance measures to be implemented in the study.  The eCOA device can be configured to require daily or weekly data entry and real-time review by site personnel prior to the next scheduled clinic visit.  Additionally, the eCOA system can send out alerts and reminders to patients (to ensure data is entered in a timely manner) and to health-care personnel (to ensure timely review and verification of data and subsequent follow-up with patients as needed).  The downsides to electronic data collection methods tend to be associated with the costs and time to implement the system at the beginning of the study.  It’s therefore essential to select an appropriate eCOA vendor  early who will work with you to design, validate, and implement the clinical assessment specifically for your study.

eCOA Regulatory Considerations

In line with the industry push for patient-focused clinical studies, recent regulatory agency guidance has encouraged the use of eCOAs to evaluate clinical outcome data.  The fifth authorization of the Prescription Drug User Fee Act (PDUFA V), which was enacted in 2012 as part of the Food and Drug Administration Safety and Innovation Act (FDASIA), included a commitment by the FDA to more systematically obtain patient input on certain diseases and their treatments.  In so doing, PDUFA V supports the use of PRO endpoints to collect data directly from the patients who participate in clinical studies but also as a way to actively engage patients in their treatment.  The 2009 FDA guidance for industry on Patient-Reported Outcome Measures: Use in Medical Product Development to Support Labeling Claims , further underscores this idea by stating “[the use] of a PRO instrument is advised when measuring a concept best known by the patient or best measured from the patient perspective.”  The 2013 Guidance for Industry on Electronic Source Data in Clinical Investigations  provides the Agency’s recommendations on “the capture, review, and retention of electronic source data” and is to be used in conjunction with the 2007 guidance on Computerized Systems Used in Clinical Investigations for all electronic data and systems used in FDA-regulated clinical studies, including eCOAs.  To support these efforts, the FDA has developed an extensive Clinical Outcome Assessment Qualification Program, which is designed to review and assess the design, validity, and reliability of a COA for  a particular use in a clinical study.  Furthermore, the newly formed Clinical Outcome Assessment Compendium  is a collated list of COAs that have been identified for particular uses in clinical studies.  The COA Compendium is further evidence of FDA’s commitment to patient-centric product development, and it provides a helpful starting point for companies looking to integrate these assessments into their clinical development programs. 

Before choosing an eCOA for your clinical development program, the following regulatory factors should be considered:

  • FDA holds COAs to the same regulatory and scientific standards as other measures used in clinical trials. Thus, it is advisable to refer to the Guidance for Industry on Patient-Reported Outcomes and the available information on the COA Assessment Qualification program and COA Compendium provided by the Agency when implementing eCOAs into your development program. If you plan to divert from currently available regulatory guidance, make sure to have a solid rationale and supporting documentation to substantiate your position.
  • The qualification of an eCOA often requires input from patients and/or health-care professionals to evaluate the effectiveness of the assessment. This input is necessary for the regulatory agency to determine whether the eCOA can accurately measure what it’s supposed to measure (validity) and to demonstrate it can measure the outcome dependably (reliability).
  • Data collected from qualified and validated eCOAs can be used to support product labeling claims. The key is to use an eCOA when it’s appropriate to do so and to make sure the eCOA supports your intended labeling claims because the instrument will be evaluated in relation to the intended use in the targeted patient population.
  • For the cases where an instrument was developed for paper based collection or an instrument is collected using multiple modes, it may be necessary to test for equivalence. This regulatory expectation is often required (especially for primary and secondary endpoints) to ensure that the electronic version of the instrument is still valid and data collected with mixed modes are comparable.

A CRO Can Help with your eCOA Strategy

CROs partner with sponsor companies to develop and execute their product development strategies.  In some cases, this involves implementing clinical outcome measures into a development program and then facilitating the interactions between the company and regulatory authorities to ensure adequate qualification of the COA prior to marketing application submission.  Whether or not you choose to engage a CRO in your development plan, consider seeking outside consultation from the experts prior to establishing your eCOA strategy to give you and your company the best chance of success.  

CROs Can Help:

  • Determine endpoints where eCOA data is appropriate
  • Determine the cost/benefit of electronic vs paper data capture
  • Determine the best mode of electronic data capture
  • Recommend eCOA vendors when appropriate
  • Perform equivalence analysis
  • Facilitate discussions with regulatory authorities
  • Manage the entire process of eCOA implementation

Webinar: ePRO and Smart Devices

Scientists Search for Answers as Antibiotics become Obsolete

Posted by Brook White on Thu, Aug 18, 2016 @ 01:41 PM

Carrie Furr, PhD, RACCarrie Furr, PhD, RAC, is a Senior Director Operations at Rho. Day-to-day, Carrie supports pharmaceutical sponsors of clinical trials, leading an integrated product development program consisting of clinical, preclinical, chemistry, manufacturing and controls, and regulatory components. Before diving into the clinical trials industry, Carrie spent 7 years earning her PhD in biochemistry, bacteriology and bacteriophage biology at Texas A&M University. A biologist by training, Carrie’s dissertation and doctoral research focused on how a bacteriophage (phage) protein causes bacteria to die. In theory, phage proteins can be used in phage therapy to combat any bacteria-based disease. 

As anyone who has ever had a bacterial infection or seen an end-of-the-world survival movie knows, antibiotics are an important tool in a doctor’s arsenal. Since the discovery of penicillin in 1928, antibiotics have been extremely effective at treating and preventing a variety of infections. But imagine life without them – doctors unable to prevent infections after surgery, your child’s minor cut might morph into a major infection, and there wouldn’t even be a treatment for pink eye.

With the increasingly popular use of antibiotics, however, microbes have mutated and learned to resist the drugs. Penicillin was once extremely effective against most strains of bacteria, but now it is used far less frequently as many strains have built up resistance. In many cases antibiotics are becoming obsolete.

What can be done? Without antibiotics, a huge range of diseases -- from pneumonia to strep throat to syphilis – would become much more difficult, if not impossible, to treat. While there are a few things you can do to help prevent antibiotics from further increasing resistance – for example, taking all prescribed antibiotics even if you feel better or only taking antibiotics when you truly have an infection – it isn’t enough. Scientists need to work on other solutions.

phage, bacteriophageBacteriophages, or “phages” for short, may be able to help. Certain phage proteins cause bacteria to die. Researchers are working to determine if these proteins or whole phages can be safely converted into therapies to combat bacteria-based diseases. Developing an alternative treatment to antibiotics could have huge implications on the treatment of bacterial infections around the world. Additionally, phage therapy holds the promise of providing a dynamic solution to the dynamic problem of antibiotic resistance.

While academics and pharmaceutical companies work on the research, people like me are working on smoothing the road to U.S. Food and Drug Administration (FDA) approval for these novel therapies. As a postdoctoral researcher, I focused on how bacteriophages can kill certain bacteria. As a senior regulatory scientist, I work on the practical steps required to bring pharmaceutical products, including phage therapy, into the market to treat patients. Currently, the path to develop phage therapy through to regulatory approval is unclear.

In the face of antibiotic resistance in the U.S. and around the world, it is important to understand our alternatives and what must be done to advance alternative treatments.

Webinar: Tips for a Smooth NDA Submission

Top Trends in Drug Development from This Year’s DIA Annual Meeting

Posted by Brook White on Tue, Jul 12, 2016 @ 02:41 PM

During the last week of June, the Drug Information Association held its 52nd Annual Meeting in Philadelphia.  As one of the largest conferences in our industry, DIA covers a wide range of topics over the entire spectrum of drug development, and it would be nearly impossible to provide a comprehensive accounting of the meeting.  However, I will try to share the most notable trends and themes from the meeting.

Big Data

big data in drug developmentBig data was possibly the hottest topic this year.  Not only did FDA Commissioner Dr. Robert Califf participate in a panel session on the topic, but when listing his top four priorities, one of them was greater use of existing data in EMR/EHR systems.  At DIA, people were really talking about big data from a handful of sources—electronic medical records data, data from wearables, data from social and digital media and genomics (and other -omics) data.  FDA is taking a lead role in the use of big data and real world evidence through initiatives like Sentinel, which enhances the FDA’s ability to proactively monitor the safety of medical products on the market, and precisionFDA, a community platform for next generation sequencing (NGS) assay evaluation and regulatory science exploration.  Big Data is an idea that has been talked about for some time, but based on this year’s meeting it is clear we’ve moved beyond idea to reality.  For anyone wondering how soon we might see full genomic sequencing of all patients in a clinical trial, you will be interested to learn that the cost is now on par with a chest x-ray, Genentech has sequenced 30K genomes to date, and AstraZeneca recently entered into a partnership with Human Longevity to sequence 500K genomes over the next 10 years.

Patient Centricity is Still Big

patient-centricityPatient centricity was the theme of last year’s meeting, and continued to play a central role in this year’s meeting.  But while last year was big on ideas and optimism, this year saw early adopters sharing lessons learned from programs already up and running.  Patients and patient advocacy groups made up a noticeable group of attendees and were outspoken during sessions.  Several companies including Bristol Myers Squibb (BMS) and GlaxoSmithKline (GSK) shared specific programs and tactics they’ve been using to move to a more patient focused research model.  Some examples include creating frameworks that allow greater number of employees to engage with patients and the public about the work they are doing and developing minimum standards for patient engagement that reflect geographic and cultural differences.  From a regulatory perspective, patient-centricity made Dr. Califf’s list of his top four priorities.

7 Tips to Use Social and Digital Media to Recruit and Engage with Clinical  Trial Patients

The Swinging Pendulum on Outsourcing

swinging pendulum on outsourcingFor many years now, it seemed the trend was always to more and more outsourcing with innovator companies keeping fewer and fewer activities in house.  Several of this year’s outsourcing sessions are hinting that the pendulum on that trend may be starting to swing back.  From internal frustrations with outsourcing groups, to dissatisfaction with vendors in terms of both quality and performance, to the failure of preferred provider relationships to deliver on expected savings and improvements, the talk from a number of pharmaceutical and biotech companies is that they are keeping more work in-house. That said, there certainly is not agreement among sponsors or vendors/suppliers on this issue.  Many pointed to issues at sponsor companies such as refusal to hear feedback from CROs on the feasibility of their budgets, timelines, or study designs as well and disagreement between outsourcing personnel and study team personnel about the providers being selected.

Drug Development as a Calling

DIA opened with keynote speaker Dr. Larry Brilliant, a physician and epidemiology who participated in the World Health Organization’s (WHO) successful small pox eradication program.  Dr. Brilliant talked through a number of health research and outreach efforts that have dramatically changed the world for the better, including the small pox and polio eradication programs, the development of electrolyte solutions to treat cholera and diarrhea, and more recently the efforts of the Carter Center to eradicate guinea worm.  He brought into sharp focus the idea that what each of us in the pharmaceutical industry does has the potential to change the world for the better.  The idea of drug development as a calling was furthered by Dr. Califf’s call to all of us to donate the information in our electronic health records for the betterment of research and medicine—a reminder that we should be willing to open ourselves up in the same way that we ask patients and research participants to do.  Finally, several of the patient-centricity speakers focused on the value of identifying employees who themselves were patients or care-takers of patients in their private lives in addition to being part of the research and development process.  These people are uniquely qualified to help us better understand the patients’ needs and experiences.

Greater Engagement by FDA

Finally, it was interesting to me to see the level of participation by the FDA in this year’s meeting. While they always send some presenters and a larger number come just to attend, this year did seem different. Dr. Califf presented in multiple sessions and was open and engaging during Q&A sessions. Additionally, numerous sessions included speakers and panelists from the FDA providing valuable insight into their point of view.

Did you attend DIA this year? If so, let me know what you thought.

Protocol Design and Development Webinar: Follow-up Q&A

Posted by Brook White on Thu, Feb 04, 2016 @ 11:07 AM

Thank you to everyone who attended our recent webinar on protocol design and development.  During the webinar, we weren't able to get to all of the questions.  Below, Dr. Shoemaker and Dr. Kesler have answered the remainder of the questions.

If you didn't have an opportunity to attend the webinar, it is now available on demand.  

Watch Webinar

Why do you think the adoption of the PRM has been so long in the coming?

The Pharmaceutical industry is nototiously slow to adopt novel techniques due to the siloed structure and because the current protocol development process has been in place for decades. Not until the current protocol authors understand the concept of CDISC and the importance of generating consistent data across their program will their methods change. That will only happen if protocol authors are responsible for writing marketing applications.

What are the major consequences of redundancy in the protocol?

Inefficiency due to the need for redundant editing to ensure replacement of all instances and ultimately the cost of amendments if the redundant information is not edited correctly.

How long does it take to properly develop a clinical protocol?

Given adequate time to develop a novel protocol for a new indication with a new molecular entity depends on coordinating the time of all the people whose input is required. Depending upon peoples' priorities and availability it typically takes between one and two months.

If I am developing my drug as an add-on to an approved drug, why not conduct Phase I in patients (not healthy volunteers) taking stable doses of the approved drug? I want to know the safety of a range of doses of study drug when so administered. Pros/cons

Pros are that you save time and money with this approach. Cons are that you won't know if a safety event is due to your product, the approved product, or the combination. You also won't know whether the patients' compromised condition contributed in any way to the safety event.

It is said that no amount of good monitoring can fix a bad protocol. Do you have an example of such a situation and what should the monitoring team look out for to avoid such a situation?

By the time the monitoring team starts reviewing the data at the site or in house it is too late, the die has already been cast by the design of the clinical study. The monitors should endeavor to participate in protocol design to assist in mistakes made at this stage. Otherwise they can only make recommendations to amend the protocol if they see the data being generated is not answering the intended objectives of the study.

Is it advisable to write into the protocol the duration of acceptable periods during which study drug may be suspended without automatically discontinuing the subject?

If your study drug planned to be titrated within subject (e.g. some hypertension drugs) then it is advisable to have not only a duration of suspension, but also dose escalation/de-escalation processes as well. For other situations where study drug is being suspended due to concomitant events, like hospitalization, it is also advisable to have windows for the duration of acceptable suspension. If you don't have expected reasons for suspension and don't expect it to happen often, then it is probably a level of detail you don't need.

Do you have any template?

Yes we have an internal protocol template that we provide to all our clients developing protocols.

Please remind us what data we need to provide for you to determine a sample size for a clinical trial.

It depends on the type of primary outcome. If it is dichotomous, you need to provide the expected percent responding in both the active and control arms. If it is continuous, you'll need to provide the expected mean and variance (or standard deviation) for each group, or the expected difference in means. Other types of outcomes (e.g. survival, multiple categories) require additional information. All studies need a Type I level (alpha) specified as well as the desired power of the study. Estimates of the rate of dropout are also needed for most studies.

We will be conducting another webinar on Thursday March 17th at 1 PM ET on Clinical Research Statistics for Non-statisticians.  We will go into more depth about sample size calculations during that webinar.

Register Now

Is this protocol process impacted if/when combo solutions are involved? Combo is defined as drug/sensor based, or subcutaneous drug-illuting solutions.

Not really. Obviously you have to understand the combination product and its properties to the same extent that you understand your drug from a nonclinical and manufacturing properties perspective.

When is unblinded medical review warranted in Phase 2 studies?

There is a new guidance from FDA as of December 2015 advocating the use of a Safety Assessment Committee to review unblinded data from the totality of the data on your product and this should be implemented with the advent of controlled studies in Phase 2.

When can multiple repeat dose safety study be done with parallel dosing of multiple dose groups?

Never. Parallel dosing of multiple dose groups can be done for efficacy comparisons after safety has been demonstrated.

What is the proper endpoint for oncology trial now? it is overall response, tumor shrink, survival or quality of life?

It depends on the type of tumor being studied, but overall response is the preferred SURROGATE clinical endpoint in most cases for accelerated approval with follow-up measurement of survival used to validate this SURROGATE clinical endpoint. Quality of Life is usually montored with one of several patient reported outcomes (PROs) as a secondary clnical endpoint.

You mentiond that CDISC was advising avoidance of the use of "Day 0" terminology to describe intervention date and that this would be required after a certain date. Can you please restate when this goes into effect?

Trials started after December 2016.

Do you know of any company which can offfer to write protocol for their product?

Rho provides protocol design and development services. You can learn more on our website or by contacting us.

Check out our other on-demand and upcoming webinars here.

David Shoemaker, SVP R&DDavid Shoemaker, Ph.D.
Senior Vice President R&D

Dr. David Shoemaker has more than 25 years of experience in research and pharmaceutical development.  He has served as a Program Leader or Advisor for multi-disciplinary program teams and has been involved with products at all stages of the development process. Dr. Shoemaker has managed the regulatory strategy for programs involving multiple therapeutic areas, including hematology, oncology, cardiology, pulmonology, infectious diseases, genetic enzyme deficiencies, antitoxins, and anti-bioterrorism agents.  He has extensive experience in the preparation and filing of all types of regulatory submissions including primary responsibility for four BLAs and three NDAs.  He has managed or contributed to more than two dozen NDAs, BLAs, and MAAs.  Dr. Shoemaker has moderated dozens of regulatory authority meetings for all stages of development.  His primary areas of expertise include clinical study design and regulatory strategy for development of novel drug and biological products.

Karen-1.jpgKaren Kesler, Ph.D.
Assistant Vice President Operations

Dr. Karen Kesler earned both a Master’s and Doctoral degree in Biostatistics from the University of North Carolina at Chapel Hill and has over 20 years of experience in the industry.  Dr. Kesler currently serves as the Primary Investigator of the Statistics and Data Management Center for a NIH sponsored coordinating center researching asthma, allergies, autoimmune disorders, and solid organ transplant.  Dr. Kesler is deeply involved in researching more efficient Phase II and III trials and has led many adaptive studies including sample size recalculations, pruning designs, Bayesian dose escalation studies, and adaptive randomizations.  She has given numerous professional presentations and has over 25 publications and manuscripts to her credit.

4 Regulatory Considerations for Using Social Media in Clinical Trial Patient Recruitment

Posted by Brook White on Thu, May 21, 2015 @ 10:56 AM

kristin-headshotKristen Snipes, Project Director

Social and digital media can be powerful tools for recruiting clinical trial participants and speeding up enrollment; however, there are several regulatory as well as operational considerations you need to keep in mind.  Here are some important ones:

IRB Approval

IRB approvalSocial media and digital media can have a much more informal feel than traditional advertising venues, like a newspaper or radio ad. While most of us would ask someone else to review anything we wrote for a newspaper, most of us don’t ask anyone before we post something on Facebook or Twitter. From a regulatory perspective, however, these two activities are the same if they are being used for patient recruitment. If it is visible by potential patients, you should assume it will need IRB approval and plan accordingly.

Depending on the type and format of social and digital media you are using, you will need to give some thought to how you will present the information to an IRB. For example, if you are promoting a study through Google AdWords, you will need to be able to provide the text of the ad and a static view of the page to which the ad will direct traffic. For a Facebook ad, you will need to include the text used as well as any images and links that are included. A web page will require a visual of the page layout, text included as well as any links provided to the subjects.

Sponsor Perceptions and Approval

Make sure the study sponsor is on board both with what information will be shared in the recruitment process and how it will be shared.  Social media potentially has a very broad audience and once you’ve posted something you have much less controlled with how it will be shared and distributed than you do with traditional media.  Sponsors may be less comfortable with providing detailed study information through social media channels than they would, for example, a poster designed for doctors’ offices.  The information posted on Clinicaltrials.gov is a good starting point as it will contain information the sponsor is comfortable sharing externally. Use of web pages and other social media can raise questions about the ownership of data and the restriction of access in a public domain.

Use Caution with Patient Provided Information

warningIf you social or digital media plans include collection of patient information (even if there is a potential that patients will submit data), you need to be particularly careful.  Two important questions to ask are what information are you collecting and how will you use the information.  Avoid collecting anything subject to HIPAA and be careful that nothing they submit would require informed consent.  In some cases, you may find that a process that is less efficient is still the way to go.  For example, rather than collecting information through a web form that is then submitted to sites, you may want to direct the patient to contact a site directly or use a call center with appropriately trained personnel.  This plan will also require review and approval by IRBs.

Picking an Agency

Sometimes CROs and sponsors don’t have the necessary expertise in-house to set up a social or digital media campaign, so it makes sense to outsource some of these activities.  If this is the case, you will likely be better off using a recruitment vendor that offers social and digital media services than a traditional ad agency.  If you do go with an ad agency, make sure they have experience working in regulated industries.

Free Webinar: Selecting Inclusion/Exclusion Criteria for Your Next Trial

Late-Stage Biostatistics Submission Services

Posted by Brook White on Tue, Apr 21, 2015 @ 09:52 AM

Rob Woolson, Chief Strategist-Biostatistics and Data Standards for Regulatory SubmissionsRob Woolson, Chief Strategist-Biostatistics and Data Standards for Regulatory Submissions,  has led SDTM/ADaM dataset conversion projects in multiple therapeutic areas. He has held a leadership role in six CDISC-compliant regulatory submissions, having guided the creation of ISS/ISE statistical analysis plans; integrated analysis dataset design and production; integrated display design and production; and submission-related documentation development. 

Bridging the Gap between Technical/Data Standards and Regulatory/Medical Writing in NDA Submissions

Many CROs currently offer two categories of services to sponsor companies preparing for an NDA or marketing application submission. The first category contains technical data standards services. These services include converting existing clinical datasets to CDISC compliant formats and generating the datasets for the integrated summary of safety (ISS) and integrated summary of efficacy (ISE). The second category includes regulatory and medical writing services associated with the marketing application submission.

There are a number of activities that need to take place that fall in between these two categories to ensure a successful marketing application submission. While some CROs offer these services on an ad-hoc basis, and some sponsors have the internal expertise and capacity to fill these gaps, there remains an unmet need to define and consistently deliver these services. We see these late-stage biostatistics submission services falling into two sub-categories—marketing application services and regulatory interaction support.

Marketing applications services include activities like analysis plan development; data standards plan development; supporting regulatory and medical writing during the submission process; pooled data assessment; and analyses to support key messages. For example, an analysis development plan would help a sponsor devise an analysis strategy to support the key messages in their submission, such as:

  • Defining the safety parameters of interest based on FDA expectations, a priori special interests/concerns, and safety concerns identified during data review.
  • Determining an analysis and presentation approach that gets at the true value of parameters of interest and allows for meaningful comparisons between the active and placebo/comparator groups.
  • Developing a strategy for combining and pooling data.  (For example, while you have a regulatory obligation to present all safety data, there is no requirement to pool all studies for analysis into a single group.)
  • Determining which subgroups should be explored for consistency of effect. 

Regulatory interaction support includes providing biostatistics input and analysis for both face-to-face meetings and written communications with FDA. In the period leading up to your submission and following the submission, it is likely you will have increasing interaction with FDA. These services prepare you to have the biostatistics and data standards expertise on-hand that you’ll need to effectively respond to FDA communications, and to recommend when you should initiate conversations with FDA over your data.

So, as you approach the end of your development program, make sure you have the statistical support in place that you need.

Watch Now: Understanding the New FDA Guidance on Data Standards Webinar

Are Your Data Submission Ready? 5 Questions You Should Be Asking

Posted by Brook White on Thu, Apr 09, 2015 @ 09:41 AM
Rob Woolson, Biostatistics and Data Standards for Regulatory SubmissionsRob Woolson, Chief Strategist-Biostatistics and Data Standards for Regulatory Submissions,  has led SDTM/ADaM dataset conversion projects in multiple therapeutic areas. He has held a leadership role in six CDISC-compliant regulatory submissions, having guided the creation of ISS/ISE statistical analysis plans; integrated analysis dataset design and production; integrated display design and production; and submission-related documentation development. 

In December 2014 the Food and Drug Administration released two final guidance documents and one technical conformance guide related to the content and format of electronic submissions.  These documents require the electronic submission of standardized non-clinical and clinical study data for nearly all submission to CDER and CBER, including INDs, NDAs, ANDAs, and BLAs by 2016.  In other words, study data must comply with a number of standards enumerated by the FDA, most notably CDISC data standards (for example, SDTM and ADaM).  These guidance documents are binding.  Submissions that do not follow technical data conformance standards will not be filed or received by the FDA, meaning if you don’t comply you could receive a refuse to file letter.  So, what do you need to do?  Ask yourself these five questions.

How much of my data currently conforms to these standards?

It may be the case that you’ve been working with CDISC standards from the start of development.  All of your clinical datasets are formatted to SDTM and ADaM formats and you plan to create your Integrated Summary of Safety (ISS) and Integrated Summary of Efficacy (ISE) datasets to conform as well.  It may also be the case that you know that none of your existing data conforms and that extensive remediation will be necessary.  In our experience, however, many situations are more complex.  For example,  some late stage development work may conform to CDISC standards with much of the earlier development work following a legacy standard (or no standard at all).  The first step is to figure out what you have.

For the datasets that don’t conform, do they all need to be converted to CDISC standards?

It is likely that if you are preparing for a submission now, at least some of your datasets will need to conform to CDISC standards.  However, that doesn’t necessarily mean that all of your legacy datasets will need to be converted.  By preparing a Data Standards Plan and submitting it to the FDA, you may be able to limit the amount of conversion and remediation work that is done, saving both time and money.  Getting feedback from the FDA at this stage is critical, so that there won’t be any surprises when you file.

Do I have the necessary expertise and capacity in house to make these assessments, perform needed remediation, and get the data submitted?

Moving quickly at this stage in development is critical.  Everyone wants to get their product to market as soon as possible, so once the final studies are finished you want the data submission work to be fast and accurate.  If you don’t have the internal capacity to move at the rate you want it may make sense to bring in an outside vendor.  Data submissions are also a pretty specialized area, and many companies may not have the internal expertise required.  This can be another reason you should consider an outside vendor.  

If I need a vendor to help with these activities, what should I be looking for?

It’s likely that you’ve used one or more CROs at some stage of development, but the CRO that performed your clinical studies may or may not be the best choice when it comes to getting your data ready for submission.  Here are some things to consider:

  • Experience
    • How many protocols have they mapped to SDTM or ADaM? Were any of them for products in the same or similar indication as yours?
    • How many marketing applications have they filed in the last five years?
    • What is their technical acceptance rate? Have any of their submissions resulted in a refusal to file?
    • What level of experience do the individuals assigned to your project have? You should expect a greater level of experience for statisticians and data standards staff working on a marketing application submission than you would need for working on a typical single study.
    • Does the vendor have experience communicating directly with the FDA on data standards plans?
    • Which standards do they have experience with? At a minimum, we recommend:
      • CDASH
      • SDTM
      • ADaM
      • define-XML
      • CDISC controlled terminology
    • Which version of the standards have they used?  Are they familiar with the versions you are using and plan to use?  
  • Tools and Technology
    • What tools are they using to assess compliance? Are they the same as those in use at FDA?
    • Do they have tools for metadata mapping, creating define.xml files, and performing QC on clinical databases?

What Types of Services Can I Expect a Vendor to Provide?

A high quality vendor should be able to look at your situation and make recommendations about the services you need.  Typical services include:

  • Assessing and reporting on the current state of clinical databases relative to external and FDA standards and requirements. If you don’t already have one, they should be able to provide a plan for preparing databases for submission.
  • Performing any needed data remediation.
  • Preparing and submitting a Data Standards Plan to the FDA.
  • Executing a test data transfer to obtain FDA reviewer feedback.

Be wary of vendors that provide a set package of services.  Each submission is unique, and these packages may include costly and time consuming services you don’t really need.

Watch Now: Understanding the New FDA Guidance on Data Standards Webinar

Objectives and Considerations in Developing an Integrated Summary of Efficacy (ISE)

Posted by Brook White on Mon, Jan 13, 2014 @ 01:25 PM

Ben Vaughn, Integrated Summary of Efficacy (ISE)Ben Vaughn is a statistical scientist who has participated in nearly 20 regulatory submissions and is an expert on CDISC implementation and standards. His work has included serving as lead statistician to complete displays and datasets for several ISS/ISEs (as well as the associated pivotal studies); coproducing the ISE for two opioid products; and statistical consultation, display generation and submission work for four separate products for OA knee pain.  He has authored responses to various FDA queries regarding NDAs, PMAs, IDEs, and SPAs. Additionally, he has represented sponsors in FDA teleconferences and face-to-face meetings for both OA knee pain products and opioid products.

A new drug application (NDA) covers information about a product from inception through clinical trials.  The integrated summary of efficacy (ISE) is a section of the NDA that provides a focused summary of efficacy data primarily from the pivotal phase 3 trials.  This article discusses the primary goals and considerations for developing an ISE.


The primary objective of the ISE is to orient the reviewer to the key measures of efficacydata analysis, integrated summary of efficacy (ISE) presented in the pivotal studies and the associated results.

Beyond the information available in the individual study results, the ISE analyses should also:

  1. Obtain a precise estimate of the primary endpoint.
  2. Obtain precise estimates of key secondary endpoints which provide supporting evidence for the primary endpoint.
  3. Conduct key a priori subgroup investigations to examine consistency of effect.


Here are a several considerations for developing the ISE:

  • The ISE should rely on results from the pivotal phase 3 studies. It is possible to include phase 2 results if the primary endpoint is the same as for the pivotal study, but this can be risky. Because phase 3 studies are typically designed by using the phase 2 study with the best outcome (or the best endpoint from a given phase 2), this results in an inflated alpha (alpha, also known as type 1 error, is the likelihood that the study shows a difference between active and control when in reality there is no difference). Therefore, it may not be legitimate to include the phase 2 results with the phase 3 results.

  • Focus only on primary endpoints for which there was tight alpha control in the pivotal trials. 

  • In the best case scenario, all phase 3 trials had the same primary endpoint and collected and analyzed it in the same way, however, this isn’t always the case. Often, lessons learned in earlier studies are applied to later studies, so there are minor differences. It is acceptable to look back at the data collected in the earlier studies and analyze it in the same way as was done in later studies- as long as you can still show alpha control was maintained and understand that the reviewer’s assessment of success will largely be based on the pre-specified analysis.

  • It is not helpful to perform many additional analyses that are different than those planned for the pivotal trials in hopes of creating supporting evidence. FDA reviewers are keenly aware that the more analyses are run, the more likely it is to find some significance and will not find that strategy acceptable. They are likely to accept a change in model or analysis when the move is from a significant result with a flawed analysis to another significant result with a better analysis. Anytime there is move from an original analysis with an insignificant result to a new analysis that shows a significant result, reviewers will object.

  • Include only one or two secondary endpoints. A laundry list of secondary endpoints will not persuade FDA reviewers. Secondary endpoints should be clinically relevant and should support the story told by the primary endpoint.

  • Conduct analyses of therapeutic response in subgroups of interest. Heterogeneity of response will be of interest to reviewers and it is better to identify any instances of this in ISE rather than leave it to a reviewer to discover. However, subgroup investigations and the interpretations of these results should be done with some caution. Sample sizes are often insufficient to get stable results; therefore, focus on subgroups with sufficient sample size and a scientifically plausible rationale that they would have a meaningful difference in response.

  • Including everything is not OK: While there is a tendency to want to include any possible tidbit that may look positive for the product, be aware that it will not improve chances of approval. It is far better to have an efficacy summary that tells a clear and concise tale.

FDA reviewers are less likely to spend as much time looking at the ISE as they do the integrated summary of safety (ISS) and the results from the individual phase 3 trials.  Use the ISE to make a clear, concise, and focused case that the product is efficacious.

Download Accelerated Regulatory Approvals Whitepaper

3 Key Questions When Developing the Integrated Summary of Safety (ISS)

Posted by Brook White on Tue, Jan 07, 2014 @ 02:01 PM

Rob Woolson, Integrated Summary of SafetyRob Woolson is a senior biostatistician with 12 years’ experience in the analysis of complex data. He has conducted statistical analyses in all phases of drug development (Phase I through IV, including NDAs) and has led SDTM/ADaM dataset conversion projects in multiple therapeutic areas.  He has held a leadership role in six CDISC-compliant FDA submissions, having guided the creation of ISS/ISE statistical analysis plans; integrated analysis dataset design and production; integrated display design and production; and submission-related documentation development.

A new drug application (NDA) covers information about a product from inception through clinical trials.  The integrated summary of safety (ISS) is a section of the NDA that provides comprehensive safety information collected throughout the development program.  The goal of the ISS is to characterize the overall safety profile of the drug and to identify risks that should be included on the product label.  This article discusses three key questions to address as a part of your ISS analysis plan.

(1) What are the safety parameters of interest?

Safety parameters of interest typically include those specified in FDA guidance, those that are a priori special interest or concern for the program or compound, and those identified during data review. Some examples of safety parameters are exposure, concomitant medications, deaths, adverse experiences (occurrence, relatedness, severity, seriousness, duration, timing, etc.), laboratory measures, and vital signs. A good test of whether one has selected the right parameters is to ask whether the summary of the estimates of these parameters sufficiently describes the overall drug safety profile.

keys to ISS(2) How does one present and analyze data to convey key safety messages and describe the overall safety profile?

Once safety parameters have been selected, one must decide the best way to present them. Some guiding principles we’ve come up with are:

  • The presentation should help a reviewer get at the true value of the parameter of interest.

  • The parameter should be presented with a high degree of confidence (maximum precision, minimum bias).

  • The presentation should permit the reviewer to make meaningful comparisons between active and placebo. An absolute means little unless it is compared to something.

There are a number of ways we can characterize these parameters including proportions (i.e., a crude rate), incidence rate (per unit time), total incidence rate (events per unit time which may be useful where there are multiple events per subject and different exposure times), time to event, and change from baseline.  One challenge frequently encountered in presenting safety data is that different trials have collected data differently which lead to additional complexity in how data should be presented in the ISS.  For example, one may have to deal with several different follow up times.  The solutions selected must convince a reviewer that they are provided an unbiased presentation of the data.  There are several methods that can be used for making between group comparisons.  These methods include difference of proportions, ratio of proportions (mentioned in FDA guidance), difference in rates, ratio of rates, hazard ratio, survival curves, and difference in means.

(3) Should safety data from all studies be pooled and, if so, how?

One of the first things to consider in characterizing the information is if and how data will be pooled. There is a regulatory obligation to present all safety data; however, there is no requirement that data from all studies be pooled.

It is important that whatever pooling strategy is taken, one is prepared to justify it to FDA reviewers and/or an advisory committee.  

  • A reason to pool data is that one may be able to provide more precise and more reliable estimates of safety parameters. Also, pooled data may allow conclusions to be drawn (e.g., comorbidities) that wouldn’t be seen by looking at the studies individually. 

  • A reason not to pool data is that studies and the populations in those studies may be so different that it is difficult to make sensible comparisons. Additionally, analyses of pooled data can be time consuming and expensive when compared to summarizing and presenting the analyses that were performed as part of the studies. (Going too far down that path, however, may lead reviewers to believe the analyses are insufficient.)

In most cases, it will make sense to pool at least some data.  Some general principles for creating a pooling strategy:

  • Combine the data in the most valid way(s)

  • The safety message should drive the pooling strategy
  • Summaries should produce transparent results

  • No masking of safety signals
  • Estimates produced should be as unbiased as possible

Some factors to consider when looking at individual protocols to determine whether it makes sense to pool data from those protocols are:

  • Design similarity

  • Doses studied

  • Duration

  • Controlled/uncontrolled/choice of control

  • Region

  • Population

Keep in mind that differences in these factors introduce variation to the safety parameters of interest.

In conclusion, it is important that the finished ISS tells FDA reviewers a clear and focused story about the drug’s safety profile.  Doing this effectively is necessary to move through the approval process efficiently.

Register for a free CDISC webinar

Key Tips for Orphan Product Development

Posted by Brook White on Tue, Aug 27, 2013 @ 10:47 AM

David Shoemaker discusses orphan product developmentInformation for this article was contributed by David Shoemaker, Senior Vice President R&D. Dr.  Shoemaker has over 25 years of experience in research and pharmaceutical development.  He has managed or contributed to dozens of INDs/CTAs and over a dozen successful NDAs, BLAs, and MAAs.  Dr. Shoemaker has authored or overseen dozens of Orphan Drug Designation applications, has developed several successful Accelerated Approval programs, and has secured several Priority Review applications.

Selecting a partner for drug development is tricky.  This is especially true when selecting a CRO to assist with orphan product development.  Finding a partner that has both the experience and expertise needed as well as being a good cultural fit for your company is critical to achieving your goals. 

  1. Work with CROs that have strong scientific, regulatory, and statistical expertise
    A strategic approach with a focus on key milestones is critical to gain approval as quickly as possible. Look for CROs whose strengths include the ability to conduct challenging clinical trials, knowledge of the regulatory process, and scientific and statistical expertise to develop a plan for success at the outset to reach approval in an expedited speedy manner. Your CRO should have successfully obtained marketing approval for other orphan products previously. Marketing applications for orphan products require creative regulatory and statistical strategies to leverage the data obtained on populations much smaller than typically seen by regulators.
  2. Know the “ins and outs” of the U.S. Food and Drug Administration’s approval mechanisms to help speed orphan drug approval
    Many orphan diseases represent serious or life-threatening conditions. Consequently, working with a development partner that understands each of the accelerated development pathways (i.e., Accelerated Approval, Priority Review, Breakthrough Therapy, and Fast Track) and the potential benefits or lack thereof is critical. Making an informed decision on the best mechanism at the start of the orphan drug approval process is the fastest path to approval.
  3. Apply for US and European Orphan Drug Designation Simultaneously
    There is a combined form that can be used to obtain orphan drug status simultaneously in the US and EU. It is an option that is not being used broadly, but can result in significant reduction of time and effort.
  4. Look for a CRO partner with experience working in small patient populations 
    Working with small patient populations requires building communities and developing close connections with research foundations, advocacy groups, patients and health care providers for a purpose-driven approach to product development. It will also be important to gain buy-in from Key Opinion Leaders.
  5. Validate your population
    Before investing time and energy in an orphan drug application, make sure you are eligible. Regulators are on the lookout for developers who try to “slice the salami” meaning that your orphan population is really just a subset of a larger population from which there is no substantive difference.

Pharmaceutical and biotechnology companies can accelerate successful development of orphan products by partnering with product development service providers with a culture of solving challenges and the scientific and regulatory expertise to navigate complex trials and approval processes.

The National Organization for Rare Disorders reports nearly 7,000 orphan diseases affecting nearly 30 million Americans. As more drug companies search for new approaches after mass-market drug revenues are lost to generic competition, orphan drug development is gaining momentum.

At Rho, we share a passion for discovering new treatments and have experience successfully helping companies navigate the FDA’s orphan product approval processes. But just like anything that sounds too good to be true, sound product development program decisions should stem from a keen understanding of the requirements and potential benefits of each approach. Selecting the right product development services partner can help deliver new treatments to improve and save lives as quickly as possible.

Click Here to View a Slideshow on Key Tips for Orphan Products