Rho site logo

Rho Knows Clinical Research Services

Patient Reported Outcomes and the Common Terminology Criteria for Adverse Events (CTCAE)

Posted by Brook White on Tue, Jan 08, 2019 @ 09:44 AM

Kristen Mason, MS Senior BiostatisticianKristen Mason, MS, is a Senior Biostatistician at Rho. She has over 4 years of experience providing statistical support for studies conducted under the Immune Tolerance Network (ITN) and Clinical Trials in Organ Transplantation (CTOT). She has a particular interest in data visualization, especially creating visualizations within SAS using the graph template language (GTL). 

The  Common Terminology Criteria for Adverse Events (CTCAE) was developed to standardize reporting of adverse events (AEs). Originally developed for oncology trials by the National Cancer Institute (NCI), the CTCAE is now used widely across all types of clinical trials, and as a result has become an essential tool for evaluating trial safety. The CTCAE provides a library of over 800 AEs with guidelines for grading event severity. However, it fails to capture the day to day effects a patient might experience. In order to integrate the patient’s perspective into AE reporting, the NCI developed a patient-reported outcomes version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE). 

What is PRO-CTCAE?

The PRO-CTCAE consists of 78 AEs with questions regarding the presence/absence, severity, frequency, and/or interference of symptomatic treatment side effects. These AEs were identified as symptoms that can be meaningfully reported by the patient, such as pain, fatigue, and nausea. Responses are provided on a 5-point Likert scale (generic form shown below) and refer to the patient’s experience in the last seven days. The tool is currently available for public use (https://healthcaredelivery.cancer.gov/pro-ctcae) and can be modified to reflect relevant toxicities of interest. The PRO-CTCAE has been translated into 18 languages with more on the way. Finally, the PRO-CTCAE is not intended to replace, but rather enhance, the CTCAE with the patient’s perspective, and therefore these should be used together.


How do the perspectives of the clinician and the patient differ?

Clinicians are often primarily concerned with patient safety, especially in a clinical trial setting. On the other hand, patients have to deal with the reality of managing symptoms that impact their daily life. Clinicians are also frequently exposed to patients with very serious toxicities, which may bias their perception. Furthermore, while an event may not seem serious to the clinician, it could interfere with the patient’s daily activities. Finally, patients may under-report symptoms for a variety of reasons, which could include time constraints of the visit, concerns about discontinuing treatment, or even being removed from the study entirely.  

What else can be learned?

Another limitation of the CTCAE is that typically only the maximum severity of the event is reported. In other words, any potential change in AE severity cannot be detected. But, PRO-CTCAE can help answer these questions, such as: did the AE develop slowly or all at once? How long did the patient experience the worst of it? 

What are the next steps?

The PRO-CTCAE is a work in progress and continues to be improved. Presently the tool is only intended to assess adults; however, another version is being developed for children and adolescents. Additionally, guidelines will need to be created that provide a standard approach for analyzing this new patient-focused data. 

Webinar: ePRO and Smart Devices

Age Diversity in Clinical Trials: Addressing the Unmet Need

Posted by Brook White on Tue, Jul 10, 2018 @ 09:23 AM

Ryan2Ryan Bailey, MA is a Senior Clinical Researcher at Rho.  He has over 10 years of experience conducting multicenter asthma research studies, including the Inner City Asthma Consortium (ICAC) and the Community Healthcare for Asthma Management and Prevention of Symptoms (CHAMPS) project. Ryan also coordinates Rho’s Center for Applied Data Visualization, which develops novel data visualizations and statistical graphics for use in clinical trials.

elderly patient with nurseIn a recent New York Times article, Paula Span raises the concern that elderly subjects are frequently omitted from clinical trials.  Consequently, physicians know very little about how a given treatment may affect their older patients.  Is a medication effective for the elderly?  Is it safe?  Without data, how is a physician to know?  

Span’s article is timely and aligns well with similar industry trends toward increased patient centricity and trial diversity.  Yet, expanding trials to include older patients poses a challenge for research teams because it brings two tenets of quality research into conflict with one another – representative study populations and patient safety.  

The fundamental assumption of clinical trials research is that we can take data from a relatively small, representative selection of subjects and generalize the results to the larger patient population.  If our sample is too constrained or poorly selected, we hinder the broad applicability of our results.  This is not merely a statistical concern, but an ethical one.  Unfortunately, our industry has long struggled with underrepresentation of important demographic groups, especially women, racial and ethnic minorities, and the elderly. 

At the same time, researchers are keenly concerned about protecting subject safety in trials.  Good Clinical Practice is explicit on this point: 

2.3 The rights, safety, and well-being of the trial subjects are the most important considerations and should prevail over interests of science and society.

Such guidance has engendered broad reluctance to conduct trials in what we deem “vulnerable populations,” namely children, pregnant, and the elderly.  The risk of doing more harm than good in these patient groups often leads us to play it safe and exclude these populations from trials.  Span, however, provides an astute counterpoint: expecting providers to prescribe a medication to a group of patients who were not included in the original research is equally irresponsible.  

No case illuminates the challenging catch-22 we face like the awful thalidomide debacle of the 1950s-60s.  Thalidomide, which was widely regarded as safe, was prescribed off-label for pregnant women to treat morning sickness.  Tragically, the drug was later linked to severe birth defects and banned for expecting mothers.

On one hand, the physicians prescribing thalidomide did so based on limited knowledge of the drug’s safety in pregnant women.  Had a trial had been conducted that demonstrated the risk to children, they would clearly know not to prescribe it to expecting mothers.  Yet, the very risk of such dangerous complications is why such trials are not conducted in vulnerable populations in the first place.  Risks for the elderly are different than for pregnant women, but the principal of protecting sensitive populations is the same.  

Span notes that even in studies that don’t have an explicit age cap, many protocols effectively bar elderly participants via strict exclusion criteria that prevent participation by people with disorders, disabilities, limited life expectancy, cognitive impairment, or those in nursing homes.  It must be stated, however, that the reason for such conditions is not to be obstinately exclusive but to reduce confounding variables and minimize risks to vulnerable patients.  In most cases, it would be patently unethical to conduct research on someone with cognitive impairment or in a nursing home where they may be unable to give adequate informed consent, or they may feel coerced to participate in order to continue receiving care.

So, how do we negotiate this apparent impasse?  Span offers a few general suggestions for increased inclusion, including restructuring studies and authorizing the FDA to require and incentivize the inclusion of older adults.  Changing the laws and enforcement can certainly drive change, but what can we do in the near term, short of legislative intervention?  

elderlycoupleA few quick suggestions:

  1. Reconsider age limits and avoid an all-or-none mentality to enrolling geriatric subjects.  The mindset that older adults are, as a whole, too vulnerable to enroll is usually an overreach.  In most cases, age limits are imposed as a convenience for the study, not a necessity.  Instead, consider evaluating eligibility on a subject-by-subject basis, which will still allow exclusion of patients deemed too frail, risky, or comorbid for the trial.  
  2. Actively recruit older subjects. The lack of geriatric patients in our trials is a result of many years of both passively and actively excluding them, so effort is needed to reverse these trends.  Beyond recruitment for an individual trial, researchers and providers should seek to educate older adults about clinical research.  Many elderly patients may be research-naïve – unfamiliar with clinical trials and how to participate, or unaware of available trials in their area.  
  3. Learn from other efforts to recruit marginalized populations.  As we’ve shared previously, improving trial diversity starts with an effort to thoroughly understand your patient population and their needs, and reduce obstacles to their participation.  
  4. Engage patient advocacy groups that focus on elderly patients.  Ask how trials can be better designed to meet their needs and include them.  Partner with these groups to aid in information sharing and outreach.
  5. Learn what is already expected from agencies like the FDA and NIH when it comes to inclusivity. 
    1. Span alludes to a recent NIH policy revision (stemming from the 21st Century Cures Act) that will require new NIH grantees to have a plan for including children and older adults in their research.
    2. In 2012, the Food and Drug Administration Safety and Innovation Act (FDASIA) required the FDA to create an action plan to improve data quality and completeness for demographic subgroups (sex, age, race, and ethnicity) in applications for medical products. 
  6. Design studies to examine effectiveness (demonstrating that a treatment produces desired results in ‘real world’ circumstances) not just efficacy (demonstrating that a treatment produces desired results in ideal conditions).  This is probably the most labor intensive because it requires additional investment beyond the typical Phase III randomized controlled clinical trial.  Yet, it is becoming more common to explore effectiveness through pragmatic trials, Phase IV studies, and post-market surveillance.   

Key Take-aways from the 2nd Annual Strategies in Patient Centered Clinical Research Conference

Posted by Brook White on Tue, Aug 01, 2017 @ 10:16 AM

In July, delegates from Rho, Shann Williams, David Cass, and Ryan Bailey, attended the 2nd Annual Strategies in Patient Centered Clinical Research conference in Boston.  They share their thoughts on the conference below.

Patient centricity is garnering broad attention in our industry, but finding time to thoroughly explore the principles and practices of the movement can be difficult.  Needless to say, we were excited to spend two full days immersed in discussion and collaboration with some of the industry pioneers in patient-centered clinical research.  Condensing all of the great content into a few paragraphs is impossible, but we wanted to share our take on the main themes and trends that stood out to us from the event.  

A Rapidly Evolving Movement

changes aheadWe are excited to see the patient centricity movement evolving and growing before our eyes.  We were fortunate to attend the inaugural conference in 2016, and the change from last year to this year was noteworthy.  Last year, much of the focus was on establishing the rationale for patient centricity, and convincing stakeholders that patient engagement was worthwhile.  This year, the presentations were much more focused on action and implementation.  The “why” and “what if” questions continued to permeate our conversations, but it’s encouraging to see that we’re not letting some uncertainty hinder progress.

Integrated Patient Engagement Specialists

integrationMany companies are prioritizing patient centricity by creating new positions and departments solely dedicated to patient engagement.  More importantly, these new patient engagement leaders are not merely figureheads, but integral corporate leaders with the authority to enact change at the business unit and study level.  Several presenters shared how their patient engagement teams are integrated into their R&D, study design, and protocol development efforts throughout the product lifecycle.  One presenter noted that, “every project coming to our protocol review committee will be asked if they considered patient insights when designing the trial.”

New Tools and Processes

With companies appointing patient-focused leaders and new departments, it is not surprising that many companies are now developing processes and tools to support better patient integration. The evolution of tools and procedural documentation provide a much needed structure and framework for moving patient centricity from concept to practice.
A sample of the tools mentioned:

  • Protocol Quality Assessment Tool with a patient engagement component
  • Patient Engagement Toolkit for Study Development
  • Unique Patient Plans for each therapeutic area
  • Redesigned Informed Consent Forms using professional designers and user interface experts
  • Simulation Labs to test drive the protocol with staff and patients before a trial launches   

Technology is a Tool, Not a Panacea


Several presenters discussed using emerging technologies to improve trials – e.g., using Uber and Lyft for transporting patients to visits, employing social media listening to determine patient needs, creating virtual, on-demand sites to conduct visits – but a prevailing theme was that technology alone will not achieve patient centricity.  One presenter lamented that in the past 35 years, pharmaceutical development has made no gains in time-to-approval or rate of approval, but cost has gone up an average of $3.0 billion.  Despite all the technological advances in the past three decades, none of the key performance indicators of our industry have improved.  We should leverage technology in our continuous improvement efforts, but not at the expense of investing in other strategies – like patient engagement – to reduce cost and time, and improve success rates. 

Metrics and Measurement

measurements and metricsYes, the dreaded “M” word.  From the outset, patient centricity has been hampered by a lack of data quantifying its benefit to trials.  Common sense arguments in favor of patient engagement – e.g., improved recruitment and retention, improved adherence, fewer protocol amendments – have propelled the movement, but the desire for metrics has not dwindled.  Capturing metrics takes time, and it can be difficult to determine the value of a set of broad and somewhat nebulous patient engagement strategies, but some groups are making headway.  Several presenters shared data reinforcing the value of patient centricity.


  • Internal review at a top-10 (annual revenue) Pharma company found that 26% of their protocol amendments were attributable to participant burden.  Another top-10 Pharma company reported seeing a year-over-year decrease in the number of protocol amendments after implementing patient-centered strategies.
  • Research between DIA and the Tufts Center for the Study of Drug Development found that the single strategy of engaging Patient Advocacy Groups in trial design, added 2-4 months of additional up front planning time, but ultimately saved millions of dollars in prevented protocol amendments and led to faster FDA approval.
  • Research out of the Clinical Trials Transformation Initiative (CTTI) found that improving the patient experience (e.g., user friendly informed consent, simpler eligibility criteria, reduced patient burden) could save 10s of millions of dollars in Phase II and Phase III trials. 


For an industry that is often criticized for moving too slowly and resisting change, it is inspiring to see the evolution in just one year’s time.  As companies move beyond concept and into implementation, we are eager to see how these changes will impact patients and improve our research.  

One of the quotes that stood out to us from the conference is “the biggest tragedy in clinical research is not a failed trial, it is a successful trial that fails in the real world because we didn’t design with the patient in mind.”  This serves as a poignant reminder that our work must be fundamentally focused on healing patients, not merely getting approval or getting to market, if we are to make a positive impact.

Webinar: Putting Patient Centered Principles Into Practice

Practical Strategies to Simplify Patient Centricity: Part 4—Practical and Easier than You Might Think

Posted by Brook White on Tue, Jun 20, 2017 @ 11:00 AM

Ryan Bailey, Senior Clinical ResearcherRyan Bailey, MA is a Senior Clinical Researcher at Rho.  He has over 10 years of experience conducting multicenter asthma research studies, including theInner City Asthma Consortium (ICAC) and the Community Healthcare for Asthma Management and Prevention of Symptoms (CHAMPS) project

This is the fourth in a series of blog posts on putting patient-centric principles into practice (view part 1,  part 2, or part 3)

In the previous post, we explored the rationale behind patient centricity and acknowledged the danger we face if we fail to pay heed to the rising tide of patient dissatisfaction with our trials.  In this post, we focus on the more positive and practical aspects of patient centricity, and how a patient-centered approach can improve our work and reduce our costs.

To start, I think it helps to step back and consider why we do what we do.  We are in the business of healing people, of eradicating painful, debilitating, life-taking diseases, and of giving people hope.  How do we do that?  

We rely entirely on the willing and voluntary participation of patients – the real heroes of clinical research.  When patients are in our trials, they put their body on the line, they give their time, they attend visits, they're poked and prodded, and they share blood and tissue and data with us so we can help heal others.  We have an ethical and moral imperative to do right by them.

superhero patient.jpgWhen it comes to ethical protections and patient safety and well-being, we have a number of invaluable guidances that steer our work – the Nuremberg Code, the Declaration of Helsinki, the Belmont Report, Good Clinical Practice, to name a few.  Unfortunately, our familiarity and history working with these seminal documents can sometimes lead to the misleading notion that we've "solved the patient rights stuff."  

However, we cannot be content to rest on our laurels.  We need to consider what lies ahead.  What is next on the continuum of evolving ethical milestones that define our work?  I believe patient centricity is a key aspect.  With patient centricity efforts, we are called to move past our historical approach of saying “we want to do this for patients" to saying “we want to do this with patients."  We change our traditional paradigm of “patient as a participant” to “patient as a partner” in the research space.  We invite patients to contribute their voice to what we're doing, to help us figure out what matters to them, and then we do it.

Of course, this leads to an inevitable question – how do we do that?  Answering the “how” question can seem daunting, especially given the apparent novelty of patient centricity as a movement.  A common impulse is to feel like this is “yet another” set of tasks to burden research teams.  Fortunately, “patient centricity” while relatively new as a term, is not new in concept. 

patient-centricity pyramid

Patient centricity is in our DNA as researchers. Yes, this is an "emerging trend,” but it didn't come out of nowhere.  It's a natural extension of a variety of disciplines and movements we care about – things like adherence, retention, patient advocacy, patient rights, patient-reported outcomes – all of which laid the groundwork for this.  Patient centricity isn’t so much a new trend, as it is a culmination of many different efforts that have been pushing us for years to better accommodate the needs and desires of the patients in our trials.

For example, we already know that patient engagement efforts lead to better recruitment and retention because research on these efforts shows the benefit of understanding and accommodating patient needs.  Likewise, research shows that engaged patients have better adherence and better health outcomes.

There are, rightfully, concerns about adding costs and time to our processes; but the good news is that the investments of patient centricity can prevent many of the “unforeseen” costs that commonly handicap our studies: delayed or insufficient recruitment, high dropout, and poor participant adherence.  What's even better is that research coming out of DIA and the Tufts Center for the Study of Drug Development (available to download here) shows that many of the simplest, most affordable, and lowest time-commitment investments promise some of the best returns.  They found that simple things like involving patient advocacy groups and patient advisory groups in study planning and development, and engaging patients on social media, can have some of the highest impact.

Consider also that patient centricity is a virtuous cycle.  Once you learn what works for a particular group of patients, you can build on that for future research and gain efficiency over time.  Likewise, as research sites and study teams develop reputations for good patient engagement, patients will be more likely to participate in future studies and advocate on your behalf to others in their social groups.

Patient centricity is still evolving as a movement, and you can expect to hear a lot on this topic in the years to come.  For now, the question is one of when to act.  Patient empowerment is happening in healthcare and patient dissatisfaction threatens our industry.  If you’ve experienced the pains of not being patient centric, through slow or stalled recruitment, poor retention, weak adherence, or disappointing patient-reported outcomes, taking a wait-and-see approach is a risky endeavor.  On the other hand, if you are motivated to adapt to the changing patient population, value continuous improvement, and want to see better return on investment, patient engagement is a good place to start.

To get you started, I propose a new acronym (because we don’t have enough in our industry as it is).

AIR – Ask, Inform, Respect

It’s pretty straightforward, but you would be surprised what keeping these principles in mind can do for your relationship with your patients.  To start, ask patients what matters to them and how we can better design trials to meet their needs, and listen to what they say.  Then, make an effort to keep patients as informed as you can throughout the project without threatening study integrity.  Ultimately, respect the patients.  Every one of us is going to be patient at some point in our life. We're only extending to them the same respect and empathy we would hope to get if we were sick, in pain, and anxiously hoping to find a cure that would improve our lives.

Webinar Registration: Putting Patient Centered Principles Into Practice

Practical Strategies to Simplify Patient Centricity: Part 2—Patient Stories & the Importance of Communication

Posted by Brook White on Tue, May 30, 2017 @ 11:45 AM

Shann Williams, Senior Director OperationsShann Williams has over 10 years of experience managing clinical trials. She is the Director of Operations of the Statistical and Clinical Coordinating Center for the division-wide Consolidated Coordinating Center sponsored by the National Institute of Allergy and Infectious Disease (NIAID). In addition, Shann serves as Rho's Project Management Operational Service Leader, an internal expert sharing project management best practices, processes and training.

This is the second in a series of blog posts on putting patient-centric principles into practice. Click here to view the first post.

Excellent retention practices, such as those shared in the first article in this series, are a great first step but before we implement them and call it a day, we need to ask ourselves, why are we doing this? To keep patients in our study for longer and thereby help ensure study success? Or to put patients’ needs first and thereby help to ensure study success? The difference described may be subtle and but I believe it’s one that can makes all the difference to the patients. Our motivations do tend to guide our actions and communications.
In this post, I’ll share two patient stories that changed the way I thought about my motivation for the work we do in our industry. For me, these highlight the importance of putting patients first in our communications as a first step to putting them first in our clinical trials.

Patient 1

The first patient I describe in my story is me.  My personal struggle with my 2004 diagnosis of lupus and rheumatoid arthritis was compounded by the confusion about what having an autoimmune disease cocktail meant for my body.  Not only was every single joint in my body swollen and painful to the extent that I could not dress myself in the morning without tears, but my rheumatologist was unable to control my symptoms.  The only thing that kept the inflammation down in my joints was a prednisone dosage too high to maintain safely in the long-term.  

Throughout this difficult time when I was learning about autoimmune diseases, my future quality of life, and what it meant to lose the ability to run – something I had always enjoyed – the thing I most appreciated about the health care I received were those providers who listened, who did not rush through to their next appointment, and who offered materials so that I could better understand what was happening to my body: the diagnosis, treatments, and long-term options.

One particular afternoon after a series of frustrating failed call-backs from my doctor and scheduling roadblocks that I continued to run into in order to get a follow-up appointment, I was finally able to speak with a human physician over the phone about additional treatment options. During our brief call, a clinical trial was mentioned. The study was explained hurriedly and almost in an exasperated tone. It was a trial to see whether a product used typically as chemotherapy treatment would work for the severe symptoms I was experiencing in people with my disease.  Unfortunately I never found out if this treatment could have helped me because I flat refused it solely due to the poor communication used when it was described to me. Certainly the company or Government sponsor funding that study would have been horrified to hear how their study was introduced.  Perhaps that care giver was having a bad day.  I’ve often thought that I was only one of many millions and surely my experience was an unfortunate one-off. 

Sadly, however, I don’t believe my story is all that dissimilar from others.  As a matter of fact there is a reason why the Center for Information and Study on Clinical Research Participation (CISCRP) and ICON partnered last year and held the “Inspiring Hope Ideathon” to solicit greater awareness of and participation in research studies—because it’s severely lacking.  

As a matter of fact,according to this study on clinical trial enrollment demand, it would take 1 in 6 people to enroll in a clinical trial to meet enrollment goals for the current studies listed on clinicaltrials.gov.  Why is this the case?  

It would be foolish to assume the challenge lies with communication alone and in the way patients are approached to participate in research.  But, like the old adage says, “how do you eat an elephant?”  It could be a small “bite” in the right direction.   

eating the elephant of patient centricity

Patient 2

Thankfully the story of patient 2 is in stark contrast to my story.  I was waiting with my husband last fall getting ready to get called back for pre-op to have my thyroid removed. We were sitting next to a family in a crowded surgical suite waiting area with their nine-month-old baby girl also waiting for her buzzer to be paged.  Her scheduled surgery was to remove a very rare kidney growth. 

A Study Coordinator sat with them and explained a research study to the mother, father, and grandparents and discussed the potential enrollment of their baby.

The parents were paged and they and the study coordinator went back to pre-op.  Apparently they had signed the consent form because the study coordinator came back to chat with the grandparents.  The thing that struck me was that the study coordinator came back. She sat with them just to see how they were, asked if they needed anything and got them some water, and answered questions about their granddaughter’s disease. The grandparents asked her a slew of questions ranging from “what if it’s cancer?” to specific questions about some of the long-term potential outcomes. The coordinator carefully navigated the questions and often said, “I’m not a doctor but…” and then relayed to them her personal experiences with infant renal dysfunction based on the study.  She spent 20 minutes with them and afterwards as she handed them her card and offered to answer any additional questions they had, and to put them in touch directly with the physician overseeing the study.  As she stood to leave, the grandmother said that she was the first person to care enough to answer their questions and explain what was going on with their granddaughter in months.  The grandmother began to cry and reached to hug the study coordinator. 

Isn’t this why we’re in this industry in the first place?  To help people live healthier, longer lives and increase their quality of life?  Isn’t “bedside manner” and general caring the bare minimum in easing patient burden?  What if all clinical sites vetted and hired competent and caring communicators?  What if we offered specific trainings on how to better approach patients as a requirement for any potential clinical trial recruiter?

Or perhaps a simple change in our motivations and approach could move clinical research participation forward and make the experience better for patients overall. 

Free Webinar: Protocol Design