Rho site logo

Rho Knows Clinical Research Services

Embracing Open Source as Good Science

Posted by Brook White on Wed, Nov 30, 2016 @ 09:37 AM

Ryan Bailey, Senior Clinical ResearcherRyan Bailey, MA is a Senior Clinical Researcher at Rho.  He has over 10 years of experience conducting multicenter asthma research studies, including theInner City Asthma Consortium (ICAC) and the Community Healthcare for Asthma Management and Prevention of Symptoms (CHAMPS) project. Ryan also coordinates Rho’s Center for Applied Data Visualization, which developsnovel data visualizations and statistical graphics for use in clinical trials.

open source software in clinical researchSharing. It's one of the earliest lessons your parents try to teach you - don't hoard, take turns, be generous. Sharing is a great lesson for life. Sharing is also a driving force behind scientific progress and software development. Science and software rely on communal principles of transparency, knowledge exchange, reproducibility, and mutual benefit.

The practice of open sharing or open sourcing has advanced these fields in several ways:

We also feel strongly that the impetus for open sharing is reflected in Rho's core values - especially team culture, innovation, integrity, and quality. Given our values, and given our role in conducting science and creating software, we've been exploring ways that we can be more active in the so-called "sharing economy" when it comes to our work.

One of the ways we have been fulfilling this goal is to release our statistical and data visualization tools as freely-accessible, open source libraries on GitHub. GitHub is one of the world's largest open source platforms for virtual collaboration and code sharing. GitHub allows users to actively work on their code online, from anywhere, with the opportunity to share and collaborate with other users. As a result, we not only share our code for public use, we also invite feedback, improvements, and expansions of our tools for other uses.

We released our first open source tool - the openFDA Adverse Event Explorer - in June 2015. Now we have 26 team members working on 28 public projects, and that number has been growing rapidly. The libraries and tools we've been sharing have a variety of uses: monitor safety data, track project metrics, visualize data, summarize every data variable for a project, aid with analysis, optimize SAS tools, and explore population data.

Most repositories include examples and wikis that describe the tools and how they can be used. An example of one of these tools, the Population Explorer is shown below.

Interactive Population Explorer

interactive population explorer, clinical trial graphics

Access summary data on study population and subpopulations of interest in real time.

One of over 25 public projects on Rho's GitHub page - available at: https://github.com/RhoInc/PopulationExplorer

Over the next few months, we are going to highlight a few of our different open source tools here on the blog. We invite you to check back/subscribe to learn more about the tools we're making available to the public. We also encourage you to peruse the work for yourself on our GitHub page: https://github.com/RhoInc.

We are excited to be hosting public code and instructional wikis in a format that allows free access and virtual collaboration, and hope that an innovative platform like GitHub will give us a way to share our tools with the world and refine them with community feedback. As science and software increasingly embrace open source code, we are changing the way we develop tools and optimizing the way we do clinical research while staying true to our core purpose and values.

If you have any questions or want to learn more about one of our projects, email us at: graphics@rhoworld.com

Big Data: The New Bacon

Posted by Brook White on Wed, Nov 16, 2016 @ 04:10 PM

Dr. David Hall, Senior Research ScientistDavid Hall is a bioinformatician with an expertise in the development of algorithms, software tools, and data systems for the management and analysis of large biological data sets for biotechnology and biomedical research applications. He joined Rho in June, 2014 and is currently overseeing capabilities development in the areas of bioinformatics and big biomedical data. He holds a B.S. in Computer Science from Wake Forest University and a Ph.D. in Genetics with an emphasis in Computational Biology from the University of Georgia.

big data is the new baconData is the new bacon as the saying goes. And Big Data is all the rage as people in the business world realize that you can make a lot of money by finding patterns in data that allow you to target marketing to the most likely buyers. Big Data and a type of artificial intelligence called machine learning are closely connected. Machine learning involves teaching a computer to make predictions by training it to find and exploit patterns in Big Data. Whenever you see a computer make predictions—from predicting how much a home is worth to predicting the best time to buy an airline ticket to predicting which movies you will like—Big Data and machine learning are probably behind it.

However, Big Data and machine learning are nothing new to people in the sciences. We have been collecting big datasets and looking for patterns for decades. Most people in the biomedical sciences consider the Big Data era starting in the early to mid-1990s as various genome sequencing projects ramped up. The human genome project wrapped up in 2003, took more than 10 years, and cost somewhere north of $500 million. And that was to sequence just one genome. A few years later, the 1000 Genome Project started, whose goal was to characterize genetic differences across 1000 diverse individuals so that we can predict who is susceptible to various diseases among other things. This effort was partially successful, but we learned that 1000 genomes is not enough.

cost of human genome sequencingThe cost to sequence a human genome has fallen to around $1,000. So the ambition and scale of big biomedical data has increased proportionately. Researchers in the UK are undertaking a project to sequence the genomes of 100K individuals. In the US, the Precision Medicine Initiative will sequence 1 million individuals. Combining this data with detailed clinical and health data will allow machine learning and other techniques to more accurately predict a wider range of disease susceptibilities and responses to treatments. Private companies are undertaking their own big genomic projects and are even sequencing the “microbiomes” of research participants to see what role good and bad microbes play in health.

Like Moore’s law that predicted the vast increasing in computing power, the amount of biomedical data we can collect is on a similar trajectory. Genomics data combined with electronic medical records combined with data from wearables and mobile apps combined with environmental data will one day shroud each individual in a data cloud. In the not too distant future, maybe medicine will involve feeding a patient’s data cloud to an artificial intelligence that has learned to make diagnoses and recommendations by looking through millions of other personal data clouds. It seems hard to conceive, but this is the trajectory of precision medicine. Technology has a way of sneaking up on us and the pace of change keeps getting faster. Note that the management and analysis of all of this data will be very hard. I’ll cover that in a future post.

View "Visualizing Multivariate Data" Video

Statisticians and Critical Variable Review Help Streamline Data Management and Clinical Operations Activities

Posted by Brook White on Mon, Oct 24, 2016 @ 10:11 AM

Heather Kopetskie, Senior BiostatisticianHeather Kopetskie, MS, is a Senior Biostatistician at Rho. She has over 10 years of experience in statistical planning, analysis, and reporting for Phase 1, 2 and 3 clinical trials and observational studies. Her research experience includes over 8 years focusing on solid organ and cell transplantation through work on the Immune Tolerance Network (ITN) and Clinical Trials in Organ Transplantation (CTOT) project.  In addition, Heather serves as Rho’s biostatistics operational service leader, an internal expert sharing biostatistical industry trends, best practices, processes and training.

risk.jpgAssessing risk and using it to determine the focus of clinical trial activities has been an important goal in clinical research for a number of years now. One way statisticians can contribute to this is through critical variable review. In critical variable review, statisticians map the case report form (CRF) to the primary and secondary endpoints.

This review is important for several reasons and impacts all team members managing the clinical data. First, it ensures upfront that all of the data needed for the planned analyses are being collected. While this seems obvious, in an unfortunate number of cases, it is not until the end of the study that teams discover that all of the information needed has not been collected. Second, data managers can incorporate critical variables into the data management plan to focus edit checks, cross checks, and data cleaning activities on forms containing critical variables. Third, clinical monitors can focus the clinical monitoring plan on critical variables and ensuring key variables are reviewed during on-site visits but also during remote data monitoring.

Statisticians, data managers, and clinical monitors should start reviewing data as early as possible after first patient first visit with a focus on critical variables. Initially focusing on a report showing for each critical variable how many subjects are expected to have the data, how many have completed data entry, and how many are missing the variable. These reports can be used to identify issues early in a trial and determine how to address issues such site retraining that’s required, process changes when an assessment isn’t standard of care at a site, protocol deviations resulting from missing data, etc. As the study progresses descriptive statistics can be performed on the critical variables for investigators to review and ensure the study is progressing as expected without unblinding the study.

data-review.jpgAs data management evolves from data primarily being collected in an EDC system to data being collected from multiple sources such as EDC, ePRO systems, health electronic records, and central laboratories, additional strategies need to be implemented to ensure a clean integrated database for analysis. Instead of data managers providing all the data cleaning data managers, programmers, statisticians, and clinical monitors will need to collaborate. All members of the team should meet regularly to discuss progress and develop tools that will facilitate cleaning across multiple data sources. New tools and strategies will need to be implemented. We outline a few strategies we’ve piloted for collaboratively reviewing data early after database launch. Early looks at the data can provide a sense of how sites are entering data. Dealing early on with issues that arise will prevent lots of dirty data at the end of the study.

One strategy is an evaluation of all free text fields completed in the database. Sites may be entering data in the wrong place or collecting data that is not needed which can be fixed through site re-training. Additionally, this review can highlight additional fields or updates that need to be added to the CRFs.

Another strategy is code book reviews. A code book is a file which provides descriptive statistics on all fields in the EDC system that can be reviewed by all members of the study team. This is an easy way to identify outliers by data field and site-to-site differences. (Codebook examples and macros are available in Github.)

Statisticians and programmers can also compile data across multiple sources to identify what data fields are missing (ePRO not entered), what information doesn’t reconcile (e.g. biopsy date in EDC versus specimen collection system), what deviations may be expected from data sources outside of EDC, etc. and provide one succinct report for the data managers to facilitate communication with the site to reconcile and update data.

Additionally, constant communication between team members can bring to light common themes the clinical monitors are seeing during their visits, data managers are seeing through queries, and statisticians are observing during data preparation. This allows for early action which can minimize time spent at the end of the study to clean the data and lock the database.

One thing that has become abundantly clear is that a risk-based approach to clinical trials requires close collaboration between disciplines. Data managers, clinical monitors, and statisticians must work together in ways they have not in the past. Traditional models that rely on functionally-aligned silos will not allow risk-based approaches to succeed.

Webinar: Clinical Research Statistics for Non-Statisticians

Scientists Search for Answers as Antibiotics become Obsolete

Posted by Brook White on Thu, Aug 18, 2016 @ 01:41 PM

Carrie Furr, PhD, RACCarrie Furr, PhD, RAC, is a Senior Director Operations at Rho. Day-to-day, Carrie supports pharmaceutical sponsors of clinical trials, leading an integrated product development program consisting of clinical, preclinical, chemistry, manufacturing and controls, and regulatory components. Before diving into the clinical trials industry, Carrie spent 7 years earning her PhD in biochemistry, bacteriology and bacteriophage biology at Texas A&M University. A biologist by training, Carrie’s dissertation and doctoral research focused on how a bacteriophage (phage) protein causes bacteria to die. In theory, phage proteins can be used in phage therapy to combat any bacteria-based disease. 

As anyone who has ever had a bacterial infection or seen an end-of-the-world survival movie knows, antibiotics are an important tool in a doctor’s arsenal. Since the discovery of penicillin in 1928, antibiotics have been extremely effective at treating and preventing a variety of infections. But imagine life without them – doctors unable to prevent infections after surgery, your child’s minor cut might morph into a major infection, and there wouldn’t even be a treatment for pink eye.

With the increasingly popular use of antibiotics, however, microbes have mutated and learned to resist the drugs. Penicillin was once extremely effective against most strains of bacteria, but now it is used far less frequently as many strains have built up resistance. In many cases antibiotics are becoming obsolete.

What can be done? Without antibiotics, a huge range of diseases -- from pneumonia to strep throat to syphilis – would become much more difficult, if not impossible, to treat. While there are a few things you can do to help prevent antibiotics from further increasing resistance – for example, taking all prescribed antibiotics even if you feel better or only taking antibiotics when you truly have an infection – it isn’t enough. Scientists need to work on other solutions.

phage, bacteriophageBacteriophages, or “phages” for short, may be able to help. Certain phage proteins cause bacteria to die. Researchers are working to determine if these proteins or whole phages can be safely converted into therapies to combat bacteria-based diseases. Developing an alternative treatment to antibiotics could have huge implications on the treatment of bacterial infections around the world. Additionally, phage therapy holds the promise of providing a dynamic solution to the dynamic problem of antibiotic resistance.

While academics and pharmaceutical companies work on the research, people like me are working on smoothing the road to U.S. Food and Drug Administration (FDA) approval for these novel therapies. As a postdoctoral researcher, I focused on how bacteriophages can kill certain bacteria. As a senior regulatory scientist, I work on the practical steps required to bring pharmaceutical products, including phage therapy, into the market to treat patients. Currently, the path to develop phage therapy through to regulatory approval is unclear.

In the face of antibiotic resistance in the U.S. and around the world, it is important to understand our alternatives and what must be done to advance alternative treatments.

Webinar: Tips for a Smooth NDA Submission

Helping Physicians Provide Personalized Asthma Care – the CHAMPS Project

Posted by Brook White on Tue, Aug 09, 2016 @ 02:02 PM

Ryan2.jpgRyan Bailey, MA is a Senior Clinical Researcher at Rho.  He has over 10 years of experience conducting multicenter asthma research studies, including theInner City Asthma Consortium (ICAC) and the Community Healthcare for Asthma Management and Prevention of Symptoms (CHAMPS) project. Ryan also coordinates Rho’s Center for Applied Data Visualization, which developsnovel data visualizations and statistical graphics for use in clinical trials.

Does your healthcare provider offer a comprehensive, personalized, asthma management plan for treating your asthma?

Personalized medicine is considered the future of healthcare, and for good reason. The more we learn about patients, genetics, and diseases, the more we realize that a ‘one-size-fits-all’ approach to healthcare is not the best way forward. Personalized medicine, by contrast, is designed to provide a treatment plan that is customized to each patient.

respiratory.pngWhile this approach offers tremendous promise for patients, it also poses a number of challenges for doctors who treat complex, multi-faceted diseases like asthma. Asthma can be influenced by a wide range of factors, including: genetic traits, allergen sensitivities, environmental exposures, stress, socioeconomic conditions, and access to healthcare. With so many variables to consider, developing a patient-tailored treatment can be daunting for healthcare providers. Personalized care can also require more time – time to learn about each patient and the various factors contributing to their disease, and time to administer a comprehensive care plan.

Fortunately, research and advocacy programs are seeking to overcome these challenges for patients with asthma. One such program is the Community Healthcare for Asthma Management and Prevention of Symptoms or "CHAMPS" project, funded by the Merck Childhood Asthma Network.

The CHAMPS project was based on 25 years of National Institutes of Health asthma research that began with two landmark clinical trials – the National Cooperative Inner City Asthma Study (NCICAS) and the Inner City Asthma Study (ICAS). NCICAS and ICAS focused on providing patient-tailored asthma care using a team-based approach. Each patient was assessed by physicians and information was collected about their asthma. This information was used to create a personalized care plan, which was delivered by a team of healthcare providers and specially-trained asthma counselors.

Both NCICAS and ICAS demonstrated strong results for their patient-tailored asthma interventions, but one important question remained – could these results be repeated in the ‘real world’? CHAMPS was designed to answer this question. Using NCICAS and ICAS as models, CHAMPS researchers set out to fulfill two objectives:

  1. test whether the clinical trial interventions could be implemented in the real world setting of health clinics, and, if successful
  2. provide resources that other health practices could use to implement CHAMPS within their centers

In the first asthma study of its kind, CHAMPS researchers found that health clinics in three different states, with different healthcare teams, insurance plans, and levels of resources were capable of successfully implementing CHAMPS’ cost-effective, patient-tailored asthma program. Once they determined that the program was successful in real world settings, CHAMPS researchers turned their attention toward helping other health practices implement the program.

In partnership with the Environmental Protection Agency, the CHAMPS team released a series of freely-available resources on the Asthma Community Network website, including short eLearning videos, educational handouts, research, and a detailed procedural manual. These CHAMPS materials are designed to teach any health practice how to conduct the CHAMPS asthma program, and are available to the public at http://www.asthmacommunitynetwork.org/Champs.
What does this mean for asthma sufferers?  

The CHAMPS project was supported by The Merck Childhood Asthma Network, Inc. and coordinated by researchers at George Washington University Milken Institute School of Public Health and Rho, Inc. Additional support was provided by the RCHN Community Health Foundation.


Rho Participates in Graduate Student Workshop

Posted by Brook White on Mon, Aug 01, 2016 @ 03:30 PM

At Rho, we are proud of our commitment to supporting education and fostering innovative problem solving in the next generation of scientists, researchers, and statisticians. One way we have been excited to promote innovation is by participating in the annual Industrial Math/Stat Modeling Workshop for Graduate Students (IMSM) hosted by the National Science Foundation-supported Statistical and Applied Mathematical Sciences Institute (SAMSI).  This summer marked our 6th consecutive year as a Problem Presenter.  We were joined by fellow presenters from Sandia National Laboratories, the US Army Corps of Engineers, the Environmental Protection Agency, the Cooperative Institute for Climate and Satellites, Pfizer Inc., and faculty from North Carolina State University, Clemson University, and the University of Cincinnati.

SAMSI participants 2016

Agustin Calatroni (second from left), Hoang Tran (second from right), and Emily Lei Kang (third from right) with students from the SAMSI program.

For the 2016 workshop, Rho was represented at the workshop by investigators Dr. David Hall, Dr. Herman Mitchell, Agustin Calatroni, and Bioinformatics Intern Hoang Tran, who was a student participant in the 2015 workshop. With the assistance of Dr. Emily Lei Kang from the University of Cincinnati, Rho presented their problem to a team of math and stat graduate students: using peptide microarray data, a highly sensitive procedure which is prone to high variability, identify the ‘global’ set of epitopes within a group of tree nut and peanut allergic sufferers by separating biologically-relevant signals from background noise. The application of this project is to better understand the molecular mechanisms of the interactions between the immune system and food allergens, which can be used to develop immunotherapy-based treatments. The IMSM students decided to address this problem by using Positive Unlabeled learning, a machine learning approach, to predict which signals in the microarray data are noise.

Rho is honored to have the opportunity to work with exceptional students and faculty to apply state of the art mathematical and statistical techniques to solve real world problems and advance our knowledge of human diseases.

You can visit the IMSM Workshop website to learn more about the program, including the problem Rho presented and the students’ solution (page 45).

Ryan2.jpgRyan Bailey, MA is a Senior Clinical Researcher at Rho.  He has over 10 years of experience conducting multicenter asthma research studies, including theInner City Asthma Consortium (ICAC) and the Community Healthcare for Asthma Management and Prevention of Symptoms (CHAMPS) project. Ryan also coordinates Rho’s Center for Applied Data Visualization, which developsnovel data visualizations and statistical graphics for use in clinical trials.


Webinar: Clinical Research Statistics for Non-Statisticians

Rho Participates in Innovative Graduate Student Workshop for 5thYear

Posted by Brook White on Tue, Aug 04, 2015 @ 04:21 PM

For the fifth consecutive summer, Rho participated in the Industrial Math/Stat Modeling Workshop for Graduate Students (IMSM) hosted by the National Science Foundation-supported Statistical and Applied Mathematical Sciences Institute (SAMSI). The workshop, a joint program of Duke University, North Carolina State University, the University of North Carolina at Chapel Hill, and the National Institute of Statistical Sciences, introduces graduate students in mathematics, engineering, and statistics to real-world challenges arising in industrial and government laboratory research. IMSM students are divided into teams and are given 10 days to collaborate on a solution to a “real world” problem presented by the participating corporate and government research teams.


2015 IMSM Participants

For the 2015 workshop, Rho was one of five ‘problem’ presenters that included the EPA, the MIT Lincoln Laboratory, Sandia National Laboratories, and the US Army Corps of Engineers.  Rho was represented at the workshop by CEO Dr. Russ Helms, Vice President and Senior Research Scientist, Dr. Herman Mitchell, and Senior Statistical Scientist, Agustin Calatroni.  With the assistance of Dr. Emily Lei Kang from the University of Cincinnati, Dr. Mitchell and Mr. Calatroni worked with math and stat graduate students from various universities around the country on a complex “big data” problem related to microbial exposures in U.S. homes of children with asthma.  The Inner City Asthma Consortium, a national multi-center National Institutes of Health project coordinated by Rho, has been exploring the impact of more than 50,000 specific bacteria found in homes and how this community of bacteria (the “microbiome”) influences a child’s immune system so that it may protect against, or promote, the development of asthma.  The students developed statistical and mathematical models to examine how home characteristics and behaviors determine the microbial exposure mix observed in the families’ homes.

We are proud to continue this annual partnership with IMSM for a fifth year.  This collaboration between exceptional students and faculty is another way we are applying cutting edge mathematical and statistical techniques to solve real world problems and advance our knowledge of human diseases.

You can visit the IMSM Workshop website to learn more about the program, including the problem Rho presented and the students’ solution (our team report begins on page 89).


Q&A: Using ePRO with Smart Devices

Posted by Brook White on Fri, Jul 17, 2015 @ 10:58 AM

Emily Cantrell, Senior Director OperationsBecky Baggett, Senior Project ManagerEmily Cantrell, Senior Director Operations, and Becky Baggett, Senior Project Manager recently completed enrollment six weeks early on a phase 3 study using ePRO with a tablet.

On June 10th, we hosted a webinar featuring a case study using ePRO with a smart device. If you missed the webinar, you can click here to register and view the webinar on-demand.

Several questions came up during the webinar that we did not have time to address. Below, we’ve responded to the unanswered questions that were submitted during the webinar.

Q: You mentioned that the guidance on Equivalence Validation is vague and your decision to include was a conservative approach due to the trial being a Phase III pivotal. Can you provide additional comment on when EV will and will not be required?

A: It is difficult to say when this equivalence validation should or should not be performed; however, the impact of the data collected for your marketing claims may help you decide. In our case, since our primary efficacy endpoint for our pivotal phase III was captured on the tablet, the Sponsor and Rho felt it was an important step in support of the data validity. At the time this was a very expensive decision, but it may be the case that the vendor has PROs in their library that have previously been validated, perhaps reducing the impact of cost on your decision.

Q: Are you aware of any tablets that utilize thumb print technology in lieu of a subject PIN?

A: We did not use this technology for our study nor have we been presented with this technology from our vendors.

Q: Can you comment on smart phone apps and other methods for patients to do ePRO at home?

A: Rho does have experience working with a vendor that issues smart phones as at-home devices. The challenges here are ensuring the subjects keep the batteries charged, take care of the device and remember to enter their outcomes. To date, Rho does not have experience using a smart phone application, but we are exploring this option. We have learned that many of the challenges one would experience with the average smart phone technology may complicate the use in clinical trials as well. For example, subjects may inadvertently delete the app from their phone, operation system upgrades may change the way the app interacts with the user, or subjects may lose or damage their phone. One other point of consideration is how subjects may be compensated for the use of their data plan.

Q: Does validation have to be specific to the exact device such as an iPad vs a smart phone app vs another tablet type?

A: In our case, we utilized the exact device with the validation subjects to ensure they were reviewing exactly what the study subjects would be using. For studies that have VAS outcomes, Rho would strongly recommend using the exact device to ensure that the size and scaling did not impact the outcome. For NRS or simple questionnaires, it’s not as pressing, but Rho would recommend obtaining the vendor's prior experience interacting with the FDA to inform the decision.

Q: Do you have experience using an app vs a separate device?

A: To date, we have not yet had experience with using an app versus a device. We realize there may be challenges for this and would want to carefully weigh the options with the Sponsor and study team during study start-up.

Q: How is the translation vendor selected? Does Rho typically select the translation vendor, or do you leave that up to the sponsor or ePRO vendor?

A: Rho has preferred providers for translation, and we do a cost comparison for each bid. There are times where the vendor has been selected by the Sponsor, so we will use their vendor.

Free Webinar: Selecting Inclusion/Exclusion Criteria for Your Next Trial

5 Tips for Selecting ePRO Vendors

Posted by Brook White on Tue, May 12, 2015 @ 09:46 AM

Emily Cantrell, Senior Director OperationsBecky Baggett, Senior Project ManagerEmily Cantrell, Senior Director Operations, and Becky Baggett, Senior Project Manager recently completed enrollment six weeks early on a phase 3 study using ePRO with a tablet.

Over the past 18 months, we’ve been working on a study that utilizes electronic patient reported outcomes (ePRO) on tablet devices for a large phase 3 pain study.  We’ll share a number of considerations and recommendations for selecting an ePRO vendor for a clinical study based on the challenges we’ve faced and the lessons we’ve learned. 

ePRO is not yet a fully mature technology, so it is likely that you will face some challenges regardless of the vendor you use. 

  1. Carefully review the bids that you are sent to make sure they cover everything you need.  Did they include validation if you need it?  Did they include enough support time or calls?  In addition to the devices needed for each site or subject (depending on your study set-up), have they included some extras for the study team and for back-ups?
  2. Make sure the vendor specifies exactly which device and accessories they will use.  Ask about plans to change or upgrade devices during the course of your study and make sure they have enough devices in stock or readily available to meet your needs and timelines.  We ran into two issues during our study.  The first is the stylus used during the presentation for us wasn’t the stylus the sites were sent (and the quality was different).  The second issue was that the ePRO vendor made a decision to upgrade to a new device between their presentation to us and study live date. The sponsor had already seen the previous device and it was built into the Work Order. This made things more difficult from our perspective.
  3. Be involved in user acceptance testing (UAT) for the device. More than likely you will be given a script to follow, but don’t be afraid to test other scenarios to confirm the device is working appropriately. Performing UAT opened our eyes to the fact that some assumptions we made about how the tablets worked were not always true. This was also helpful for the study team to develop the training for the sites and CRAs. 
  4. Consider how you want to handle support.  The vendor provided technical support was really only helpful if the issue was only technical in nature.  We chose to provide frontline support to sites through our own client support call center.  Using our own client support call center, we were able to weed out training issues and issues related more to the study than the technology while providing better customer service.  
  5. If you haven’t already selected a vendor, check with site staff you trust.  It is likely that they have previous experience with multiple systems and may be able to make good recommendations.  Even if they don’t, they may appreciate that their perspective is being considered.

Despite all of the challenges,  we still believe it was worth it to use ePRO for our study.  The quality of the data is good, there are economies of scale, and the duration and scope of the study warranted it.  

Free Webinar: ePRO and Smart Devices

Introducing Rho's Center for Applied Data Visualization

Posted by Brook White on Fri, Nov 21, 2014 @ 10:02 AM

Our industry is driven by data. Every phase of our trials requires us to collect, monitor, analyze, and report data. While each of these steps is equally important, reporting is arguably the most impactful step. When we report data, we give them to key decision-makers and invite them to interpret the data and draw conclusions.
Is the trial being conducted correctly? Is participant enrollment on schedule? Are we protecting our participants' safety? Was the investigational product effective? Was our hypothesis confirmed? We rely on effective data reporting to answer these questions.
scatterUnfortunately, our industry doesn't always use the best tools or practices when it comes to data reporting. If you've ever had to make sense of 50 pages of data listings or spend hours creating figures using spreadsheet software, you know what we mean. If these methods feel outdated, it's because they are. We have been using the same basic technologies to report data for the past few decades with little improvement. The good news is that there are plenty of alternatives available to us and our industry is ripe for change.
Granted, some of the formats for reporting are mandated by formal regulations. We may not be able to do much about these reports, but many of the methods we use to report data are left up to us as clinical researchers. As such, we argue that clinical researchers have a beeswarmresponsibility to do the data justice and communicate them as clearly and effectively as possible.

What does this mean for our industry? It means looking for newer and better ways to communicate data. It means thinking carefully about how the method of reporting impacts perception and comprehension of data. It means researching novel technology tools for sharing data.
At Rho, we took these challenges to heart and created a new Center for Applied Data Visualization (ADV) to research and promote the best practices and tools for visualizing and reporting data. The ADV was founded by a team of senior biostatisticians, web programmers, and a study coordinator who have years of experience directly supporting sunburstclinical trials. This first hand experience with clinical research gave our team a unique perspective on the data reporting needs at all stages of clinical research from study design, to participant enrollment, monitoring, data collection, analysis, data exploration, to publication and reporting. Hence, the ADV marries clinical research experience with the technical skillset to create innovative, cutting-edge data visualizations in support of our research projects. Moreover, the ADV provides trainings throughout the company on graphics best practices and tool development.
In support of our projects, the ADV has developed dozens of novel graphics for both static reports and interactive web-based use. In both cases, the response from our clients and research partners has been overwhelmingly positive. Beginning this month, the ADV is expanding their focus to also provide resources external to Rho. Members of the ADV have been presenting their work and tools in public forums for years, but now we are moving toward releasing some of our tools, resources, and graphics research open source (free to use) on our new graphics sharing website: graphics.rhoworld.com. The site currently has two tools available, and additional graphics will be posted, and discussed here, on a regular basis.

Data visualization has tremendous potential to improve the way we communicate, understand, and interact with data. If you would like to learn more about Rho’s data visualization work, we would love to hear from you at: graphics@rhoworld.com

View "Visualizing Multivariate Data" Video