Rater training and certification are critical to the success of clinical trials. Well-trained and calibrated raters ensure that study data are accurate, valid and reliable. This directly impacts the quality of the trial’s outcomes, allowing researchers to detect meaningful signals in the data and make confident conclusions about a treatment’s effectiveness.
Without proper rater training, there is a risk of excessive data variability, which can adversely affect analysis results, leading to wasted resources and, ultimately, failed trials.
While providing excellent rater training is crucial, developing effective training models presents significant challenges for all stakeholders. Maintaining high-quality standards must be balanced against practical constraints in implementation.
The challenge starts with sites, where even seasoned raters must invest time and resources to complete training. This can create significant burden for investigators at busy sites and may limit the number of trials they can support at a given time. Lengthy and expensive training programs also impact sponsors and CROs, which can slow down study start-up timelines. This delay ultimately impacts pharmaceutical companies’ ability to bring new treatments to market.
Rater training can also inadvertently decrease diversity in clinical trials. Sponsors may be reluctant to use novice raters, fearing that such raters may not pass rater training certification thresholds, or that their data might not be as reliable as data from experienced raters.
Consequently, new raters and sites are frequently excluded due to a lack of experience with clinical assessments. This practice limits the diversity of clinical trials and accessibility of treatments by underserved populations, leading to global healthcare disparities.
Signant Health’s Solution: The Rater Excellence Program
At Signant Health, we rely on a powerful rater training matrix that combines sensitivity to variable rater experience histories with years of data from our Data Quality Monitoring (DQM) and PureSignal Analytics (BDA) Programs. Through this dynamic and comprehensive framework, training is tailored to each rater’s level of experience and known areas of required improvement.
Our approach customizes rater training based on three distinct experience levels: novice, experienced, and expert raters. Novice raters receive comprehensive, hands-on training, including scale scoring exercises, ongoing scale administration, and scoring quality reviews to ensure their performance meets established standards.
Experienced raters with a solid track record benefit from streamlined, more targeted training focusing on expert-level scale nuances and protocol-specifics, alongside regular calibrations. Expert raters with consistently high-quality data could be exempt from most of the training and only require calibration exercises.
A More Efficient and Inclusive Approach to Rater Training
This approach does not just customize rater training to streamline the process for more experienced raters; it makes rater training more efficient and inclusive. By reducing the burden of unnecessary training for experienced raters, trials can start up faster.
Simultaneously, novice raters are given the tools they need to succeed, which opens the door for more diverse sites and populations to participate in clinical trials.
This training approach can be particularly valuable in reaching geographical areas that pharmaceutical companies traditionally overlook, enabling qualified raters in these regions to participate and thereby increasing access for previously underrepresented patient populations in clinical trials.
Pharmaceutical sponsors can benefit from fresh raters in their studies when the rater training program is customized and designed to ensure valid and reliable data. For raters and clinical sites, Signant Health’s Rater Excellence Program provides a more flexible, tailored training experience that matches rater skill level without adding unnecessary complexity.
Conclusion: A Win-Win for Clinical Trials
Signant Health’s Rater Excellence Program offers a smarter, more adaptive solution to rater training, improving the quality of clinical trials and making them more efficient and inclusive at the same time. It’s a win-win for sponsors, sites, and the future of clinical research.
Please contact us if you would like to partner with us on your Rater Training Program.
About the Authors
Marcela Roy is a Senior Clinical Director in Signant’s Digital Health Science department. She has been with Signant for over 15 years and has over 20 years of clinical and research experience. Her focus is Mood Disorders and Endpoint Reliability quality monitoring. She provides strategic direction in the organization, as well as team leadership and business development support.
Juliet Brown, Director of Endpoint Reliability and a Clinical Thought Leader at Signant Health, has over 25 years of clinical and research experience, specializing in MDD, Bipolar Disorder, Anxiety Disorders, Psychotic Spectrum Disorders, Substance Use Disorders, and Cognitive Behavioral Psychotherapy. She holds a PhD and Master's Degrees in Clinical Psychology from Drexel University. Before joining Signant Health 8 years ago, Dr. Brown provided psychotherapy to individuals with Severe Mental Illness and treated Substance Use Disorders. At Signant, she oversees phase 1-3 global trials, offers clinical guidance, and serves as a Blinded Data Analytics Scientist and Subject Matter Expert.
Martina Micaletto is a Clinical Scientist at Signant Health, with extensive experience in the pharmaceuticals industry. She specializes in computerized cognitive batteries, scale administration and scoring, rater training and certification, and endpoint assessments. Martina is skilled in statistics, research, negotiation, psychology, and customer care. She holds an MSc in Social and Cultural Psychology from The London School of Economics and Political Science (LSE) and an MSc in Development and Psychopathology from The University of Reading.
Gary Sachs, Therapeutic Area Leader in bipolar disease and mood disorders at Signant Health, is a recognized expert in clinical trial methodologies. He founded the Bipolar Clinic at Massachusetts General Hospital and is an Associate Professor of Psychiatry at Harvard Medical School. With over 200 publications, Dr. Sachs also serves on the Scientific Advisory Boards of the National Alliance on Mental Illness and the Depression and Bipolar Support Alliance.