Factors Affecting Signal Detection in Schizophrenia Clinical Trials
Multiple factors challenge signal detection in schizophrenia clinical trials, including insufficient understanding of the biological mechanisms underlying schizophrenic psychopathology, inadequacy of trial designs, challenges in patient selection, and marginal sufficiency of efficacy endpoints [2-3].
Increased placebo response
In recent years, placebo response has increased while drug response has remained stable in acute schizophrenia clinical trials [2]. Moreover, there have been recent, unexpected phase 3 acute schizophrenia trial failures following robust phase 2 success. Additionally, robust placebo-drug separation in phase 3 trials with stable patients showing predominantly negative symptoms is rare, with no pharmacological treatments demonstrating clear effectiveness [3,4].
Complex, subjective rating scales
Compared to other CNS and non-CNS therapeutic areas, rating scales utilized in in schizophrenia clinical trials, especially those used to assess negative symptoms, are relatively complex and subjective. This presents many challenges for the investigator, who is required to measure symptom severity with accuracy and precision while managing expectation bias from patients and informants that might enhance placebo response.
Rater performance
Factors modulating successful selection and calibration of raters and their performance rating subjects once the study is underway are poorly understood [5]. Phase 3 trials may be vulnerable to failure after successful phase 2 trials due to expectation bias and greater challenges calibrating a larger number of sites, languages, and cultures. Loss of anticipated placebo-drug separation has been attributed in part to excess placebo response, the COVID-19 pandemic, geopolitical conflict, and “bad apple sites”.
Data Quality Issues are Common Even Among Experienced, Trained Investigators
In a large sample of clinical trial PANSS ratings, Rabinowitz et al found that almost 40% of PANSS study visits had at least one inconsistency flag raised and 10% had two [6]. This mirrors our experience in which a wide variety of data anomalies are detected even among experienced, well-vetted raters.
Precision in Measurement Among Investigators Impacts Sample Size Requirements and is Readily Achievable
The impact of calibration and reliability of ratings on sample size, statistical power, and the ability to detect placebo-drug differences in clinical trials is well documented [7]. Calibration of raters increases confidence in trial results, reduces costs, and saves time by allowing for smaller sample sizes. A rater’s performance in the certification process to rate the PANSS appears to be modestly but statistically significantly predictive of performance rating patients at the site [8]. Inconsistent interviewing practices can alter patient responses and obscure drug signals. Interviewing a live actor portraying a subject may be employed to assess and calibrate raters’ interview practices [8]. Sufficient probing to distinguish among the anchor points of lengthy rating scales, objectivity, and efforts to neutralize expectation bias and thus reduce placebo response should be evaluated [9,10].
The impact of the informant on signal detection
Training and standardization of interviewing procedures typically focus on directly assessing the patient. However, the basis of rating numerous PANSS questions includes the informant [11]. Not including the informant information, as sometimes done in clinical trials, appears to result in lower PANSS total scores and reduced changes in symptom severity over time [11]. Further, inconsistent use of informant information across visits may obscure the study signal.
Informants, like patients, may be subject to expectation bias that can impact placebo response. Thus, PANSS interview training should focus on both the patient and informant. The Informant Questionnaire (IQ–PANSS) is sometimes utilized in schizophrenia clinical trials to assure informant information is systematically collected [11]. Once the study is underway, rating scale interviews of both the patient and informant may be recorded for external review of rating and interview quality.
Interview training to mitigate placebo response
Critical but sometimes ignored aspects of interview training are placebo response mitigation measures such as reduction of expectation bias and dissuasion of the natural tendency to guess treatment allocation [1, 10]. The former may be a particularly potent source of placebo response in phase 3 trials due to positive expectations from successful phase 2 trials. For optimal effect, placebo response mitigation training measures should directly address the rater, patient, informant, and everyone else at the site who has contact with the patient and informant. Cohen and colleagues (2021) observed that in subjects with psychotic and major depressive disorders, a participant-focused psychoeducational procedure, educating and subsequently reminding participants about key factors known to amplify placebo response, was associated with a systematic reduction in symptom reports and global subjective impressions of change over the study period [12].
In part two, we will discuss additional data quality indicators and remediation strategies to optimize signal detection in schizophrenia clinical trials.
The content of this blog was derived from our recent article in the Journal of Schizophrenia Research.
ABOUT THE AUTHORS
Dr. David Daniel leads scientific, clinical, and strategic direction for CNS solutions at Signant Health. With over 30 years of experience in psychiatric clinical trials, he has published extensively and holds patents for treatments in epilepsy, anxiety, and psychotic disorders. Dr. Daniel earned his MD from Vanderbilt University, where he also completed psychiatry training as chief resident, and he is a Phi Beta Kappa graduate of Emory University. He previously founded Global Learning, LLC, and held leadership roles at the NIMH and Stanley Foundation.
Dr. Busner has over 35 years of experience as an academic psychiatric researcher, serving as Principal Investigator for 49 clinical trials and Sub-Investigator for 35 more. She has authored or co-authored over 140 peer-reviewed articles and presentations. Before joining Signant Health, she directed psychiatric clinical trials at two major medical schools and served on University IRBs for 20 years. Currently an Affiliate Associate Professor of Psychiatry at Virginia Commonwealth University, Dr. Busner leads studies at Signant on pediatric, rare, and psychiatric disorders, and has trained thousands of clinical trial investigators worldwide.
Dr. Alan Kott is the Practice Leader for Data Analytics at Signant Health, with both academic and industry experience in clinical trials. He has led the development of Signant’s Data Analytics Program, overseeing data analytics in over 200 clinical trials across multiple indications. Prior to joining Signant, Dr. Kott was an Assistant Professor at Charles University and a house officer in psychiatry at General Teaching Hospital in Prague. He holds a Medicinae Universae Doctor (MUDr.) from Charles University.