Understanding and Optimizing Independent Psychiatric Eligibility Reviews in CNS Trials: What, Why, When, and How?

Ensuring accurate psychiatric eligibility assessments is critical to the success of CNS clinical trials. Variability in diagnostic practices, investigator subjectivity, and the complexity of psychiatric conditions can introduce inconsistencies that impact data quality and study outcomes. Independent Psychiatric Eligibility Reviews help mitigate these challenges by providing a standardized, objective approach to screening decisions.

Let's explore the key aspects of Independent Psychiatric Eligibility Reviews—what they are, why they matter, when they are needed, and how they can be optimized to enhance trial integrity and patient safety. By leveraging expert clinical adjudication, sponsors can improve diagnostic accuracy, reduce bias, and strengthen the overall reliability of CNS trial data.

What are Independent Psychiatric Eligibility Reviews?

Standardized, independent adjudication of screening data and supporting information is common in CNS clinical trials. In some trials, adjudication takes the form of Psychiatric Eligibility Reviews, in which an independent cohort of highly trained clinicians considers relevant psychiatric screening information, including documented diagnostic evaluation results, inclusionary scale data, data from other key scales, safety data, site investigator clinical notes, and audio/visual recordings of site rater assessments if collected at the site.

Why are independent Eligibility Reviews needed?

Independent Psychiatric Eligibility Reviews ensure standardization of the information that is considered for psychiatric eligibility, as well as uniformity of the processes used in subject disposition decision-making across cases, sites, and geographical regions. Such reviews also provide assurance that key pieces of information are considered in the final eligibility decision.

Third party Psychiatric Eligibility Reviews also ensure sound clinical decision-making and appropriate subject disposition.

“Wisdom is knowing what you don’t know.” - Socrates

Not all site investigators have clinical expertise in disease course, varying disease symptom manifestations and severity levels, and common comorbidities, especially in studies for which the indication is a rare or complex disease.

Additionally, rates of diagnostic scale administration errors and misapplication of psychiatric diagnostic conventions are surprisingly high.², ⁵Eligibility Review-related site investigator queries combined with provision of scale administration and scoring feedback has a two-pronged positive result – Confidence in subject selection is increased, and data validity and reliability improve prospectively.

When subjects who do not actually have the target indication or those who fail to satisfy all eligibility criteria are randomized, data noise and placebo response may increase, and the drug’s efficacy signal may be clouded. Having an external expert review the eligibility data can help ensure that only appropriate subjects are admitted into the study, thus increasing the likelihood of study success.

Independent Psychiatric Eligibility Reviews also control for the influence of cognitive bias.

“Bad decisions made with good intentions are still bad decisions.” – James C. Collins

Debiasing one’s own beliefs is complex. It requires both awareness of one’s personal biases and the deleterious effects they can have on subject safety and study success, and successful application of debiasing strategies, which vary from person to person.⁷

A neutral "second opinion" can be invaluable, particularly in pivotal trials or studies involving more complicated primary indications and exclusionary diagnostic rule-outs. Third party, centralized eligibility reviewers are divorced from site-related pressures and objectivity-hampering relationships with screening subjects and have more time to dedicate exclusively to each case.

A robust, independent Psychiatric Eligibility Review system can mitigate the risk of excessive heterogeneity in a study population that could lead to atypical endpoint data variability and impede signal detection.

The diagnostic confidence and consistency afforded by secondary reviews will lesson potential noise introduced by subject psychosocial, gender, sex, and cultural variability. Controlling for excessive heterogeneity is critical in large, global trials and those purposively attempting to gather a diverse sample to achieve research and treatment equity goals.

In clinical trials, there is wide agreement that patient safety is paramount. Third party Psychiatric Eligibility Reviews add an extra layer of subject safety protection. Such safety cross-checks can be invaluable in busy trials and studies of high-risk populations.

WHEN are Independent Psychiatric Eligibility Reviews needed in CNS trials?

There is a clear case for the use of third party Eligibility Reviews in studies of more complex or at-risk psychiatric diagnoses (e.g., Borderline Personality Disorder), those investigating treatment of indications with more heterogenous symptom presentations (e.g., Dementia3), or studies involving diagnoses that commonly present with features that can increase the differential diagnostic challenge and lower diagnostic confidence (e.g., Major Depressive Episode with Mixed Features).⁶

Approximately one third of adults with a confirmed diagnosis of psychiatric disorder within the past year had a comorbid psychiatric disorder.¹ Differential diagnosis can be daunting, particularly with subjects who are suboptimal reporters, have an unclear history, or have overlapping or conflicting comorbidity courses, or in studies where the chosen diagnostic scale does not allow for formal evaluation of all potential differentials.

The effects of clinical trial misdiagnoses extend well beyond the subject’s trial completion, with potential stigma and adverse impact to treatment (e.g., unnecessary hospitalizations, inappropriate pharmacotherapies) and quality of life (e.g., occupational impairment) - risks that cannot be overstated.² Some commonly encountered differential diagnostic mistakes include:

• Misdiagnosis of Major Depressive Disorder in a younger adult patient with Bipolar I or II Disorder

• Misdiagnosis of Bipolar I Disorder in a patient with Major Depressive Disorder within which some Major Depressive Episodes have included Mixed Features

• Misdiagnosis of Schizophrenia in a patient with Bipolar I Disorder with a history of positive psychotic symptoms in the context of one or more lifetime Manic Episodes

• Misdiagnosis of Generalized Anxiety Disorder in a patient with Adjustment Disorder with Anxiety caused by a current, major life stressor

• Misdiagnosis of Bipolar Disorder in a patient with Polysubstance Abuse

Agreement between a site-based study diagnostician and a centralized, independent clinician helps ensure ascertainment of appropriate subjects who can safely participate in a trial safely.

Independent Eligibility Reviews can be especially helpful when a protocol’s inclusion and exclusion criteria allow room for investigator subjectivity. For example, in studies whose protocols disallow psychiatric disorders that are commonly comorbid with the illness under study only if the comorbidity is the primary driver of treatment relative to the study indication-related diagnosis, determination of the primary vs. secondary nature of psychiatric disorders is complicated and prone to decision-making bias.

The degree of complexity increases for subjects who are not currently receiving standard of care treatment. A third-party psychiatric adjudicator can provide a fresh, objective perspective and increase diagnostic and decision-making certainty.

Steps to Better Eligibility Decisions

We propose that independent Eligibility Review clinicians employ the DSM-5-TR Differential Diagnosis Model to ensure accurate Diagnostic Validation decision eligibility decisions in CNS trials. Following the model’s step-wise logical flow described below will bring uniformity and thoroughness to the independent review process.

1. Rule out malingering and Factitious Disorder

• Consider the face validity of the presenting syndrome

• Consider signals that an individual is a career subject

2. Rule out substance-related etiology

3. Rule out medical etiology (direct/biological or indirect/psychological)

4. Determine the primary diagnosis in cases with psychiatric comorbidities (i.e. determine which diagnosis is the main driver of treatment decisions)

5. Rule out the possibility that that the primary syndrome is representative of Adjustment Disorder or an Other Specified or Unspecified Disorder

6. Rule out syndromes that are on the boundary of normalcy and choose subjects for whom there is confirmed pathological impairment or distress

• Consider signals that a subject may be an overzealous reporter whose symptoms are less severe than reported

This step-wise approach ensures that reviews go well beyond a protocol-associated ‘checkbox’ approach. Extended from the above system for diagnostic validation, Signant Health reviewers are trained to also consider the following additional case factors:

• Potential psychosocial, medical, and psychiatric data confounds that could lead to excessive data variability

• Temporal instability of illness severity, particularly recent improvement that could be indicative of a subject who is ‘on the road to wellness’

• Ability of subjects to clearly and frankly report symptoms

• Prospective safety risks

How can we optimize Independent Psychiatric Eligibility Reviews?

At Signant Health, we strive to enhance the value of our Eligibility Review service by incorporating the following:

• Robust and extensive reviewer training at the universal and site-specific level

• Ongoing reviewer access to clinical consultation and case discussion meetings

• Ease of access to all relevant information, including electronic data, paper source, audio or video recordings of site rater scale interviews

• Timely, collegial contact with site investigators via email or phone for information clarification and provision of scale administration and scoring reminders

• Use of the Signant Health Clinical Validation Inventory for Study Admission (C-VISATM ) as a low burden electronic platform for site investigators to synthesize all case information for independent reviewer reference⁴

• Review by a second independent clinician for particularly complex cases

• Understanding that site investigators are the experts regarding each screening subject and are partners in ensuring subject safety and appropriate subject ascertainment

• Close collaboration with study sponsors

Pulling It All Together

Incorporation of Signant Health Psychiatric Eligibility Review services combined with its Data Quality Monitoring (DQM) offerings (e.g., eCOA exclusion and safety alerts, edit checks and data quality flags, Central Scoring, Tandem Rating, Central Quality Reviews of administration and ratings associated with endpoint scales, PureSignal Analytics) result in high quality data and mitigation of risks to study success.

Signant Health’s clinical and scientific experts will recommend and customize services to meet your unique study needs. Explore Signant Health’s suite of study optimizing solutions today!

References 

1. Valerie L., et al. (2018). Comorbid mental disorders among adults in the mental health surveillance survey. Annals of Epidemiology, Volume 28, Issue 7, 468-474.

2. Bradford A, et al. (2024). Diagnostic error in mental health: a review. BMJ Quality & Safety, 33:663-672.

3. Schneider, L.S., et al. (2022). Adjudicating mild cognitive impairment due to Alzheimer’s disease as a novel endpoint event in the TOMMORROW Prevention Clinical Trial. J Prev Alzheimers Dis, 9, 625–634.

4. Targum, S.D., & Pendergrass, J.C. (2014). Site-independent confirmation of subject selection for CNS trials: ‘dual’ review using audio-digital recordings. Ann Gen Psychiatry 13, 21.

5. Al-Huthail YR. (2008). Accuracy of referring psychiatric diagnosis. Int J Health Sci (Qassim). 2(1):35-8.

6. Steven D. Targum, et al. (2016). Major depressive disorder with subthreshold hypomania (mixed features): Clinical characteristics of patients entered in a multiregional, placebo-controlled study. Progress in Neuro-Psychopharmacology and Biological Psychiatry, 68:9-14.

7. Corrao, S., & Argano, C. (2022). Rethinking Clinical decision-making to improve clinical reasoning. Front. Med., 9: 900543.