Clinician Ratings
Common Cognitive Assessments in AD Trials
Cognitive assessments are the backbone of Alzheimer’s disease (AD) clinical trials, providing crucial data on disease progression and treatment efficacy. Among the most widely used tools, the Mini-Mental State Examination (MMSE) and the Alzheimer’s Disease Assessment Scale-Cognitive Subscale (ADAS-Cog) serve as gold-standard measures for assessing cognitive decline.
The MMSE, a quick 30-point screening tool, evaluates key cognitive domains such as orientation, memory, attention, language, and visuospatial skills. Its efficiency and ease of use make it a preferred tool for both initial screening and clinical trial endpoints.
The ADAS-Cog, designed specifically for tracking cognitive changes in AD patients, has long been a cornerstone in research. The original 11-item version, with scores ranging from 0 to 70, is widely used, but a newer ADAS-Cog 13 has been introduced to improve sensitivity in detecting mild cognitive impairment (MCI) and early-stage AD.
Both assessments play an essential role in trials, but ensuring their accurate administration is just as critical as choosing the right scale.
Why Data Surveillance Matters
High-quality data in AD trials depends on rigorous data surveillance programs. At Signant, we use centralized reviews of collected responses, video-recorded written or drawn responses, and audio evaluations of clinical assessment interviews to identify rater errors in time. By identifying deviations, data surveillance ensures assessments remain accurate, standardized, and reliable across sites. This timely approach prevents inconsistencies from skewing trial results.
Some instruments or tasks are prone to mistakes due to complexity, lack of unfamiliarity, or other factors. The MMSE and ADAS-Cog are no exceptions; unique factors make it challenging to standardize the administration and scoring of these instruments.
Challenges in Standardized MMSE and ADAS-Cog Assessments
One major issue with ADAS-Cog is confusion between different versions of the manual. The 1994/1998 version and the 2013 version have distinct instructions and scoring conventions, yet raters who have worked with both often mix them up. Many earlier large-scale AD trials used the 1994/1998 version, while newer studies tend to favor the 2013 version. However, some modern trials still revert to the older version, increasing the risk of administration errors.
The MMSE, in use for over 50 years, presents its own set of challenges. As one of the most widely administered cognitive screening tools, it is routinely used in both clinical practice and research. While this widespread use makes it accessible, it also results in inconsistent administration. Multiple MMSE versions exist, including the official version sold by Psychological Assessment Resources (PAR) and various free adaptations found online. Unlike ADAS-Cog, the MMSE manual lacks detailed scoring guidelines, making it difficult to maintain consistency across different raters and trial sites.
In clinical practice, minor deviations in MMSE administration may not significantly impact a patient’s diagnosis. However, in clinical trials, even small inconsistencies can introduce noise into the data, making it harder to detect meaningful treatment effects. Standardization is crucial—without it, trial results become less reliable.
Uncovering Common MMSE Rater Errors
To better understand and reduce inconsistencies in MMSE administration, we analyzed central review data from two large multinational Phase 3 AD trials. A total of 10,203 MMSE assessments were reviewed, revealing 26.8% flagged for administration errors and 27.0% flagged for scoring errors.
The most frequently observed issues included:
• Administration errors in Orientation to Place (11.8%)
• Administration errors in Attention and Calculation (9.7%)
• Scoring errors in Orientation to Place (9.5%)
• Administration errors in Orientation to Time (7.0%)
Common MMSE Rater Errors and Their Impact
One of the most common MMSE administration errors occurs in Orientation to Place, where raters within the same site apply inconsistent scoring criteria. Each trial site establishes pre-approved correct responses—such as the official name of the building where the assessment takes place—but some raters fail to follow these guidelines, leading to discrepancies. Another frequent issue is providing leading cues, which can unintentionally influence participant responses, compromising data integrity.
In the Attention and Calculation task, particularly the Serial 7s subtest, errors often arise when raters provide feedback or prompts that aren’t allowed. For example, reminding participants of their last response or the number they are meant to subtract fundamentally alters the nature of the task. Since this test is designed to measure independent cognitive processing, any outside assistance distorts its validity.
Similarly, Orientation to Time errors often involve raters giving multiple-choice options rather than allowing participants to recall the correct answer independently.
Another issue is failing to prompt participants to complete responses. For instance, if a subject responds with “hospital” when asked to name the building, they should be encouraged to give the name of the hospital. Ignoring these nuances leads to inconsistent data, ultimately weakening trial reliability.
Uncovering Common ADAS-Cog Rater Errors
Signant Health conducted an internal study in an effort to better understand the most common errors in the ADAS-Cog. We pooled from 14 global dementia clinical trials where the ADAS-Cog was used as an efficacy outcome. A total of 47,238 ADAS-Cog assessments were reviewed.
Findings included the following:
- ADAS-Cog administration and/or scoring errors occurred in 9,288 (19.6%) visits.
- Administration errors were found in 4467 instances (9.46%) and scoring errors were found in 6494 instances (13.75%).
- The items with the largest number of errors were the following:
- Number Cancellation (23.38%)
- Constructional Praxis (20.48%)
- Orientation (12.25%)
- Word Recognition (11.9%)
- Naming Objects and Fingers (10.84%)
Our study identified a substantial prevalence of scoring and administration errors on the ADAS-Cog, which tend to occur independently of one another. This number of flags decreased across flag reviews over the course of the clinical trials, which can be partially explained by ongoing remediation and rater re-training.
The lack of association between administration and scoring errors may be explained by unique scale specificities such as differences in manual versions and unfamiliarity with administration and scoring conventions.
Turning Insights into Action
Our analysis of the central review data provides critical insights that can improve rater training and data monitoring for MMSE and ADAS-Cog.
Effective rater training should go beyond theoretical instruction and integrate real-world data on administration and scoring errors to prepare raters for the challenges they may encounter. Training programs should combine effective tools such as didactic sessions, quizzes, scoring exercises, targeted tip sheets, and/or supplemental materials that highlight common pitfalls and reinforce standardized administration.
Beyond rater training, these insights also play a critical role in central data monitoring. Incorporating knowledge of common rater errors into training materials for central reviewers helps ensure focused attention on specific areas of concern, fosters a calibrated and consistent review process, and enables standardized feedback to raters.
Final Thoughts
Accuracy and standardization in cognitive assessments are critical to the success of Alzheimer’s disease clinical trials. By leveraging data-driven insights to refine rater training and enhance central data monitoring, we can improve the reliability and validity of these assessments, ultimately strengthening the quality of trial outcomes.
About the authors
Marcela Roy is a Senior Clinical Director in Signant’s Digital Health Science department. She has been with Signant for over 15 years and has over 20 years of clinical and research experience. Her focus is Mood Disorders and Endpoint Reliability quality monitoring. She provides strategic direction in the organization, as well as team leadership and business development support.
Sayaka Machizawa, Psy.D., is an Associate Director of Clinical Science at Signant Health, bringing over 18 years of expertise in neurodegenerative and psychiatric diseases. She has played a key role in supporting large-scale global clinical trials across a wide range of indications. Fluent in both Japanese and English, Sayaka has led rater training sessions at numerous Investigator Meetings worldwide.
With a Doctorate in Clinical Psychology, she has also dedicated 12 years to academia, teaching graduate-level Psychology courses, and conducting neuropsychological evaluations for diverse populations. Her extensive experience bridges clinical research, education, and applied neuropsychology, making her a valuable contributor to advancing scientific rigor in clinical trials.
Marta Pereira, PhD, is a Clinical Scientist at Signant Health, specializing in Neurology and Cognition within the Science & Medicine - Digital Health Sciences division. She holds a PhD in Neurosciences from the University of Sao Paulo, Brazil.
Dr. David Miller is a geriatric psychiatrist with over 20 years of clinical, research, and teaching experience. Prior to joining Signant, he served as Chief of Geriatric Psychiatry and Medical Director of ECT at Friends Hospital in Philadelphia, PA. He has been a Principal Investigator in multiple dementia trials and has lectured internationally on dementia research. Dr. Miller co-chairs the ISTAART and ISCTM working groups on neuropsychiatric syndromes in dementia and is a co-author of the updated ADCS ADAS-Cog manual. As Clinical Vice President at Signant, he consults on dementia protocols and has presented at investigator meetings worldwide.
