Development and Validation of a Dementia Risk Prediction Model in the General Population: An Analysis of Three Longitudinal Studies.

OBJECTIVE:
Identification of individuals at high risk of dementia is essential for development of prevention strategies, but reliable tools are lacking for risk stratification in the population. The authors developed and validated a prediction model to calculate the 10-year absolute risk of developing dementia in an aging population.


METHODS:
In a large, prospective population-based cohort, data were collected on demographic, clinical, neuropsychological, genetic, and neuroimaging parameters from 2,710 nondemented individuals age 60 or older, examined between 1995 and 2011. A basic and an extended model were derived to predict 10-year risk of dementia while taking into account competing risks from death due to other causes. Model performance was assessed using optimism-corrected C-statistics and calibration plots, and the models were externally validated in the Dutch population-based Epidemiological Prevention Study of Zoetermeer and in the Alzheimer's Disease Neuroimaging Initiative cohort 1 (ADNI-1).


RESULTS:
During a follow-up of 20,324 person-years, 181 participants developed dementia. A basic dementia risk model using age, history of stroke, subjective memory decline, and need for assistance with finances or medication yielded a C-statistic of 0.78 (95% CI=0.75, 0.81). Subsequently, an extended model incorporating the basic model and additional cognitive, genetic, and imaging predictors yielded a C-statistic of 0.86 (95% CI=0.83, 0.88). The models performed well in external validation cohorts from Europe and the United States.


CONCLUSIONS:
In community-dwelling individuals, 10-year dementia risk can be accurately predicted by combining information on readily available predictors in the primary care setting. Dementia prediction can be further improved by using data on cognitive performance, genotyping, and brain imaging. These models can be used to identify individuals at high risk of dementia in the population and are able to inform trial design.


Introduction
Reliable identification of individuals at increased risk of dementia is essential for individualized risk management in both primary and clinical care, but also optimal design of preventive trials (1).
This necessity was aptly demonstrated by the recent findings from large randomized controlled trials that showed potential efficacy of multi-domain interventions to prevent cognitive decline in high-risk individuals (2)(3)(4)(5). The FINGER trial (2) showed that an multi-domain lifestyle intervention resulted in a significant protective effect on cognition. The success of this trial has in part been attributed to the tailored approach of targeting these preventive interventions only to an at-risk segment of the general population (2). This strategy was further corroborated by secondary analyses from the preDIVA trial (3), demonstrating that intensive vascular risk management had the strongest effect among participants with untreated hypertension (3). It is now increasingly recognized that such preventive strategies might be most effective in an at-risk population (3,4,(6)(7)(8).
Several models have been developed to predict dementia in the general population (9), but external validation recently showed that these have limited incremental predictive value above and beyond age (10). These models were mostly based on lifestyle factors, social factors, and comorbidities. So far, models are lacking that include information on markers that reflect the underlying disease process, especially in its early stages. Such markers include subjective memory decline, APOE genotype and neuroimaging (11)(12)(13)(14). On the other hand, such markers are usually not available in a primary care setting. It is therefore conceivable that different models are required, depending on the setting: simple non-laboratory models for a primary care setting and extended biomarker-based models for a clinical setting. Note however, that for purposes of risk stratification in healthy individuals in a primary care setting, models should preferably be based on risk factors that can be obtained without invasive diagnostics such as CSF-sampling or imaging requiring substantial amounts of ionizing radiation such as PET.
Another important consideration is that dementia prediction models should take into account the competing risk of death from other causes, given the generally late-life onset of dementia among community-dwelling individuals. Failure to account for such competing risks inflates apparent dementia risk predictions, limiting the practical utility of currently available models (15).
In this study, we aimed to develop a dementia prediction model for use in a primary care setting and we examined whether an extended model including cognitive, genetic, and imaging markers could improve the performance. Both models were developed while accounting for competing risks.

Study population
This study was embedded in the Rotterdam Study, a prospective population-based cohort study (16). Since 1990, inhabitants aged 55 and older residing in Ommoord, a district of Rotterdam, the Netherlands, were invited. Of the 20 744 invited inhabitants, 14 926 (72%) agreed to participate.
Follow-up examinations take place every three to four years. In addition, a random sample of Rotterdam Study participants was invited for brain MRI in 1995-1996 (N=563). From 2005 onwards, brain MRI became part of the core study protocol of the Rotterdam Study (17). For the current study, we selected participants aged ≥60 years who had baseline data available on clinical, cognitive, genetic, and MRI parameters (Appendix A, Figure 1). We excluded participants who had dementia or incomplete screening for dementia at baseline (N=40), did not provide informed consent to access medical records (N=11), or were no follow-up was available due to logistic reasons (N=35). In addition, we excluded participants without valid imaging data due to artifacts or logistic reasons (e.g. contraindications, or signs of claustrophobia during acquisition) (N=124), or had missing data on APOE carriership (N=134). Therefore, in total 2710 participants were included in analysis for this study.

Candidate predictors
Detailed methods on predictor data collection and predictor definitions are described in Appendix B. We pre-specified candidate predictors based on previous literature, expert knowledge, and availability in clinical practice. For a primary care model, we considered the following candidate predictors: age, sex, level of education, systolic blood pressure, smoking, history of diabetes, history of stroke, presence of depressive symptoms, parental history of dementia, presence of subjective memory decline, and assistance needed with finance or medication. For the extended model, we considered the addition of cognitive tests (Word Fluency Test, Letter Digit Substitution Test, Stroop Interference, and Delayed Word Learning Test), APOE-ε4 genotype, and brain MRI parameters (white matter hyperintensity volume, total brain volume, hippocampal volume, and presence of infarcts [lacunar/cortical]). White matter hyperintensity, total brain and hippocampal volume were all entered into the models as a percentage of intracranial volume to correct for differences in head size.

Assessment of dementia
Participants were screened in-person for dementia at baseline and subsequent centre visits with the MMSE and the Geriatric Mental Schedule organic level (18). Those with a MMSE <26 or Geriatric Mental Schedule score >0 underwent further investigation and informant interview, including the Cambridge Examination for Mental Disorders of the Elderly. The information from in-person screening was supplemented by data from the electronic linkage of the study database with medical records from all general practitioners and the regional institute for outpatient mental health care. In the Dutch healthcare system, the entire population is entitled to primary care that is covered by their (obligatory) health insurance. The entire cohort is thus continuously monitored for detection of interval cases of dementia or cognitive disturbances between centre visits. Study physicians biannually evaluate all records, and combine information from medical records with in-person screening to draw up individual case reports. In these reports, the physicians covered all gathered relevant information to establish the presence, probability and subtype of dementia. A consensus panel led by a consultant neurologist established the final diagnosis according to standard criteria for dementia (DSM-III-R) and Alzheimer's disease (NINCDS-ADRDA). All participants were followed for incident dementia until Jan 1, 2015. Follow-up was virtually complete (97.2% of potential person-years).

External validation
For external validation of the models, we used the EPOZ (Epidemiologic Preventive Investigation Zoetermeer) Study from the Netherlands and the Alzheimer's Disease Neuroimaging Initiative cohort-1 (ADNI-1) from the United States. The EPOZ study started in 1975 and aimed to assess the prevalence of several chronic diseases and their determinants in the city of Zoetermeer, the Netherlands (19). Response rates were similar to those of the Rotterdam Study (72%). Between 1995 and 1996, a random subsample of the participants aged 60-90 years old underwent cognitive testing and brain MRI (N=514) and is considered as baseline for the current study. Participants were screened at study entry and follow-up visits for dementia using a strict protocol (20). All participants were followed for incident dementia until the end of study, on Jan 1, 2007 (completeness of follow-up: 90.8% of potential person-years). For validation within ADNI, we selected 228 cognitively unimpaired individuals aged ≥60 years. Data used in the preparation of this article were obtained from the Alzheimer's Disease Neuroimaging Initiative (ADNI) database (adni.loni.usc.edu). The primary goal of ADNI has been to test whether serial magnetic resonance imaging, positron emission tomography, other biological markers, and clinical and neuropsychological assessment can be combined to measure the progression of mild cognitive impairment and early Alzheimer's disease. Further details on ADNI have been described elsewhere (21).

Statistical analysis
To reduce extreme effects of the predictors, we truncated the distribution of continuous variables at the 1 st and 99 th percentile. Distributions for white matter hyperintensity volume and Stroop Interference score were skewed. We obtained normal distributions of these parameters using a natural logarithmic transformation. We modelled potential non-linear effects of age by using restricted cubic spline transformations and by adding an age 2 term, to capture the effects of age as most important risk factor for dementia most accurately.
For the basic model, we used competing risk regression proposed by Fine & Gray with all candidate predictors included and fitted into the model to calculate 10-year risk of dementia (22).
Appendix C provides further details on the development steps of the model and testing of the assumptions. We subsequently used the least absolute shrinkage and selection operator (LASSO) technique adapted to a competing risk setting to simultaneously penalize the model's regression coefficients and select important predictors for the final model (23,24). The LASSO method is particularly useful to prevent model overfitting and model misspecification (25). An overfitted model tends to underestimate the probability of an event in low risk groups and overestimate an event in high risk groups.
For the development of the extended model, we used the predictors selected by the LASSO technique in the basic model as a starting point and extended it with addition of objective cognitive tests, APOE-ε4 carrier status, and brain MRI parameters. As a reference, we used a model based on age alone for all analyses. In a step-wise, exploratory analysis, we investigated the additive predictive value for each domain separately (cognition, imaging, and genetic information) of the final extended model, compared to the basic model. All presented C-statistics from the development sample represent optimism-corrected C-statistics.

Internal validation
We evaluated the robustness of the model using bootstrap samples for each model and found consistent results in selection steps and coefficient shrinkage using the LASSO technique based on 200 bootstrap samples (Appendix D) (26). We quantified the discriminative ability of these models using the C-statistic for survival data with competing outcomes (27,28). The C-statistic is an adapted AUC-metric for use in survival analyses. It indicates the overall proportion of all pairs of participants that can be ordered such that the participant who developed dementia during follow-up, indeed had a higher predicted risk. We used the cumulative incidence function to calculate the absolute 10-year risk of dementia (29). We used the DeLong test adapted for survival analyses to infer whether C-statistics of the basic and extended models were statistically different from those of a model based on age alone.(30)

Stratified analyses
We assessed the predictive accuracy for the most common subtype Alzheimer's disease specifically and assessed model performance for men and women separately. Next, we excluded the first four years of follow-up to assess whether the predictive value extended beyond the first years of follow-up since some of the predictors may reflect prodromal or undiagnosed dementia. To further investigate model robustness across varying time horizons, we evaluated the predictive ability of the model using a 3-,5-, and 15-year time horizon. Finally, we stratified on age (80 years) at baseline, given the median age of diagnosis (31) and steep increase in incidence of dementia beyond this age in order to investigate the performance of the model at different ages.
Missing data on predictors were imputed using multiple imputation, based on all predictors, outcome status, and follow-up time. All analyses were done using R, CRAN version 3.3.2 (rms, cmprsk, mycrr (27), and crrp (24)).

Study population of the development cohort
Baseline characteristics of the 2,710 participants of the development cohort are shown in Table 1.
The mean age was 71.2 years, 52.8% of the participants were women and 33.3% had subjective memory decline. During a median follow-up of 7.0 years for those who were censored alive (interquartile range: 5.1-9.1), with a total follow-up of 20,324 person-years, 181 participants developed dementia of whom 146 developed Alzheimer's disease and 578 participants died due to other causes. This corresponds to a crude incidence rate for dementia of 9.2 per 1,000 personyears. During the 10-year predicted time horizon, 131 participants developed dementia and 444 participants died free of dementia.

Model development and internal validation
There was evidence for a non-linear relationship between age and the risk of dementia. We therefore added age 2 into the model to capture this non-linearity. The basic model considered age, age 2 , sex, educational level, systolic blood pressure, current smoking, history of diabetes, history of symptomatic stroke, depressive symptoms, parental history of dementia, presence of subjective memory decline, and assistance needed with finance or medication. In Table 2 From here, all presented C-statistics derived from the development study are corrected for optimism to represent optimism-corrected C-statistics.
Adding cognitive, APOE-ε4 carrier status, and imaging information to the basic model resulted in higher discriminative ability (C-statistic 0.86, 95%CI:0.83;0.88). In appendix Table 3, C-statistics are presented when cognitive, APOE-ε4 carrier status, or imaging information are added to the basic model separately. After shrinkage and selection, the Letter Digit Substitution Test, the Delayed Word Learning Test, APOE-ε4 carrier status, and all imaging markers except for brain infarcts were selected (C-statistic 0.86, 95%CI:0.83;0.88). Ten-year risks based on the basic model are easily calculated using a simple risk chart ( Figure 1). An excel appendix is available to calculate risks for the extended model (Appendix).

Stratified analyses
The basic and extended models showed roughly similar results for Alzheimer's disease, and for men and women separately (Table 3)

External validation
Baseline characteristics for Rotterdam Study and EPOZ study participants were largely similar, whereas ADNI-1 participants were older, attainted a higher education, reported less memory decline and more often had a history of parental dementia (Table 1) Given that ADNI is not a population-based study and recruits participants via clinical study sites, we only tested the performance of the full model. This yielded a lower C-statistic of 0.72 (95%CI: 0.63;0.83), reflecting a more homogenous and older population, yet also performed significantly better than a model based on age alone (0.54, 95%CI: 0.42;0.64, p=0.01).

Discussion
In this study, we present a simple prediction model for dementia in an ageing population in primary care. In addition, we demonstrate that this performance can be further extended into a model including cognitive testing, APOE genotyping and brain MRI. These models can be used to calculate the 10-year risk of dementia to inform individuals and optimize risk stratification for clinical trials.
The discriminative ability of our basic model was similar compared to previously published models incorporating data for use in the primary care settings (9). Most previous studies only reported on discriminative ability, ranging from 0.65 to 0.80 as measured with the C-statistic. For instance, the Brief Dementia Screening Indicator using data available in primary care, yielded Cstatistics between 0.68 and 0.78 across four cohorts. Notably, four other prediction models included in a recent external validation study did not provide additional predictive value in dementia risk prediction compared to a model with age as the only predictor (10). In our present study, the basic model we developed did show greater discriminative ability and improved calibration above and beyond age alone. Compared to the Brief Dementia Screening Indicator model, our basic model additionally included the presence of subjective memory decline. The strength of this predictor in relation to the occurrence of dementia (adjusted hazard ratio: 1.65) and the prevalence in the general population (33%), resulted in better predictive performance.
The models in this study include a history of stroke instead of various individual cardiovascular risk factors included in several previous models (9). We did consider traditional cardiovascular risk factors, but these did not pass the mark for inclusion in the final models. Several explanations may underlie these observations. First, almost a quarter of all dementia cases can be attributed to vascular risk factors, illustrating their etiological importance in the development of dementia (18,32,33). However, similar to coronary heart disease prediction in the elderly (34,35), the role of cardiovascular risk factors in dementia prediction may strongly diminish with age. Second, cardiovascular risk factors are also strongly associated with various other diseases at old age, reducing their specific discriminative ability in predicting the occurrence of dementia. For instance, smoking could lead to potentially fatal competing events, such as cardiovascular events or cancer at younger ages and thereby preclude the occurrence of dementia. As a consequence, smoking has limited specificity to predict cardiovascular disease, cancer, or dementia at older ages. Dementia risk prediction models should take into account competing risks to avoid uninterpretable C-statistics and inflated absolute risks (15). We dealt with this issue in the current study by deriving our dementia prediction models within a competing risk framework.
In line with results from a previous model based on predictors derived in a primary care setting (36), our basic model had poor discriminative ability in participants aged 80 years or older. This finding is generally of a limited concern when using a prediction model to identify high risk individuals for clinical trials, since trials generally aim to recruit younger individuals. Yet, these findings provide insight into the complexity of dementia prediction using only clinical parameters in the oldest-old. In contrast, our extended model showed substantial higher discriminative ability for individuals aged 80 years and older, highlighting the significance of objective markers of cognition and brain structure in the oldest-old, including cognitive testing, genetics, and brain imaging.
In this study, we developed and validated two complementary risk models. One basic model that could be used by family doctors and general practitioners, and one extended model that could be used in a clinical setting and that incorporates cognitive testing, brain MRI and genetics. The strength of the extended model is that it uses information that reflects the underlying disease process. At the same time, it can be argued that presence of these markers indicates that the disease is already ongoing and whether it is thus prediction or in fact early diagnosis.
Nevertheless, our sensitivity analyses excluding the first 4 years of follow up showed similar predictive accuracies, suggesting that the effect of early diagnosis as opposed to prediction was marginal. Moreover, the ability to identify persons 10 years before clinical diagnosis can inform trials aimed at intervening in the earliest phase.
Indeed, it is now increasingly recognized that preventive or treatment strategies might be more effective when targeted to individuals at increased risk of dementia (1,(6)(7)(8)(9)37). In order to target such interventions at those who most likely benefit from it, a reliable way to identify individuals at high risk for dementia is needed. The prediction models presented here address this gap, and can be used to stratify individuals in future clinical trials. Absolute 10-year dementia risk thresholds for determining low-and high-risk groups need to be established and may depend on the research question at hand, as well as the availability, costs, and risks of the intervention. These models can be combined in a two-step design, providing opportunities to identify at-risk individuals from the general population with a simple yet predictive model. Subsequently, these individuals can be referred to a clinical setting where a more refined risk assessment can be done using the extended model. It would be interesting to investigate whether the performance of the basic model could be further improved with the addition of a simple blood test (38), or a brief cognitive test, such as the visual association test (39). The extended model could be further improved by adding (1) novel imaging modalities such as cerebral microbleeds or data on diffusor tensor imaging of the brain, by (2) including rare genetic variants, and functional genomics, or by (3) extending models with more in-depth neuropsychological tests (40)(41)(42). The predictive value of other predictors that were available either in the Rotterdam Study or in the validation studies could have been interesting to explore. However, in this study, we specifically aimed to develop a dementia prediction model and subsequently validate exactly that model in these validation studies. Exploring the predictive yield of additional predictors would technically lead to the development or extension of another prediction model, which subsequently would have to be externally validated again.
We should consider limitations of the present study. First, we used a regularization method (LASSO), which automatically selects and subsequently shrinks effect sizes of important predictors. This penalization strategy may have led to some underestimation of predictor effects in the development sample, yet it increases the likelihood of replication in validation studies.
Second, this study focused on older adults of predominantly Caucasian descent (>97%).
Therefore, these models may not generalize to younger individuals or other ethnicities and further validation work in these groups is needed. Third, we developed the models in a population-based setting, which matches the primary care setting, but this will likely affect model performance when validated or used in selected populations seen in clinical care. This was in part reflected by a slightly lower discriminative accuracy in ADNI, yet in addition to differences in case-mix including the homogenous character of this highly selected sample, and a relatively high-attrition rate, discrimination remained substantially better than a model based on age alone. Fourth, we used data on brain imaging with quantitative parameters, which might influence model performance compared with qualitative analyses, such as atrophy and white matter hyperintensity scales. Finally, dementia prediction without an effective therapy at hand raises ethical concerns.
While such models are unlikely to be rolled out into clinical practice before further validation and assessment is undertaken, they have shown to be useful for selecting individuals into clinical trials (2). Strengths of this study include the large sample size and availability of detailed information on a wide selection of potential dementia predictors. Moreover, the basic model is based on questionnaire information and therefore simple to use, and requires no further testing or laboratory measurements. Finally, the models were well validated, both internally and externally.

Conclusions
In this study, we developed and validated a dementia prediction model providing accurate dementia risk stratification and estimation in a general ageing population. Addition of cognitive, imaging, and genetic features improved the predictive ability. These models can be used to identify individuals at high risk for dementia in the general population and might inform future clinical trial design.

Declaration of interests
None of the authors declare a competing interest in relation to this manuscript. The Rotterdam Study has been approved by the Medical Ethics Committee of the Erasmus MC (registration number MEC 02.1015) and by the Dutch Ministry of Health, Welfare and Sport (Population Screening Act WBO, license number 1071272-159521-PG). The Rotterdam Study has been entered into the Netherlands National Trial Register (NTR; www.trialregister.nl) and into the WHO International Clinical Trials Registry Platform (ICTRP; www.who.int/ictrp/network/primary/en/) under shared catalogue number NTR6831. All participants provided written informed consent to participate in the study and to have their information obtained from treating physicians.

Sources of funding
The Rotterdam Study is sponsored by the Erasmus Medical Centre and Erasmus University Rotterdam, The Netherlands Organization for Scientific Research (NWO), The Netherlands Organization for Health Research and Development (ZonMW), the Research Institute for Diseases in the Elderly (RIDE), The Netherlands Genomics Initiative, the Ministry of Education, Culture and Science, the Ministry of Health, Welfare and Sports, the European Commission (DG XII), and the Municipality of Rotterdam. Further support was obtained from the Netherlands Consortium for Healthy Ageing and the Dutch Heart Foundation (2012T008). None of the funding organisations or sponsors were involved in study design, in collection, analysis, and interpretation of data, in writing of the report, or in the decision to submit the article for publication. *Data used in preparation of this article were obtained from the Alzheimer's Disease Neuroimaging Initiative (ADNI) database (adni.loni.usc.edu). As such, the investigators within the ADNI contributed to the design and implementation of ADNI and/or provided data but did not participate in analysis or writing of this report. A complete listing of ADNI investigators can be found at: http://adni.loni.usc.edu/wp-content/uploads/how_to_apply/ADNI_Acknowledgement_List.pdf     year old man or woman without a history of stroke, with subjective memory complaints, and without difficulties managing his or her finance or medication, has a 6 % risk of developing dementia within 10 years.