Annual Research Review: Digital health interventions for children and young people with mental health problems – a systematic and meta‐review

BACKGROUND
Digital health interventions (DHIs), including computer-assisted therapy, smartphone apps and wearable technologies, are heralded as having enormous potential to improve uptake and accessibility, efficiency, clinical effectiveness and personalisation of mental health interventions. It is generally assumed that DHIs will be preferred by children and young people (CYP) given their ubiquitous digital activity. However, it remains uncertain whether: DHIs for CYP are clinically and cost-effective, CYP prefer DHIs to traditional services, DHIs widen access and how they should be evaluated and adopted by mental health services. This review evaluates the evidence-base for DHIs and considers the key research questions and approaches to evaluation and implementation.


METHODS
We conducted a meta-review of scoping, narrative, systematic or meta-analytical reviews investigating the effectiveness of DHIs for mental health problems in CYP. We also updated a systematic review of randomised controlled trials (RCTs) of DHIs for CYP published in the last 3 years.


RESULTS
Twenty-one reviews were included in the meta-review. The findings provide some support for the clinical benefit of DHIs, particularly computerised cognitive behavioural therapy (cCBT), for depression and anxiety in adolescents and young adults. The systematic review identified 30 new RCTs evaluating DHIs for attention deficit/hyperactivity disorder (ADHD), autism, anxiety, depression, psychosis, eating disorders and PTSD. The benefits of DHIs in managing ADHD, autism, psychosis and eating disorders are uncertain, and evidence is lacking regarding the cost-effectiveness of DHIs.


CONCLUSIONS
Key methodological limitations make it difficult to draw definitive conclusions from existing clinical trials of DHIs. Issues include variable uptake and engagement with DHIs, lack of an agreed typology/taxonomy for DHIs, small sample sizes, lack of blinded outcome assessment, combining different comparators, short-term follow-up and poor specification of the level of human support. Research and practice recommendations are presented that address the key research questions and methodological issues for the evaluation and clinical implementation of DHIs for CYP.


Introduction
The past decade has seen a significant increase in the prevalence of mental health problems in children and young people (CYP; Collishaw, 2015), with the worldwide prevalence rate of mental disorders in CYP now estimated to be 13.4% (Polanczyk, Salum, Sugaya, Caye, & Rohde, 2015). With increasing demand on child and adolescent mental health services (CAMHS), the access to evidence-based psychological treatments is severely limited by the supply of trained mental health practitioners, and it is estimated that 75% of CYP with mental health problems in the United Kingdom receive no treatment at all (Davies, 2014).
Digital technology and digital health interventions (DHIs) have been heralded as offering enormous potential as scalable tools to improve outcomes, to widen access and meet the increasing demand on mental health services. Suggested benefits of DHIs include improved uptake and accessibility, efficiency, clinical effectiveness and personalisation of mental health interventions. Given the strength of these claims with respect to health service policy and implementation, it is important that they are tested empirically.
It is commonly assumed that because young people are ubiquitous consumers and users of digital technology for social and recreational purposes, they will be equally enthusiastic recipients of DHIs (Johnson, Fuchs, Horvath, & Scal, 2015). These assumptions may often drive digital health service transformation but are rarely tested. The ability of digital technology to deliver automated and selfdirected interventions is frequently argued as a way to improve access, ease pressures on face-to-face (FtF) services and avoid the reported stigma associated with physical visits to mental health services (Hollis et al., 2015).
The range and scope of DHIs and digitally delivered healthcare services have evolved rapidly since Eysenbach's (2001) initial description of Internet-enabled or computer-enabled interventions. These early DHIs in the mental health field typically contained static content with limited interactivity and were 'fixed' in terms of access (e.g. via a PC or laptop, requiring a wired Internet connection), meaning that users needed to be in a specific location to access the intervention. Examples of these DHIs include computerised cognitive behavioural therapy (cCBT; see Table 1 for a glossary of digital health terminology) which typically mimics FtF-delivered CBT sessions by providing a series of discrete modules that users complete sequentially over a specific time period. Recent advances in computerised technologies and programming has led to the possibility of cCBT interventions becoming more interactive and adaptable for young people though use of gamification and 'serious games' . DHIs also include telecommunications processes (e.g. text messaging, emailing, videoconferencing) to support remote synchronous and asynchronous delivery of therapy (Boogerd, Arts, Engelen, & van de Belt, 2015;Naslund, Marsch, McHugo, & Bartels, 2015;Zulman et al., 2015). These approaches are often referred to as 'tele-health', 'tele-medicine' or 'tele-psychiatry' and fall within the broad description of 'eHealth' (see Table 1).
Over the last decade, the increased popularity and availability of mobile digital technologies, such as smartphones and wearable technologies, has led to the development and evaluation of mobile DHIs, also known as 'mHealth' (see Table 1). mHealth DHIs include smartphone applications ('apps'), remote monitoring and tracking devices, and wearable computers (e.g. smartwatches, virtual reality headsets). Remote active and passive monitoring of parameters, such as mood, activity and sleep, are now being integrated with therapeutic interventions. Hence, the distinction between mHealth digital monitoring and interventions is likely to become increasingly blurred.
DHIs vary widely with respect to design, mode of delivery and the mechanisms through which they aim to change mental health and well-being. Typically, DHIs include content (e.g. educational text, pictures and videos) and/or processes (e.g. games, mood trackers) that relate to the mental health problem being targeted. They can be accessed through different hardware (e.g. laptops, mobile phones, smartphones, wearables) and involve varied levels of interactivity. For example, MoodGym (a free publically available cCBT course) is accessed online through a web browser and consists of five modules relating to understanding and managing depression and anxiety. Originally launched in 2001, Mood-Gym's content is delivered through an eLearning format consisting of text, images, animations, and interactive activities and quizzes (Christensen, Griffiths, & Korten, 2002). An example of a more recently developed DHI, FindMe (a publically available app) is a game designed for children with autism to practice social skills, which can run on a tablet computer regardless of Internet connectivity (Fletcher-Watson et al., 2016).
The number of DHIs aimed at CYP's mental health is growing rapidly (particularly mHealth apps) and Table 1 Glossary of common terms and abbreviations used in the field of digital healthcare (Alkhaldi et al., 2015;Andersson, 2016;Barak, Klein, & Proudfoot, 2009;K€ allander et al., 2013;Podina, Mogoase, David, Szentagotai, & Dobrean, 2016;World Health Organisation, 2011) Term Definition Digital health intervention (DHI) Interventions that provide information, support and therapy (emotional, decisional, behavioural and neurocognitive) for physical and/or mental health problems via a technological or digital platform (e.g. website, computer, mobile phone application (app), SMS, email, videoconferencing, wearable device) eHealth Electronic Health: Internet-based healthcare delivery, or anything health-related that uses information and communications technology (ICT), incorporating computers or Internet in its delivery Internet, online or web-based interventions Usually refers to a computerised program or service delivered through the Internet (e.g. a website), designed to create a positive change in behaviour or health status with varying levels of support (e.g. completely unguided, human-supported) given to user Computer-based or computer-delivered interventions Similar to Internet-based interventions, but usually refers to a program delivered via a computer: the intervention may be via the Internet or an offline computerised program (e.g. CD-ROM, or installed software). Includes psychoeducation and psychotherapy packages, 'serious' games and neurocognitive 'brain training' interventions Computer, Internet-based or mobile-based CBT The delivery of Cognitive Behavioural Therapy (CBT) via computer (cCBT), Internet (iCBT) or mobile devices or applications (mCBT). Collectively may be referred to as electronically delivered CBT (eCBT) mHealth Mobile-delivered Health: A branch of eHealth focusing on delivering healthcare-related information, interventions and monitoring through portable electronic/mobile devices and technologies, such as smartphones, tablets and wearable devices. Examples of mHealth for mental health include smartphone applications ('apps'), text/SMS-delivered interventions and patient monitoring devices Telehealth, telepsychiatry and telemedicine outstrips the capacity of traditional randomised controlled trials (RCTs) to generate evidence of effectiveness. In 2015, mental health apps made up almost a third of disease-specific apps in app marketplaces (IMS Institute for Healthcare Informatics, 2015). As RCTs typically take 5-7 years from initiation to reporting, this time frame is too slow to keep pace with the growth of DHIs. Given the continuing process of improvement and iterations in digital technology platforms and interfaces, it is likely that a DHI may be obsolete by the time a RCT is completed (Schueller, Muñoz, & Mohr, 2013). One possible solution to rationalising the requirement for RCTs for all DHIs is to apply the concept of 'substantial equivalence' as used for medical device and pharmaceutical regulation by the US FDA and similar regulatory bodies. Essentially, if a pivotal trial exists, DHIs meeting criteria for 'substantial equivalence' would not require further RCT evidence. For example, if a pivotal RCT (or meta-analysis) demonstrated effectiveness of a cCBT DHI for depression in CYP, then each subsequent version/iteration of a cCBT DHI for depression would not be required to demonstrate further RCT efficacy and safety evidencebut rather substantial equivalence to existing 'predicate' interventions (FDA U.S. Food & Drug Administration, 2014). If substantial equivalence was established, then the relevant data to collect would then focus on usage, adherence, demographic access parameters and user preferences (Murray et al., 2016).
Reviews of previous research have highlighted the disappointingly poor quality of RCTs of DHIs. In particular, there are major difficulties combining and comparing trials in systematic reviews and meta-analyses without clear specification of the DHI, including the theoretical underpinning of the intervention, mode of digital delivery, level of therapist input and selection of intervention comparator. There have been attempts to standardise and improve reporting of trials in the field with an eHealth version of the Consolidated Standards of Reporting Trials statement ('CONSORT eHealth';Eysenbach, 2011). However, the uptake and impact of this and other reporting standards on the design and conduct of RCTs of DHIs remains to be established.
In addition to establishing the evidence-base for the clinical and cost-effectiveness of DHIs, we need better evidence on usability, acceptability and adherence with DHIs (i.e. do CYP actually want to use DHIs for mental health problems?), whether DHIs actually widen access and how they should best be integrated into mental health services.
In this review, we address the broad question of whether the promise and potential of DHIs has been realised. First, we review the evidence for the clinical and cost-effectiveness of DHIs for mental health problems in CYP by conducting a synthesis of previous reviews and an updated systematic review of RCTs of DHIs in CYP (NCCMH, 2014;Pennant et al., 2015). Second, we identify and discuss the key research questions, methodological and clinical issues related to the future development, evaluation and implementation of DHIs.

Meta-review
Inclusion criteria. We included scoping reviews, narrative reviews, systematic reviews and meta-analyses that focused on the evaluation of DHIs for improving mental health outcomes in CYP. Reviews of interventions (e.g. CBT, self-help) that were adapted for digital delivery (e.g. cCBT as an adaptation of FtF CBT) were included if they reported analyses relating to the digital version of the intervention and included ≥2 studies. Reviews that reported results for adults were included if they separately reported analyses/findings for CYP, and included >2 studies of DHIs in CYP. Included reviews had to be peer-reviewed and in English.
Search strategy. Two authors (BD and CF) devised the search strategy by collating MeSH terms and keywords reflecting: (a) CYP (e.g. child, adolescent, young person); (b) mental health disorders; (c) DHIs (e.g. Internet interventions, apps, eHealth); and (d) the subtypes of review included in the meta-review. The search was run on 11 online databases (Allied and Complementary Medicine, Ovid, MEDLINE, Psy-chINFO, PsychARTICLES, Embase, PubMed, ASSIA, Cochrane Library, CINAHL and Web of Science), and a limited keyword search was also performed on the JMIR Publications database. The search was performed between 16 and 19 November 2015, with a cut-off publication date of 1 November 2015. Reference lists of included reviews were hand-searched for additional publications. Each review's eligibility was assessed by BD through screening citation titles, with uncertainties discussed with CF and full-texts accessed if necessary. The full search terms and process are available in Appendix S1, available online.
Data extraction and synthesis. Data extraction was conducted by BD using a template to collate the review's methodology (e.g. aim of review, date of search), focus (e.g. type of intervention, age group and type of mental health problem), search findings (e.g. number of papers, information about interventions in review) and synthesis of the findings (e.g. descriptive or quantitative synthesis). The data were checked by CF for accuracy. The heterogeneity of the included reviews precluded a meta-analysis of the extracted data, and our findings are presented as a systematic narrative review.
The AMSTAR tool was used to assess the methodological quality of systematic reviews and meta-analyses included in the meta-review (Shea et al., 2007). This tool is an 11-item checklist of questions to appraise review quality: a 'yes' response is given a score of 1, with 'no', 'can't answer' and 'not applicable' responses given scores of 0. This results in scores that range 0-11, with three categories of methodological quality: 0-4 indicate 'low' quality, 5-8 'moderate' quality and 9-11 'high' quality. Following guidance from a previous meta-review (Joyce et al., 2015), an alternate categorisation system was used for systematic reviews without a metaanalysis: 0-3 indicated 'low' quality, 4-7 'moderate' and 8-9 as 'high' quality. Each included review was assessed independently by BD and CF, with any disagreements discussed to reach consensus.

Systematic review
Search strategy. Our review updated a previous systematic review, using the same inclusion/exclusion criteria, search protocol and methodology as NCCMH (2014) and Pennant et al. (2015) (NCCMH, 2014). To be included in the review, participants had to be aged <25 years. Studies that included participants aged ≥18 years were included if either (a) the sample's mean age was ≤18 years or (b) all participants were aged ≤25 years. Additional keywords were added to the search to include recent technological developments in eHealth (e.g. apps, ecological momentary assessment, virtual reality and wearable devices). The updated search identified papers published from June 2013 to December 31, 2015. The full search protocol and terms are available in Appendices S2 and S3.
Study selection and data extraction. Citations from the search were screened for eligibility by CF and categorised as either 'eligible' or 'potentially eligible'. All 'potentially eligible' citations were read in full by CF and BD to identify whether they met full inclusion criteria, with any uncertainties discussed. Data extraction for eligible studies was conducted by CF and BD using a template. This included information about study design (e.g. number of trial arms), the target condition, main intervention evaluated (e.g. theoretical approach, type of technology used to deliver intervention, location of delivery, level of human/therapist support), inclusion/exclusion criteria, type of comparator, sample size, number of participants in each trial arm, participant characteristics (e.g. age, gender, baseline symptomatology), primary and secondary outcome measures, fidelity and adherence measures, key findings and reporting of adverse events.

Findings from search: Meta-review
A total of 1,678 citations were identified through the search. After screening each citation's title, 105 were identified as potentially eligible and the abstract was read. The full texts of 37 articles were reviewed and 21 were selected for inclusion. Figure 1 depicts the search process.
The 21 reviews contained a total of 190 papers focused on the evaluation of approximately 147 unique DHIs. Appendix S4 cross-tabulates the DHIs in the 21 reviews. The majority of the reviews were focused on the clinical effectiveness of DHIs, in particular cCBT (including electronically delivered CBT and Internet-delivered CBT; see Table 1), for anxiety and depression. The reviews are summarised in Table 2.

Methodological quality of included reviews
Two included reviews used a scoping methodology and so were not included in the AMSTAR rating (Boydell et al., 2014;Seko et al., 2014). Using the AMSTAR checklist, the majority of systematic reviews (N = 9) and all reviews with meta-analyses (N = 9) were rated as having 'moderate' methodological quality. Of the remaining systematic reviews, two were rated 'low' and one as being 'high' quality. Nine reviews explicitly stated that they included grey and unpublished literature in their review (Ali et al., 2015;Calear & Christensen, 2010;Clarke et al., 2015;Farrer et al., 2013;Newton & Ciliska, 2006;Podina et al., 2016;Richardson et al., 2010;Rickwood & Bradford, 2012;Ye et al., 2014). Five reviews commented on the proportion of included studies that used intention-to-treat (ITT) analyses (Ali et al., 2015;Davies et al., 2014;Ebert et al., 2015;Farrer et al., 2013;Rooksby et al., 2015). In the two metaanalyses of cCBT for anxiety and/or depression, inspections and analyses of funnel plots suggested some possible publication bias (Arnold et al., 2013;Ebert et al., 2015;Podina et al., 2016), while Davies et al.'s (2014) review of computer and web-based interventions did not appear to find unusual symmetry in funnel plots. Using the trim-and-fill procedures, Podina et al. (2016) found no evidence of publication bias in six studies with cCBT-waitlist comparisons, but one study (out of four) using a cCBT versus FtF CBT comparison design showed a higher-than-expected effect size that did not significantly change the meta-analytic findings. Using the same procedure, Ebert et al. (2015) found that adjusting for missing studies did not result in significant changes upon meta-analysis findings.

Findings from search: Systematic review
Our updated search identified 5,291 citations, reduced to 3,748 after removing duplicates. About 120 were identified as potentially eligible and their full texts were read. Thirty-one publications met inclusion criteria: two were from the same trial, resulting in 30 unique RCTs. These trials evaluated DHIs aimed at improving outcomes in attention deficit/hyperactivity disorder (ADHD; N = 10), autism spectrum disorders (ASD; N = 3), psychosis (N = 1), anxiety (N = 4), depression (N = 6), anxiety and depression (N = 3), eating disorders (N = 2), and posttraumatic stress disorder (PTSD) and depression (N = 1). Figure 2 illustrates this process. A total of 5,273 participants (M = 175.7, Mdn = 89) were randomised across the 30 studies. Table 3 provides a summary of included studies, including the numbers of dropouts in each study. Unless otherwise stated, study results are from ITT analyses.

Clinical outcomes
Anxiety and depression: Meta-review findings.
Twelve reviews focused on anxiety and/or depression, predominantly with modularised cCBT interventions compared to either inactive (e.g. waitlist, no treatment) or active nontherapeutic (e.g. attention) controls.
Six reviews included meta-analyses of anxiety and/ or depression outcomes Ebert et al., 2015;Pennant et al., 2015;Podina et al., 2016;Ye et al., 2014), predominantly comparing the experimental intervention to 'nonactive' or 'nontherapeutic' controls (e.g. waitlist, placebo). Meta-analyses found support for the effectiveness of cCBT in CYP with small-to-moderate effects (g = 0.16-0.62) on depression outcomes, and moderate-to-large effects (g = 0.53-1.41) for cCBT targeting anxiety. Heterogeneity varied considerably across analyses (I 2 range: 0%-92.6%). One review found cCBT interventions were effective for anxiety outcomes in adolescents and young adults (age 12-25 years) but not in children (age 5-11 years; Pennant et al., 2015). Analyses comparing cCBT to an active comparator failed to show superiority of DHIs for anxiety and depression outcomes Ye et al., 2014), while Pennant et al.'s (2015) analysis of two trials supported superiority of FtF CBT over cCBT. Through the use of an evidence-base level criteria tool to classify interventions into different categories of efficacy, Reyes-Portillo et al. (2014) suggests the evidence for effectiveness is strongest for BRAVE-Online and categorises it as a 'probably efficacious' DHI.
Two reviews that compared non-cCBT DHIs (e.g. problem-solving therapy, cognitive bias modification) for anxiety and/or depression found that these interventions had mixed or uncertain effects (Pennant et al., 2015;Reyes-Portillo et al., 2014), with meta-analyses failing to demonstrate superiority of non-cCBT DHIs (Pennant et al., 2015).
In looking at whether parents were involved in the DHI delivery, Podina et al. (2016) comment that as the majority (five of six) of studies using a cCBTwaitlist comparison involved a degree of parental support, it would suggest that parents are needed in order for the intervention to produce positive outcomes. In their meta-analysis of cCBT for depression and anxiety in CYP, Ebert et al. (2015) classified interventions by their level of parental involvement: parents were involved in six (out of 13) studies. Parental involvement was not found to be associated with effect sizes (g = 0.64, compared to g = 0.83 in studies with no parental involvement), suggesting that parental support may not be needed to see Reviewed wider population and did not sub-report CYP findings (N = 5) Reviewed wider population but only reported findings from 1 CYP study (N = 1) Reviewed adult-only studies (N = 1) Type of review not defined in paper (n = 6) Reviewed specific type of therapy/intervention but did not sub-report findings from digital versions of therapy/intervention (N = 2)   For systematic reviews without a meta-analysis: scores 0-3 indicate low quality, scores 4-7 moderate quality and scores 8-9 high quality. For systematic reviews with meta-analysis: scores 0-4 indicate low quality, 5-8 moderate quality and 9-11 high quality. *NB: The AMSTAR tool is used for assessing methodological quality of systematic reviews and meta-analyses only, and so was not performed for scoping reviews.
positive results. Parental involvement may be particularly needed with younger CYP to support engagement (i.e. starting and working with the intervention; Pennant et al., 2015). Three reviews reported results separately for different age groups or reported on age as a moderating variable. The effects of DHIs on anxiety and depression outcomes were greater in adolescents and young adults than in children (Ebert et al., 2015;Pennant et al., 2015;Podina et al., 2016). In a metaanalysis combining both anxiety and depression outcomes, Ebert et al. (2015) found a larger effect size (g = 0.95) with adolescents (aged ≥13 years), compared with children (aged ≤12 years, g = 0.51) and studies combining adolescents and children (g = 0.48). Pennant et al. (2015) found effects for anxiety cCBT interventions to be greater for young people aged 18-25 years than young people aged 12-17 years. However, the 18-to 25-year olds also had higher baseline anxiety scores which may account for larger effects.       indicate that separate analyses were also performed with 'completers' (as defined by authors; usually meant completing all or certain percentage of intervention). Studies marked as 'Unsure' indicate uncertainty about whether ITT analyses were performed due to discrepancies in reporting or insufficient information reported in paper. Yang et al., 2015). Three DHIs used cCBT (Mood-Gym, StressBusters and The Journey; Lillevoll et al., 2014;Smith et al., 2015;Stasiak et al., 2014); one DHI (Project CATCH-IT) incorporated behavioural activation, CBT, interpersonal psychotherapy and community resiliency concept model (Saulsberry et al., 2013); one DHI delivered one-to-one chat room-based Solution-Focused Brief Therapy (SFBT; PratenOnline; Kramer et al., 2014); and one DHI used computer-based attention bias modification (ABM) training (Yang et al., 2015). One thousand three hundred and forty-five participants were included in the six depression trials, with sample sizes ranging from 34  Although participants were randomised to receive one of three types of email-reminder (plus a waitlist control group), it was their own choice to create a MoodGYM user account to access and use the online intervention.
Five trials recruited participants with elevated depression scores (Kramer et al., 2014;Smith et al., 2015;Stasiak et al., 2014;Yang et al., 2015) or persistent subthreshold depression (Saulsberry et al., 2013). One trial was a population-based intervention, where elevated baseline depressive symptoms were not an inclusion criterion (Lillevoll et al., 2014). Severe depressive symptomatology, or a diagnosis of major depression, was an exclusion criterion for five studies. Two of the studies involved interventions accessed at school Stasiak et al., 2014); three evaluated interventions accessed at a time and location chosen by the young person (Kramer et al., 2014;Lillevoll et al., 2014;Saulsberry et al., 2013); and one was accessed in a laboratory setting (Yang et al., 2015).
The level of human and technical support provided to participants varied greatly between DHIs. The RCT of MoodGym involved three trial arms: one condition received no prompts (i.e. completely unguided), while the other two arms received automated emails which were either tailored or untailored to participants' baseline data (Lillevoll et al., 2014). Project CATCH-IT was delivered in conjunction with a one-off FtF meeting with a primary care physician (Saulsberry et al., 2013). Adolescents received The Journey cCBT intervention within school time with 'minimal oversight from school counsellors' . PratenOnline provided online chat-based SFBT that involved secure, one-to-one, synchronous remote live therapy (Kramer et al., 2014). Two studies did not specify support provided with the DHIs Yang et al., 2015).
These recent studies were associated with greater pre-postimprovements in depression outcomes, compared to: waitlist control (Kramer et al., 2014;Smith et al., 2015); no intervention (Yang et al., 2015); a computer-delivered psychoeducational program ; a placebo version of ABM training (Yang et al., 2015); and a group who received the same Project CATCH-IT DHI but with briefer FtF advice from practitioners (Saulsberry et al., 2013). Additionally, Project CATCH-IT reduced hopelessness and self-harming thoughts (Saulsberry et al., 2013), while computer-based ABM training had no effects on ruminations (Yang et al., 2015). As a result of low-take up of MoodGym, Lillevoll et al. (2014) performed 'users versus nonusers' analyses but failed to find any significant intervention effects for depressive symptoms and did not perform ITT analysis.
Four trials reported postintervention follow-up data extending from 3 to 12 months (Kramer et al., 2014;Saulsberry et al., 2013;Smith et al., 2015;Yang et al., 2015). Over a quarter (28.2%) of young people receiving PratenOnline maintained clinically significant change at 4.5-month follow-up, compared to waitlist controls (11.4%; Kramer et al., 2014). Improvements found at 6 weeks postintervention were sustained and increased in both intervention groups at 1-year follow-up for Project CATCH-IT (Saulsberry et al., 2013). Adolescents who used StressBusters self-reported improvements in depression and anxiety from postintervention to 3month follow-up, but not at 6-month follow-up . This study did not perform ITT analysis. For university students participating in laboratory-based ABM training, postintervention reductions in depressive symptomatology were maintained at 3-and 7-month follow-up, but there were no between-group differences at 7-month follow-up (Yang et al., 2015).
Anxiety. Four RCTs evaluated DHIs for anxiety (Shechner et al., 2014;Sportel, Hullu, Jong, & Nauta, 2013;Storch et al., 2015;Vigerland et al., 2016). Two RCTs compared cCBT to treatmentas-usual (TAU; any psychotherapy and/or medication; Storch et al., 2015) or waitlist (Vigerland et al., 2016); one compared ABM training plus CBT, to attention placebo plus CBT, and FtF CBT alone (Shechner et al., 2014); and one compared cognitive bias modification (CBM) training to in-class group CBT and no-treatment control (Sportel et al., 2013). Participants in all four of the RCTs were recruited on the basis of having a diagnosis of an anxiety disorder (Shechner et al., 2014;Storch et al., 2015;Vigerland et al., 2016), or meeting the threshold for high levels of social anxiety and/or test anxiety (Sportel et al., 2013). A total of 496 participants were included, with sample size ranging from 63 to 240 (M = 124, Mdn = 96.5). Studies included both children and adolescents (range 6-18 years), with a mean age range of 9.8-14 years (M = 11.3 years, Mdn = 10.7 years). Attrition at follow-up ranged from 15.3% to 51.1%. Participants completed the intervention at a variety of locations: home (Sportel et al., 2013;Vigerland et al., 2016), a community health centre (Storch et al., 2015) or hospital (Shechner et al., 2014). cCBT was associated with significant improvements in anxiety postintervention and at follow-up, compared to waitlist (Vigerland et al., 2016) and TAU groups (Storch et al., 2015;Vigerland et al., 2016). However, Vigerland et al. (2016) did not perform ITT analysis. The augmentation of FtF CBT with computer-based ABM training significantly decreased the frequency of parent and child-rated anxiety symptoms compared to FtF CBT alone (Shechner et al., 2014). A home-based trial of online CBM training found a significant reduction in social phobia and anxiety scores for both online intervention and group CBT arms compared to no treatment, with posttreatment effects at 6-month follow-up largest for the group CBT intervention (Sportel et al., 2013).

Anxiety and depression. Four studies evaluated
DHIs targeting both anxiety and depression: three evaluated a cCBT intervention (Melnyk et al., 2015;Sethi, 2013;Wong, Kady, Mewton, Sunderland, & Andrews, 2014); the other evaluated a multitheoretical intervention incorporating motivationalenhancement, cognitive behavioural strategies and behavioural principles (Ruggiero et al., 2015). A total of 2,173 participants were recruited to these trials; age range 12-25 years, (M = 17.9, Mdn = 18.5). Participants were recruited with mild-to-moderate anxiety symptoms and/or depression symptoms (Sethi, 2013), or because they lived in an area which had experienced a significant natural disaster (Ruggiero et al., 2015). Participants completed the intervention in school (Wong et al., 2014), or in a community centre or on university campus (Sethi, 2013). The other two interventions were delivered online without specifying a location (Melnyk et al., 2015;Ruggiero et al., 2015).
The findings for the three cCBT interventions were mixed. In a four-arm trial, Sethi (2013) found that the three experimental groups (MoodGym-only, FtF CBT-only, and combination of MoodGym and FtF CBT) reported improvements in depression and anxiety symptoms, but the group who received MoodGym combined with FtF CBT reported the greatest reduction in anxiety symptoms. Melnyk et al. (2015) found that the intervention significantly reduced anxiety but only for those with elevated levels at baseline. However, this study did not perform ITT analysis. In the trial of ThisWayUp (Wong et al., 2014), participants who received the depression-focussed modules had reduced anxiety and depression scores, while those who received the anxiety-focussed modules only improved with anxiety symptoms. ThisWayUp involves FtF group discussions and worksheets to consolidate learning, and so constitutes a 'blended' online and FtF intervention. Evaluation of COPE (Creating Opportunities for Personal Empowerment) found no postintervention differences for anxiety and depression symptoms between the intervention and noaccess control (Melnyk et al., 2015). Finally, a multitheoretical online intervention (Bounce Back Now) for adolescents affected by natural disaster resulted in improvements in PTSD and depressive symptoms at 12-month follow-up (Ruggiero et al., 2015).
Summary: DHIs for anxiety and depression. Depression and anxiety were the most common clinical targets for DHIs in both the meta-review and systematic review. DHIs most frequently cited were MoodGym, BRAVE-Online, Project CATCH-IT, Master Your Mood Online (Grip Op Je Dip) and MobileType. Except for MobileType, these DHIs all provide web-delivered module-based cCBT. While these modularised cCBT DHIs follow the traditional 'sessional' approach to CBT therapy, they are somewhat limited in that they require a sit-down approach to treatment and fail to fully exploit the ubiquitous nature of modern digital technologies. For example, mobile technology (e.g. smartphones, wearables) can accommodate different styles of delivery, learning and collection of patient-centred outcomes, such as ecological momentary sampling and instant access to crisis management strategies. MobileType is a mobile phone-delivered intervention which uses a momentary sampling approach to remotely assess participants' mood, stress, current activity, and alcohol and cannabis use within their natural environment (Kauer, Reid, Sanci, & Patton, 2009). While mood monitoring is not therapeutic in its own right, these extra activities could potentially improve the personalisation of the intervention and support adherence. Increasingly, remote active and passive monitoring of mood is being integrated with therapeutic interventions. Hence, the distinction between mHealth digital monitoring and interventions is likely to become increasingly blurred.
Overall, the strongest evidence of clinical effectiveness comes from DHIs using a cCBT approach, with weaker evidence for the effectiveness of non-CBT DHIs. The largest effects for DHIs are reported in RCTs with: (a) nonactive comparators (e.g. waitlist control) versus active comparators (TAU or attention control), (b) interventions targeting older adolescents and young adults versus children and (c) facilitated therapist-guided support versus self-guided intervention. Most trials recruited participants with mildto-moderate clinical symptoms and excluded young people with severe depression or high suicidal risk. Uptake and adherence was particularly poor for selfguided interventions such as MoodGYM that included automated prompts but no human support. No studies provided data on cost-effectiveness of DHIs.

Eating disorders
Meta-review findings. Two reviews evaluated DHIs for treating and preventing eating disorders (EDs; Newton & Ciliska, 2006;Schlegl et al., 2015). Newton and Ciliska (2006)reported findings from five trials of Student Bodies, an online self-guided intervention including psychoeducational, social learning theory and cognitive behavioural approaches. Four trials were conducted with American undergraduate university students with a mean age range of 19.3-20 years; the other trial was a quasi-experimental study with high-school students (M = 15 years). Meta-analysis found no significant benefit of Student Bodies at postintervention, or at follow-up, for ED-related attitudes, behaviours or body satisfaction. In a systematic review of technological interventions for EDs, Schlegl et al. (2015) reported findings from three studies with adolescents: two cCBT programmes with weekly therapist email support (SALUD BN and Overcoming Bulimia Online) were effective in reducing binging, vomiting and ED psychopathology at postintervention and follow-up. The additional intervention (My Body, My Life), which was facilitated by weekly online group sessions with a therapist, showed moderate prepostimprovements in perceived body image. Two other reviews report findings from DHIs for EDs (Ali et al., 2015;Siemer et al., 2011). Ali et al. (2015) suggested that the inclusion of peer support in Student Bodies had little effect on ED-related attitudes.
Systematic review. Findings from the updated review found low-quality evidence of equivalence between the Student Bodies intervention and waitlist control for ED symptomatology (e.g. binging, purging, restrictive eating) and weight concerns (NCCMH, 2014). We identified two further RCTs of Student Bodies in female university students (N = 216, aged 18-25 years) who were either at high-risk or met criteria for subclinical EDs. Saekow et al. (2015) found no differences (ITT analysis) between intervention and waitlist control for ED symptomatology, weight concern and psychosocial functioning, although significant improvements in these measures were found with a non-ITT analysis of those who completed the entire intervention. Kass et al. (2014) found that participants using Student Bodies with access to the online discussion group had significantly lower weight concern scores than those without the group discussion feature, but no differences were found for ED symptomatology.

Attention deficit hyperactivity disorder
None of the 20 included reviews assessed the effectiveness of DHIs for ADHD in CYP. We identified 10 RCTs for the updated systematic review that evaluated computer-based cognitive training interventions aimed at improving ADHD-related symptoms and behaviours. Interventions included: electroencephalogram ( ; or treatment delivered via videoconferencing (N = 1; Myers, Vander Stoep, Zhou, McCarty, & Katon, 2015). All studies involved participants with ADHD (N = 9) or who met criteria for possible ADHD (Myers et al., 2015). Comparators included; placebo (N = 7), TAU (N = 1), a partially active intervention (N = 1), waitlist control and TAU (N = 1), placebo and medication (N = 1), and a computerised cognitive training program (N = 1). All placebo conditions involved a program that was identical to the experimental intervention but nonadaptive (e.g. it did not increase in difficulty as performance improved). All trials included as a primary outcome measure parent and/or parent, caregiver or teacher-rated ADHD symptoms.
The majority of trials recruited children (M = 9.98 years, Mdn = 9.82), with seven including only children aged ≤12 years. Exactly, 861 participants were randomised, with sample sizes ranging from 39 to 223 (M = 86, Mdn 80). Reported attrition was small across all studies. Three studies delivered the intervention at home (Chacko et al., 2014;Dongen-Boomsma et al., 2014;Dovis et al., 2015), one in a clinic (Dongen-Boomsma et al., 2013) and two in school (Egeland et al., 2013;Steiner et al., 2014). Excluding the RCT evaluating a videoconferencing service (Myers et al., 2015), seven studies included in-person or telephone-based support from a training aide (e.g. parent, guardian, teacher; N = 2) or a professional (e.g. certified coach, therapist, research assistant; N = 5). The other two trials failed to report whether support was provided (Arnold et al., 2013;Li et al., 2013).
EEG-based NFT aims to improve ADHD symptoms through training designed to suppress EEG theta wave activity and increase beta wave activity. All four RCTs of NFT report mixed findings for effectiveness. Two studies found that both the NFT and attention placebo groups reported pre-postimprovements in parent and/or investigator and/or teacher-rated ADHD symptoms, but there were no differences between the two groups (Arnold et al., 2013;Dongen-Boomsma et al., 2013) and the study by Arnold et al. (2013) did not perform ITT analysis. CYP who received NFT in addition to methylphenidate medication showed significant improvements in parentrated ADHD symptoms and social functioning, compared to a control group who received attention placebo plus methylphenidate medication (Li et al., 2013). However, this study did not perform ITT analysis. Finally in comparison to children who received a cognitive training intervention, children who received the NFT intervention reported better parent-reported executive functioning, behaviour regulation and metacognition outcomes, and teacher-reported attention and inattention outcomes (Stangier, 2016).
Working memory training and EFT aim to improve specific or wider deficits in cognitive functioning and attentional skills (Melby-Lerv ag & Hulme, 2012). The three RCTs of WMT report contrasting results. Chacko et al. (2014) found that WMT participants showed greater improvements in verbal and nonverbal memory compared to attention placebo, with both groups reporting pre-postimprovements in parent-reported ADHD symptoms, but no differences between the two groups. Egeland et al. (2013) reported no changes over time or between the WMT and waitlist groups in parent-and teacher-rated ADHD symptoms, but the WMT group did report improved mathematics and reading skills. Egeland et al. (2013), however, did not perform ITT analysis. Finally, Dongen-Boomsma et al. (2014) found that the WMT group showed significantly greater improvements than attention placebo in one verbal working memory task, and while both groups reported improvements over time in parent-and teacher-rated ADHD symptoms, there were no statistically significant differences between the groups. However, again, this study did not perform ITT analysis. An intervention targeting multiple aspects of executive functioning found ADHD symptoms improved over time regardless of the intervention received (Dovis et al., 2015). A trial of a videoconferencing telehealth service for remote treatment of ADHD showed that both the videoconferencing and control service improved teacher-and caregiverrated ADHD symptoms, with greater improvement in the intervention group (Myers et al., 2015).
Summary: DHIs for ADHD. Computerised cognitive 'brain training' programs for ADHD include WMT, EFT and EEG NFT. To date, the results of trials have been inconsistent, with no overall differences reported between DHIs and active placebo interventions. The negative NFT findings align with recent meta-analyses suggesting that NFT cannot be currently recommended as treatment for ADHD (Cortese et al., 2016). In previous reviews, the largest effects for nonpharmacological interventions on ADHD symptoms were found for outcomes reported by parents who may be 'unblinded' to intervention allocation where there are nonactive comparators. Similarly, in our review treatment, effects of DHIs are attenuated or nonsignificant for ADHD outcomes reported by independent observers (e.g. teachers), who are more likely to be 'blinded' to intervention allocation (Cortese et al., 2015). Given that face-toface nonpharmacological interventions for ADHD have not demonstrated efficacy, it is perhaps not surprising that digital versions would also not be effective. In summary, the results of the updated systematic review suggest that DHIs (including WMT, EFT and EEG NFT) cannot be recommended for the treatment of ADHD.

Autism spectrum disorders
Only one review identified a study of a DHI for CYP with ASD (Jang et al., 2012;cited in Boydell et al., 2014): their evaluation of an eLearning training intervention for family members of children with ASD found those who received the intervention reported greater improvements in skills and knowledge relating to applied behaviour analysis.
We identified three RCTs of DHIs for CYP targeting ASD symptoms/impairments: a mobile phone 'app' for practising and improving communication skills (FindMe;Fletcher-Watson et al., 2013); an interactive computerised software program to improve recognition of emotion in facial and vocal expressions (MindReading; Thomeer et al., 2015) and a computer-based program to improve working memory (WM) and cognitive flexibility (CF; Braingame Brian; Vries, Prins, Schmand, & Geurts, 2015; N = 218, M age 7.8 years). Findings from these studies are mixed. Fletcher-Watson et al. (2016) reported no differences between experimental (FindMe) and waitlist and TAU combined groups for ASD symptoms at postintervention and follow-up, although parents gave positive feedback about their child's enjoyment of the intervention. Compared to waitlist control, those who used the MindReading program showed significant improvements with a large effect size in social functioning, face and voice recognition at posttreatment and at 5-week follow-up (Thomeer et al., 2015). However, this study did not perform ITT analysis. All participants using Braingame Brian improved in WM, CF, attention, social functioning, quality of life and ADHD-related behaviour (Vries et al., 2015).
Summary: DHIs for ASD. DHIs for CYP with ASD are designed primarily for preadolescent children and often incorporate computer game-based formats aimed at training and improving core deficits in social understanding, empathy and emotional recognition. While these games appear popular (particularly with parents), the results of trials have failed to show consistent benefits that transfer outside the specific context of the game to affect core ASD symptoms and deficits. Less attention has been given to DHIs which target associated symptoms and behaviours in ASD such as anxiety and challenging behaviour. Given that these symptoms may be more amenable to intervention than core deficits of ASD, and emerging trials show that these interventions work in FtF delivery digital adaptations would be welcome.

Psychosis
None of the 21 included reviews assessed the clinical effectiveness of DHIs for psychosis in CYP. Our systematic review identified one study of Captain's Log, a computer-assisted cognitive remediation (CACR) program for adolescents with, or at risk of, psychosis (Urben, Pihet, Jaugey, Halfon, & Holzer, 2012). This intervention aims to train attention, concentration, memory, and visuospatial and visuomotor skills. Postintervention, both experimental and placebo control groups reported a significant improvement in attention, memory processing, general psychopathology and social functioning, with the intervention group reporting significantly greater improvement in visuospatial abilities compared to control (Holzer et al., 2014). However, there were no differences in WM, executive functioning, psychotic symptoms and psychosocial functioning at 9-week and 6-month follow-up.

Tele-psychiatry
Meta-review findings. Two reviews examined the effects of tele-psychiatry or tele-medicine for CYP (Boydell et al., 2014;Hailey et al., 2008) with telecommunication technology (e.g. telephone, videoconferencing) used to either deliver mental health treatment or to remotely diagnose mental health disorders. Descriptive reviews suggest that delivering remote services via telephone and videoconferencing is acceptable to healthcare practitioners, CYP and their families. However, studies to date have failed to report the impact on service access for 'hard to reach' groups, treatment adherence and clinical effectiveness, suggesting an important gap in the research literature.
Systematic review findings. Myers et al. (2015) evaluated a videoconferencing intervention delivering six sessions of pharmacotherapy and in-person caregiver behaviour training for children with ADHD. Both the videoconferencing group and control group (who received only one videoconferencing consultation) reported improvements in teacher and caregiver-rated ADHD symptoms, with the intervention group (who received more clinical contact) reporting significantly greater improvement. Unlike DHIs designed as interventions for specific conditions, tele-psychiatry/telemedicine refers to a generic telecommunications platform used to deliver remote assessment, monitoring and treatment by healthcare professionals across a range of conditions and interventions. Overall, remote videoconferencing appears acceptable to those recruited into studies. However, there is a notable lack of evidence of cost-effectiveness and whether this technology increases access to services for previously excluded groups.

Experience of using DHIs: Eliciting views of young people and parents
We identified nine studies that explicitly sought participants' feedback about their experience and satisfaction with the DHI, either through administering a quantitative survey and/or a qualitative approach. Feedback about satisfaction was gained from the CYP participants (Melnyk et al., 2015;Saekow et al., 2015;Stasiak et al., 2014), the CYP participant and their parents (Arnold et al., 2013;Storch et al., 2015;Thomeer et al., 2015;Vigerland et al., 2016) or from the CYP's parents only (Fletcher-Watson et al., 2016;Steiner et al., 2014). CYP and parents reported moderate-to-high satisfaction with the DHIs (Arnold et al., 2013;Stasiak et al., 2014;Storch et al., 2015;Thomeer et al., 2015;Vigerland et al., 2016), with qualitative feedback being generally positive about the DHI and its potential to help with mental health and well-being, and also provided suggestions for future improvements in designing DHIs (Fletcher-Watson et al., 2016;Melnyk et al., 2015;Saekow et al., 2015;Stasiak et al., 2014).

Adherence to DHIs and association with outcome
None of the studies in the meta-review addressed issues around dose-response, such as how much of the intervention is needed to produce beneficial outcomes. Given the current challenges of the field (e.g. difficulties with establishing a DHI taxonomy), it is still difficult to know exactly what amounts to an appropriate (or 'minimum effective') dose of an intervention.
Similarly, only four (out of 30) papers in the updated systematic review provided information about the minimum effective 'dosage' of their DHI or levels of adherence. In one trial of WMT for ADHD, a 'complier' meant the participant completed ≥20 training sessions (out of a possible 25 sessions; Dongen-Boomsma et al., 2014), while in a trial of EFT for ADHD, 'compliers' were defined as those who completed all 25 training sessions (Dovis et al., 2015). Lillevoll et al. (2014) categorised level of adherence of MoodGym into three categories: nonparticipation, one module only, and two or more modules. Finally in the evaluation of a web-based intervention for disasteraffected adolescents and their families, Ruggiero et al. (2015) defined a 'completer' as a participant (adolescent or parent) who completed ≥1 intervention module.
Furthermore, seven studies reported associations between adherence/dosage and outcomes. Four of these reported no associations between adherence or level of intervention completion and outcome (Ruggiero et al., 2015;Saekow et al., 2015;Steiner et al., 2014;Vigerland et al., 2016). In one trial of WMT for ADHD, participants who did not complete WMT were more likely to score higher on inattentive and hyperactive/impulsive measures (Dongen-Boomsma et al., 2014), while spending longer amounts of time engaging with a CACR program for psychosis was associated with greater gains in attention (Holzer et al., 2014). Finally, findings from the evaluation of the FindMe app for children with ASD found no associations between time spent engaging with the app and autism-related behaviours, but a negative correlation between game play, visual perception and motor scores was found upon removal of an outlier (Fletcher-Watson et al., 2016).

Discussion
Our review of the effectiveness of DHIs has focused on evidence from RCTs, with the majority conducted on cCBT targeting depression and anxiety in adolescents and young adults, with far less research focused on other clinical areas and therapeutic approaches or mobile-enabled (mHealth) technologies.
Overall, there is some support for the role of cCBT in improving symptoms of depression and anxiety in CYP. There is also evidence from 'head to head' trials that therapist-guided (remote) cCBT is as effective as FtF CBT (Sethi, 2013). However, existing trials have focused on older adolescents with mild/moderate symptomatology and, as a result, it is still not clear whether DHIs are useful for CYP who present with more severe symptomatology typically found presenting to mental health services. Trials with active comparators show less benefit of DHIs than those with nonactive controls. There is some evidence that human support, be that in the form of a therapist's guidance or researcher contact, may be beneficial in terms of adherence and effectiveness. DHIs for ADHD, ASD, eating disorders, psychosis and PTSD show uncertain benefits. Importantly, there is a notable lack of evidence concerning the cost-effectiveness of DHIs.
Overall, the heterogeneity of DHIs and poor quality of many studies make it difficult to draw definitive conclusions about the effectiveness of DHIs and the role they should play in mental health services for CYP. Our review highlights a number of important research questions and methodological issues that need to be considered for the field to move forward.

Research priorities and methodological issues
Obtaining evidence of cost-effectiveness. There is a remarkable lack of data on the cost-effectiveness of DHIs in CYP. This is surprising given the promise that DHIs can increase health service efficiency through the ability to deliver effective interventions at scale with minimal incremental costs. Several of the reviews in our meta-review mention the limited information about cost-effectiveness or even how DHI costs compare to usual mental healthcare and treatment. Many DHI trials include some level of human support, but the costs of this compared to usual treatment is not known. Boydell et al. (2014) note that the widely held assumption that DHIs are more affordable and associated with lower costs, more ease of administration and reduced therapist time has not been substantiated to date. There are several factors that influence the calculation of costs in delivering DHIs, such as where the DHI is delivered (e.g. 'internally' within mental health services; Palmqvist, Carlbring, & Andersson, 2007), the associated level of support given to users and who 'owns' the DHI. For example, some Internet-delivered DHIs are commercialised and have to be bought in by individuals or health service providers, while others are free to use and are publically available (e.g. MoodGym; Gilbody et al., 2015). We emphasise the need to consider sustainability and cost-effectiveness from the beginning of DHI development. The development phase for a DHI should include consideration of the long-term costs of maintenance and updating, how these costs could be met, and who will take responsibility for them.
The role of human support in DHIs. A critical research question in the design, evaluation and implementation of DHIs relates to the use of human support and how this affects engagement with the intervention and clinical outcomes. Across trials of DHIs, the level of human support or facilitation is poorly specified, which obscures the effect of human support on engagement/adherence and outcomes. Between different DHIs, the level of human support varies in terms of who is providing it (e.g. a trained layperson, parent, teacher or clinician), the degree of support provided (e.g. unguided, semiguided, fully guided), its purpose (e.g. to provide encouragement, to check for technical issues or augment therapy) and the uniqueness of the support (e.g. tailored support for one user or automated support to all users). These factors will all influence CYP's motivation and continued engagement with the intervention, and providing some sort of human or therapist support, even at a minimal level, has been previously identified as a significant moderating factor influencing therapeutic outcomes and engagement (Rickwood & Bradford, 2012). This is important for policy because DHIs are often promoted, incorrectly in our view, as a low-cost alternative to FtF services due to their automated delivery with very low or zero incremental costs.
Six adult-only studies found a positive relationship between adherence and receiving support during online interventions, with qualitative findings suggesting that participants were less likely to adhere if they had limited human contact (Beatty & Binnion, 2016). Findings from 25 RCTs of mostly adult samples evaluating cCBT for depression show larger intervention effects as the degree of human contact increased: no therapist contact, d = .21; contact before treatment only, d = .44; contact during the treatment only, d = .58; and contact before and during treatment, d = .76 (Johansson & Andersson, 2012). However, meta-analyses of cCBT with CYP report mixed impact of support upon depression and anxiety outcomes (Farrer et al., 2013;Pennant et al., 2015;Podina et al., 2016). Larger effect sizes were found for studies involving 'minimal' therapist input, compared to studies with 'significant' input in cCBT for anxiety (Podina et al., 2016). Ebert et al. (2015) found a larger effect size in cCBT studies that had no parent involvement (g = 0.83) compared to those with parent involvement (g = 0.64). In evaluating various technologies upon mental health outcomes in university students, Farrer et al. (2013) found no association between outcomes and amount of human contact provided to participants. Hence, an important research question concerns the impact of human support on both adherence and outcome for DHIs in CYP and whether the effects differ (and if so, why) from the consistent findings reported in adults.
Young people have expressed a need for some level of support in receiving DHIs or to use it in conjunction with FtF therapy Mitchell & Gordon, 2007;Pretorius, Rowlands, Ringwood, & Schmidt, 2010). Support may not have to be a 'real' person, but could be automated through the DHI itself. When asked about their 'ideal app' for managing their condition, young people and adults with ADHD highlighted the need for a virtual 'coach' or 'mentor' to provide support and encouragement (Simons et al., 2016). Developments in virtual reality, artificial intelligence and machine learning are creating 'virtual human' agents that, in the next generation of DHIs, could act as automated, interactive coaches to support personalised delivery of DHIs (Valstar et al., 2014).
Choosing appropriate comparators. The range of comparators used across trials of DHIs range from active digital and nondigital (e.g. attention control and FtF CBT) comparator interventions versus nonactive controls (e.g. waitlist). In general, effects are largest when DHIs are compared to nonactive controls, and differences are smallest when there is an active comparator.
The selection of a suitable comparator is determined by the research question addressed. In pragmatic trials that aim to determine the effectiveness of a new DHI compared to current best practice, the comparator is typically TAU. However, in trials of DHIs, the participants in the TAU group may have access to a range of other digital interventions that may be hard to prevent or track (e.g. online psychoeducational material), but risks undermining the results of the trial. In contrast, 'active' comparators control for nonspecific effects of the intervention package such as human support, attention and online usage. It is critical to understand if human support is important only for increasing engagement and adherence or is an active component in therapeutic change. If the latter is true, then an 'active' control with human support may obscure the true effect to the DHI.
Identifying active components of DHIs. Understanding which components of a DHI actually have the predicted impact on the outcome, and whether and how components interact, is critical to DHI development and evaluation. Most DHIs are highly complex, containing multiple components, so the development process needs to include a period of optimisation. This entails evaluating the performance of individual components of the intervention, and how they interact with one another. One efficient method is the multiphase optimisation strategy (MOST; Collins, Nahum-Shani, & Almirall, 2014), which involves establishing a set of components that are candidates for inclusion, specifying an optimisation criterion for the entire intervention, and then collecting experimental data to identify the subset of components that meet the criterion. Here the term 'component' is broadly defined, and may refer to aspects of the content of the intervention, including any human input; factors affecting engagement, adherence to, fidelity of or scalability of the intervention including the type of technical platform and presentation features such as gamification; variables and decision rules used to tailor intervention strategy, content or intensity to individuals; or any other aspect of an intervention that can profitably be separated out for examination (Murray et al., 2016).
The experimental approaches that can be used for optimisation include full or fractional factorial experiments (Collins, Dziak, & Li, 2009), the sequential multiple-assignment randomised trial (SMART; Almirall, Nahum-Shani, Sherwood, & Murphy, 2014) and system identification techniques (Rivera, Pew, & Collins, 2007). The factorial experimental design can be a useful and economical approach for examining the effects of individual intervention components, and is the only experimental design that enables full examination of all interactions.
Towards a taxonomy of DHIs. It is clear that the content of DHIs, their underpinning theory of change and their mode of delivery will affect the impact of a DHI, yet in most studies these components are not specified or analysed separately. This makes it difficult to judge whether a positive (or negative) outcome of a trial is the result of: the intervention content and theory of the change, the digital delivery platform or an interaction between the two.
Lack of clarity and precision in the terminology used to describe components of DHIs make it difficult to group interventions and identify active components; a situation that is likely to become even more complex as technology develops. An agreed working taxonomy of digital mental health interventions, similar to that developed for behaviour change interventions (the Behaviour Change Technique/BCT Taxonomy Project; Abraham & Michie, 2008), is required to enable interventions to be appropriately categorised and analysed. In addition, adherence to the CONSORT eHealth guidelines (Eysenbach & CONSORT eHealth Group, 2011) would make it easier to take these factors into consideration when comparing individual studies.
DHIs that use psychotherapeutic theory as their theoretical basis (e.g. CBT) often do not describe which features of the theory are being employed in the intervention. In a recent review of CBT and behavioural activation (BA) apps for depression (Huguet et al., 2016) produced a checklist of the 'core ingredients' involved in CBT and BA approaches (e.g. challenging negative thoughts in CBT; activity scheduling of pleasant and avoidance behaviours in BA), which enabled them to identify the 'ingredients' available in apps. Similarly, digital interventions with gaming features (often called 'serious games') use specific gaming elements in their delivery, such as having a storyline and setting rules, goals and objectives . It may be useful to apply a similar BCT Taxonomy approach to DHIs so that clinicians and the public can clearly see how these interventions aim to produce therapeutic change and researchers can judge their effectiveness.
Related to this issue is the large number of different digital interventions being studied, all of which differ to a greater or lesser degree according to their purpose, content, theory of change, presentation interface and mode of delivery. As a result, it is difficult for DHIs to undergo the incremental innovation seen in other areas of healthcare. An agreed taxonomy for specifying the components of DHIs is required for replication of trial results, comparison between DHI, synthesising data across trials in systematic reviews and meta-analyses.
Tailoring and personalisation of DHIs. In health behaviour change interventions, 'tailoring' typically refers to how targeted the health messages being sent to users are. For example, 'generic' communication reflects messages that are not individualised to the recipient's specific needs, but this may be 'personalised' by adding a user characteristic to the message (e.g. the user's name). 'Targeted' communication is used to provide messages to a specific group, such as those of a specific age or who screen for a specific risk of developing a health problem (Musiat, Hoffmann, & Schmidt, 2012;Noar, Benac, & Harris, 2007). Our review found similar approaches for mental health DHIs. For example, some received 'personalised' feedback in the form of information based on an assessment or data that was entered into the intervention, which can vary in its degree of personalisation (Barak & Grohol, 2011;Musiat et al., 2012). Tailoring may also mean that an intervention has different user 'pathways' depending on, for example, the user's baseline symptoms (e.g. Chiauzzi, Brevard, Thum, Decembrele, & Lord, 2008), or allows users to choose the modules or content that is most relevant to their presenting problem (e.g. Andersson, Estling, Jakobsson, Cuijpers, & Carlbring, 2011).
Research has shown that users want this type of tailoring or personalisation. Adults with experience of using computerised therapies reported a desire for DHIs to be responsive to the 'self' (e.g. sensitive to their clinical needs, feelings and personal preferences; Knowles et al., 2014). The research team responsible for the SPARX DHI also provides examples of good practice in this area. In focus groups with rural Australian adolescents , 'personalisation' of SPARX was a key theme that emerged. This reflected two separate aspects valued by adolescents. The first centres on their personal choice to use the intervention (e.g. where and when to use it) and who to share their feelings and the intervention with (e.g. with a counsellor or adult). The second centres on the personalisation of the intervention, which focused mainly on the ability of the user to choose the gender of the 'guide' avatar, which they reported led to improved relatability. This aspect was also highlighted and praised by sexual minority youth, who felt that allowing them to personalise the avatar's gender and appearance reflected their real-world experience of challenging gender expectations (Lucassen et al., 2013). Furthermore, the research team have analysed qualitative data from five studies of SPARX to understand how users' perceptions and elements of the intervention map onto the autonomy, competence and relatedness aspects of self-determination theory, which in turn have been theorised as key factors influencing engagement and adherence to computerised interventions (Cheek et al., 2015).
Although the research on tailoring and personalisation for DHIs and the benefits that may result from this is limited at present, it does suggest that the ability of a young person to personalise relatively small features (e.g. the gender and appearance of a 'guide') may have an impact on how users view and relate to a DHI. Further research is required to explore how DHIs should be tailored and how this influences uptake, adherence, satisfaction and outcomes.
Privacy and security issues. Privacy and security are important concerns when handling and managing health-related data (Hollis et al., 2015), as users may be concerned about where their information goes, how it is stored, and who it is shared with. For example, when installing a new app on a smartphone, the user will be asked for permission to access or read specific phone functions, and some users may be concerned about why an app has to access such information. When using Internet-enabled technologies, adolescents and young people have expressed concerns about data protection and security, and the information they chose to share with the technology (Ring, 2014). In seeking user opinions about developing an online mental health clinic, university students stated that data security in online peer-to-peer interactions was paramount, and anonymity may help in maintaining trust and privacy of access (Farrer, Gulliver, Chan, Bennett, & Griffiths, 2015). Users have to make the decision to trust apps based on the information provided to them, and so transparency is needed to help users understand how ethically their data are managed and stored (Huckvale, Prieto, Tilney, Benghozi, & Car, 2015). Understandably, these concerns are also applicable to other non-app DHIs, and research is needed to gather CYPs' opinions, concerns and requirements regarding data security and privacy of DHIs.
Do children and young people prefer digital health interventions? Bradford and Rickwood (2014) found no evidence to confirm the assumption that young people in Australia prefer digital or Internet-delivered help over FtF or phone-based services. While young people expressed some positive attitudes towards DHIs, overall they had a strong preference for FtF help (59%), with only 16% expressing a preference for online treatment (lower than the number of CYP who reported that they would not seek help at all). CYP reported that the perceived benefits of FtF help (e.g. more personal, can see who they are talking to, customised feedback) are valued as important when receiving mental health treatment. Furthermore, in a study of young people attending a UK CAMHS clinic  found that half were not interested in cCBT, preferring to talk to someone FtF. These somewhat discouraging findings suggest that the common assumption that DHIs are the preferred form of intervention and service contact for CYP may be unfounded, or at least over simplified and would benefit from further exploration using qualitative research methods.
Qualitative research to date has shown that CYP have a number of concerns about DHIs that could prevent uptake and adherence. Some of these issues are applicable to mental health services in general and are concerned with stigma, embarrassment and shame (Clement et al., 2015). Other factors are specifically related to accessing digital interventions, and include: reduced motivation to engage in cCBT without reinforcement; inadequate access to information about DHIs and their effectiveness; lack of technological access; the belief that DHIs may be impersonal with limited interaction; lack of tailoring to their specific presenting problems; and acceptability of DHIs for CYP at different ages and developmental stages (Fleming, Dixon, Frampton, & Merry, 2012;Lal et al., 2015;Lucassen et al., 2015;Mitchell & Gordon, 2007;Pretorius et al., 2010;Richards & Timulak, 2013).
Despite identifying their limitations, CYP also report perceived benefits to using DHIs. Research with young people both with and without experience of using DHIs has identified positive attributes of DHIs, including privacy and anonymity, flexibility, reduced pressure and ability to complete interventions on their own terms, facilitating self-management, and being experienced as less 'intense' than counselling (Bradley, Robinson, & Brannen, 2012;Fleming, Lucassen, Stasiak, Shepherd, & Merry, 2016;Mitchell & Gordon, 2007;Pretorius et al., 2010;Richards & Timulak, 2013;Simons et al., 2016). Young people value information, accessibility, self-reliance and control when accessing mental health services (Plaistow et al., 2014). DHIs are usually considered 'more accessible', and online resources are considered a way of enabling young people to have control, privacy and independence when accessing mental health resources .
While these qualitative findings are encouraging and useful, further qualitative research is required to investigate CYP's perceptions of specific DHIs including the way in which they have been implemented (e.g. within a research framework, services or 'in the wild'), their perceived mechanisms of change (positive and negative) and recommendations for improved engagement and effectiveness.
How do healthcare professionals view the role of DHIs?
The majority of research investigating the views of mental health professionals (MHPs) has focused on cCBT. There is a consensus among MHPs that FtF therapy is superior to cCBT, despite the evidence (reported in this review and others) that cCBT can be as effective as FtF therapy for depression and anxiety in CYP, at least in the short term and for mild-tomoderate symptoms. The consequence of this is that MHPs tend to believe that DHIs should not be widely and freely available online, and should not be delivered without professional support Vigerland et al., 2016). This view is consistent with the adult cCBT literature (Perle et al., 2013) and evidence that MHPs believe DHIs are best offered as an adjunct FtF therapy (Donovan, Poole, Boyes, Redgate, & March, 2015;Perle et al., 2013;Simons et al., 2016;Sinclair, Holloway, Riley, & Auret, 2013;Vigerland et al., 2016). There is also some empirical support for this approach suggesting that a combination of cCBT and FtF CBT may be superior to either delivered alone (Sethi, 2013). Interestingly, this view contrasts with the design of most clinical trials which typically evaluate DHIs in isolation from other services and interventions. MHPs have expressed concerns about DHIs in terms of a lack of therapeutic alliance and professional support, lack of tailoring to individual needs, lack of formulation taking into account CYP's family and school context, and difficulties in assessing progress due to a lack of clinical review (Simons et al., 2016;Vigerland et al., 2016).
MHPs perceive the main benefits of DHIs as offering increased availability and access to psychological help 'anytime, anywhere', digital technology being an appealing medium for CYP (particularly those who struggle to talk about their feelings), the benefits accompanying self-help such as increased self-confidence and self-awareness, and reduced stigma (Simons et al., 2016;Vigerland et al., 2016).

Do DHIs widen access to mental health services for CYP?
In considering the potential for DHIs to widen access to services, it is also important to consider CYP's attitudes towards DHIs. In receiving mental health services and making decisions about treatment, CYP have stated a need for accessibility, self-reliance and control (Plaistow et al., 2014), and adolescents have stated that cCBT allowed for more control (Fleming et al., 2016). DHIs are usually considered more accessible through their virtue of being accessed remotely via technology, at a time and place of the young person's choosing, and online resources are considered a medium that allows young people to have control, privacy and independence .
At present, there is a lack of data on whether DHIs can close the gap between supply and demand for mental health interventions in CYP and crucially, whether they reach populations currently underserved by traditional face-to-face services. There is a pressing need to understand more about the individual characteristics of CYP who benefit most from DHIs as well as those characteristics that suggest DHIs would be unhelpful or contraindicated. Research is also needed to understand where best DHIs are placed in existing care pathways. For example, should self-guided cCBT for depression be offered routinely before face-to-face therapy, or alternatively in parallel with face-to-face therapy to augment adherence and effectiveness? Similarly, should young people be signposted to specific DHIs only after online or FtF assessment? High-quality health services research is needed to answer these questions.

Summary
In recent years, there has been a rapid growth in the development and evaluation of DHIs for mental health problems in CYP. While the evidence we reviewed provides some support for cCBT as a treatment intervention approach for mild-to-moderate depression and anxiety, the benefits remain uncertain for other clinical areas. There is also insufficient research investigating the 'active' and critical components of these interventions. We recommend that future research should focus on identifying these 'active ingredients', i.e. the individual components or specific mechanisms of change in cCBT and other DHIs that are most effective for improving uptake, adherence and clinical outcomes in CYP.
A notable finding of our meta-review was that the research in this area (190 individual papers) described 147 unique DHIs. Hence, a major challenge for the field is to develop an agreed taxonomy to assist the identification of common active components of different interventions. We would argue that a more efficient and theoretically sound approach would be to develop DHIs through a process of optimisation by incorporating and testing existing evidence-based components which act as 'core' building blocks for new DHIs.
The majority of DHIs have been designed to help CYP at risk for developing, or with a diagnosis of, an anxiety disorder (including generalised anxiety, social anxiety and specific phobia) and/or depression. Our review also identified a small number of trials of DHIs for ADHD, ED and ASD. However, areas such as psychosis, PTSD (and other specific anxiety disorders) are under-researched, while conditions such as Tourette syndrome, conduct disorder, substance misuse and emerging personality disorder (or interpersonal problems) have been completely overlooked thus far. DHIs for ADHD have focussed predominantly on 'brain training' approaches using computerised WMT and CF training and EEG NFT. Future research should explore how these approaches transfer onto realworld outcomes, and if benefits are sustained and generalised outside the context of the specific computerised training tasks. Furthermore, there is a need for more nonpharmacological approaches that harness mobile (mHealth) DHIs, including wearable technologies, to treat and manage ADHD (Tarver, Daley, Lockwood, & Sayal, 2014). More research is required on the role of factors such as reminders and human facilitation to understand whether it is possible to identify an 'optimum level' or whether it is preferable and feasible for each individual to design their own.
Human facilitation/support is an important factor in influencing uptake, engagement and outcomes of DHIs. It appears that 'blended' DHIs that include human facilitation/support may achieve greater engagement, treatment adherence and improved retention in intervention trials. However, we are unable to draw any firm conclusions about what form and how much human support are most effective for CYP as a whole, let alone more specific user groups. It is important to note that the type and level of human support provided to encourage retention in trials is not necessarily practicable or transferable to routine clinical settings; therefore, it is unlikely that retention rates (and outcomes) reported in trials can be achieved if DHIs are implemented as unguided/ unsupported interventions outside trial settings. Our results suggest that the level of support provided within trials for children is more substantial, particularly for ASD and ADHD. Furthermore, the characteristics of DHIs that support engagement at different ages requires further research. It is likely that DHIs for children need to incorporate more interactive, game-like elements so that the development of skills and progression through the intervention becomes self-reinforcing. Findings from the meta-review suggest that for cCBT, greater benefits are found in older CYP. The limited effectiveness of DHIs for mood disorders in younger children may result from insufficient adaptation of interventions to children's cognitive and developmental needs (Adelman, Panza, Bartley, Bontempo, & Bloch, 2014).
Research investigating the use and effectiveness of mHealth smartphone/tablet apps was mostly absent from our review. This area of healthcare delivery is growing for adult populations (East & Havard, 2015;Mani, Kavanagh, Hides, & Stoyanov, 2015;Nicholas, Larsen, Proudfoot, & Christensen, 2015). There are several commercially available Mindfulness interventions in a mobile application format that could have trans-diagnostic benefits for mental health (Mani et al., 2015), but these have not been evaluated with CYP. Mobile apps may also be conceptualised as add-ons to online interventions, for example to make material available to users while on the go. Apps may also provide a means of supplementing therapy (both FtF and online) by allowing, for example, remote monitoring of symptoms (Simons et al., 2016). Second-generation DHIs also incorporate wearable devices (e.g. activity monitors), which have been more widely explored in the health psychology and behaviour change field for CYP (Turner, Spruijt-Metz, Wen, & Hingle, 2015). The rise in research on mental health-related apps and the potential behaviour change arising from wearables suggests that a more holistic approach to digital interventions is on the horizon, with a blurring of boundaries between digital assessment, monitoring and interventions. However, this will also add to the complexity of assessing efficacy and determining the active components of an intervention. Virtual reality interventions were also absent from our review results. In adults, virtual reality interventions for mental health have predominantly focused on enhancing existing interventions such as exposure therapy for specific phobias (Pan, Gillies, Barker, Clark, & Slater, 2012) or PTSD (Gerardi, Cukor, Difede, Rizzo, & Rothbaum, 2010). However, new research is expanding into areas such as depression , psychosis (Leff, Williams, Huckvale, Arbuthnot, & Leff, 2013) and EDs (Marco, Perpiñ a, & Botella, 2013).

Conclusions
DHIs offer huge potential for widening access, increasing efficiency and improving healthcare outcomes. However, existing research indicates that benefits have yet to be fully realised and effectiveness of these approaches remains uncertain. For the field to realise the full potential of DHIs, it is necessary to simultaneously harness the latest technological innovations while maintaining a robust evidencebase of clinical and cost-effectiveness. Meeting this challenge requires a novel integration of innovative approaches and research methods drawn from disparate disciplines and academic traditions. To date, the methods for developing and evaluating DHIs have borrowed largely from approaches used with psychological and pharmacological interventions. However, the development and evaluation of DHIs requires different approaches with integration and interdisciplinary collaboration between methodologies and approaches drawn from engineering, computer science, human factors, human computer interaction, psychology and mental health services research.
From a clinical perspective, we recommend that an integrated approach should be developed that takes into account the views of CYP, the opinions of MHPs (gatekeepers of DHIs) and seeks to blend DHIs with FtF therapy. MHPs emphasise the importance of DHI's being adjuncts to traditional FtF therapies. Consideration needs to be given to the possible adverse effects of CYP using DHIs outside mental health services on publically available apps. Adverse effects may result from an ineffective or unsuitable intervention, inaccurate health information, leaking of personal information, lack of support, lack of motivation or an exacerbation of symptoms. Consequences of a negative experience of DHIs for CYP such as a lack of faith in efficacy or specific feelings of helplessness, hopelessness and low selfworth may reduce future help-seeking behaviour in CYP, which is a particular concern given the recurrent nature of mental health problems (Watsford & Rickwood, 2014). An additional safeguard for the use of DHIs would be to explicitly highlight the potential negative effects of DHIs. Many interventions do advise users to seek professional help if symptoms deteriorate or do not improve, but we would argue that this does not go far enough.
Future generations of DHIs will also offer seamless integration of real-time passive and active monitoring with personalised therapeutic interventions. In this way, the huge potential of digital technology (realtime connectivity of data, ubiquitous reach, personalisation and convenience) can be best harnessed to improve the effectiveness and reach of evidencebased psychological therapies for CYP.

Supporting information
Additional Supporting Information may be found in the online version of this article: Appendix S1. Search strategy process for meta-review. Appendix S2. Review protocol for updated systematic review. Appendix S3. Search strategy process for updated systematic review. Appendix S4. Table of interventions and studies in meta-review.
• The last decade has seen a rise in the development and assessment of DHI for mental health problems in children and young people (CYP).
• Our meta-review and updated systematic review reveals some evidence for the short-term efficacy of DHI for adolescent and young adult depression and anxiety but not for other psychopathologies or in younger children.
• The development of and research into DHI for CYP must address several key methodological issues that are widespread within the field (e.g. specifying the level of human support, blindness of assessors and comparators), and which ultimately contribute to uncertainty around efficacy and clinical recommendation.
• For the field to progress, it is essential that future research identifies and separates out aspects of DHIs that are likely to be stable (i.e. content and theoretical underpinnings) and changeable (e.g. technology platform and mobility).
• It is important for researchers to understand which components of DHIs influence efficacy (e.g. theory and level of human support) and which components of the digital delivery platform determine access, acceptability, uptake, usage and adherence.