The Impact of Research on Policy: A Case of Qualifications Reform

ABSTRACT The relationship between research and policymaking has been discussed repeatedly. However, the debate tends to be in general, abstract terms or from a macroeconomic perspective with any examples described in a fairly cursory way. Despite the inherent complexity of the research-policy interface, analyses tend to homogenise ‘research’ and ‘policy’ as coherent entities with discussions often focusing on products (research and policies) rather than on the relationships between producers (researchers and policymakers). Here, we take one piece of research on qualifications that has influenced policy rhetoric over the last 5 years. We trace the career of the research from its production in the late 1990s in order to understand the conditions of its dormancy, re-emergence and use over the ensuing years. The paper serves to document the case, which is important in its own right, but also proposes a typology of ways in which research gets adopted and adapted into policy.


INTRODUCTION
In 2014, the UK's periodic assessment of university-based research included a measure of 'impact' for the first time. The Research Excellence Framework (REF2014) graded the carefully crafted impact case studies in each 'unit of assessment' from universities across the UK. Twenty percent of the core research income to UK universities is now allocated on the basis of those judgments about the impact of research. Early analysis (Jump, 2015) suggests that the most common type of claimed impact was that made upon government policy.
Much has been written about the relationship between social research and public policy (e.g. Nutley et al., 2007) and this includes a sustained discussion in education (see Lingard, 2013;Lubienski et al., 2014;Orland, 2009;Saunders, 2007; for more recent examples). This paper considers an explanatory case study (Yin, 2003) in which research that was dormant for many years become topical and was reframed to suit policy. It offers insights into how research works its way into policymakers' thinking. Carol Weiss argued that research is a 'supporting player in the drama of policymaking. The empirical question is under what conditions research gets on the stage at all, and when it does, what consequences it has for the unfolding action ' (1991, p. 308). Given the call for more empirical work in this area (Lingard, 2013), we use this case to illuminate the settingstaged and/or improvisedof this particular policy drama. It should be noted that although we present a case of policy-informing research, it was originally policy-driven research, what Lingard (2013) terms 'research for policy'. So whilst it is the reappearance onto the policy stage that is of interest herein, the motivation for the original research should not be forgotten.
There have been regular criticisms of education research over recent years (see Oancea, 2005, for a summary). Ex-Secretary of State for Education Michael Gove's attack on university departments of education as the Marxist blob and the 'modern Enemies of Promise' (Gove, 2013) exposes some of these criticisms as politically and ideologically motivated and this is an important point to which we will return. [As Whitty (2006) explains, David Blunkett's view of educational research was not altogether different]. In contrast to Gove's polemic, an earlier report from the Department of Business, Innovation and Skills had identified the UK's education research as 'a global leader' (BIS, 2011). Discussions about the purposes and quality of education research will no doubt continue unabated, as will debates about the complex interface between what Orland (2009) calls the 'separate orbits' and 'misaligned cultures' of research and policy or researchers and policymakers (see Whitty, 2006, for one such discussion).
Were all of those REF case studies incorrect in claiming their impact upon policy? It would seem, based on one of the present author's experiences in and around research and policy advisory work, that research has limited chance of impact without brokerage across pre-existing or emerging networks. These tangled webs of 'bridgers and brokers' (Ball and Exley, 2010) or 'boundary spanners' (Williams, 2002), which nowadays includes a diverse array of researchers, think tanks, advocacy groups, philanthropists and the like, are part of the new forms of networked governance (Ball, 2008(Ball, , 2009Ball and Junemann, 2012) that make the analysis of policy work challenging. Such policy interlockers and networks, and what Lubienski et al. term 'intermediaries' (2014), will be clearly evidenced in our analysis. Ball and Exley explain that: policy networks are relatively unstable structures of positions and sitesthink tanks, social enterprises and advisersbut are also flows of ideas and people… within the capillaries of these networks, ideas have careers and are diffused. (2010, p. 155) They argue that ideas and people 'flow' simultaneously and unpredictably and are, in some sense, indistinguishable. In their analysis, ideas are anthropomorphised (i.e. have careers) so that the capillaries of the network can be people/sites through which ideas flow, or alternatively, the capillaries are ideas through which people connect and flow. However, the use of flow as a metaphor omits a notion of force or power. In response to Ball's earlier work on governance, Goodwin (2009) calls for such a greater focus on power in the analysis of policy networks, and we try to exemplify this in the case herein.
In the following section, we set out the current qualifications reform agenda in England. Given the importance of the people, positions, sites and circumstances in the case, we present an outline timeline, including key changes in government, ministerial positions, other relevant connections and legislative moves. The list is a partial, post hoc construction that introduces the key players in this drama. The final piece of context setting is the original research so, we sketch out for the general reader the findings and limitations. Having presented the current reforms, policy history and the original research, the paper then explores how the findings have come to be embedded in policy discourse.
Before proceeding, we sketch out the relevant part of the qualifications landscape in England. All young people complete their General Certificate of Secondary Education (GCSE) examinations at age 16. Grades are awarded from A* to G and schools are compared and judged on the basis of student outcomes. Over half of young people achieve at least five or more GCSEs at the 'good grades' of C and above (including English and mathematics) and the majority of these proceed to study Advanced Level (A level) qualifications. Typically, students take 3 or 4 A levels over the following 2 or 3 years. These in turn become their entry qualifications for university.

THE QUALIFICATIONS REFORM CONTEXT
The UK government has been pursuing a major programme of curriculum and qualifications reform for England's schools; here, we focus on mathematics. A key plot line in these reforms is that specialist mathematics and quantitative skills will increasingly be required for future jobs (Truss, 2013a(Truss, , 2013b. There is no space herein to discuss the political privileging of mathematics in much depth but it can be seen in (a) the growing influence of international tests (e.g. Programme for International Student Assessment (PISA) and Trends in International Mathematics and Science Survey (TIMSS)) in which mathematics is central (OECD, 2010;Sturman et al., 2007), (b) the ring fencing of the Science budget and concerns about the 'leaky' pipeline into Science, Technology, Engineering and Mathematics (STEM) careers (Jacobs and Simpkins, 2006) and (c) repeated complaints from business groups such as the Confederation of British Industry (2011, 2012) regarding the mathematical capabilities of employees at all levels. The common underpinning theme is the economic rationale for better mathematics education (Bynner and Parsons, 1997;Crawford and Cribb, 2013;Dolton and Vignoles, 2002) and the fact that this economic rationale plays more strongly in relation to mathematics than any other subject, in particular English. The Prime Minister, David Cameron reinforced this point when he announced further investment in mathematics and science education: 'There's no secret to success in the modern world. If countries are going to win in the global race and children compete and get the best jobs, you need mathematicians and scientistspure and simple' (BBC 8.12.14).
Elizabeth Truss, who was until July 2014 the UK's Parliamentary Under Secretary of State for Education and Childcare, was a major influence in this privileging of mathematics, although her interest in this areaand in qualification reformbegan much earlier. The co-authored Reform Group report in the timeline below was written shortly after she became Deputy Director of the think tank and the strong views of the report, combined with her proximity to senior politicians, points to the influence of her thinking on mathematics education well before she became a junior minister in September 2012.
Winning the battle of the maths economy will be critical to the UK's future success. Current Government policy is too small scale to deal with the pressing nature of the problem. Radical measures have to be taken to move mathematics from 'geek to chic'. Rigour must be central to this approach. The Government should step in and reverse the current inexorable drift towards modularising GCSE mathematics. A new Alexander is needed to cut the Gordian knot of state control and open up individuals' and institutions' ability to improve their own capability in the subject (Kounine et al., 2008, p. 5).
The language of battle is noteworthy, as is the elision of mathematics and economy. Perhaps Truss considered herself to be a liberating 'Alexander' setting out with a battle strategy for mathematical liberation. Importantly for this discussion, the report introduces what Ball and Junemann (2012) call a 'nodal actor' (i.e. Professor Alison Wolf) who connects the research of interest herein and the mathematics education policyscape of today, as we show below.
Following the Wolf Report (2011), the Conservative-led government committed to ensuring that students continue to study mathematics post-16 until they have reached a certain level of proficiency (a GCSE grade C). Those who have already attained this level at age 16 should be expected to continue with advanced mathematical study. New 'Core Maths' qualifications were developed and introduced in 2015 to sit alongside the traditional A levels in order to support the realization of this goal. Accountability measures have been developed that could be used to monitor the impact of this policy move. This 'maths for all' development points to the influence of the research that will be discussed below.
One of the arguments made for the drive to increase post-16 mathematics participation is a comparative one (Hodgen et al., 2010). However, the specific argument that is most frequently used is that 'People who take A-level maths earn on average 10% more than their peers' (Truss, 2013b). The research from where this claim originates was linked to a major review of A-level qualifications in the 1990s, undertaken amidst concerns about declining take-up of mathematics and science A levels (e.g. Hawkes and Savage, 1999). Dolton and Vignoles' work was published by the Centre for Economic Performance (at the London School of Economics (LSE)) as a working paper entitled The Economic Case for Reforming A Levels (Dolton and Vignoles, 1999) and later as an academic article focused on mathematics (Dolton and Vignoles, 2002).
Dolton and Vignoles' research utilised the 5th wave of the National Child Development Study (NCDS), as well as the 1986 Graduates and Diplomates Survey (GDS), and suggested that A-level mathematics (and/or computing) was unique in terms of its economic return. The extent of this return was modelled as 7-10% at age 33 for this cohort. There were some important caveats that have gone largely ignored by those quoting the findings today and we take some time to consider these in the following section. The headline figure is all that has survived in nearly all instances.
At this point, it is instructive to locate Dolton and Vignoles' work: 1999 New Labour Government (from 1997) has continued neoliberal education policy direction from previous Conservative government; national review of 16-18 qualifications led by Lord Dearing (1996) has precipitated major reform, that is, Curriculum 2000; Dolton and Vignoles' research first published.

2000-2004
Curriculum 2000 impacts qualification landscape; 'disastrous' (Smith, 2004) reduction in A-level mathematics numbers requires immediate remediation; increasing number of reports on importance of STEM to economic security (Roberts, 2002) Conservative-commissioned Vorderman Report (2011) published; Michael Gove's (2011) Royal Society speech sets out vision that 'within a decade the vast majority of pupils are studying maths right through to the age of 18'. 2012 Truss becomes junior minister at the Department for Education; during a short term in office Truss advocates strongly for mathematics, particularly A-level and new Core Maths qualifications.
The following section outlines Dolton and Vignoles' research and some of the caveats of that work. We then explore how the research has been used.
3. THE RETURN ON POST-COMPULSORY MATHEMATICAL STUDY Dolton and Vignoles' research employed a series of logistic regression models to explore labour market returns to advanced mathematical skills. A number of important caveats were reported by the authors. The NCDS has a number of limitations that impact upon the statistical modelling processes. Moreover, approaches to missing data were limited at that time by the computing power available. These limitations and other historical/contextual factors should be borne in mind when using the findings, but most have been lost in the reporting of the research. This section briefly sets out these issues, without going into great methodological depth.
The NCDS follows the lives of around 17,000 children born in 1 week of 1958. The participants took their A levels in 1976 and are now around 58-years old. The data sweep for Dolton and Vignoles' analysis tool place in 1991 when the participants were aged 33. The first question that arises is the extent to which this group is similar to today's 16 year olds. The education and economic landscape is fundamentally different from 1974, that is, the year in which they decided to take A-level mathematics. Since then, the numbers taking A levels has changed considerably; the proportion of people entering higher education has increased dramatically and education has become increasingly marketised. The economic and employment landscape is vastly different to that of the 1980s and early 1990s and the economic growth experienced in the working lives of the NCDS cohort is, arguably, significantly different to what young adults now face.
The original analysis combines mathematics and computing as an analytical category. Admittedly, the numbers of A-level computing students were considerably smaller than mathematics students. The research also analyses two different datasets. The part based on the NCDS is limited to males with at least one A level and, because of missing data, the final models include fewer than 500 males. Dolton and Vignoles' work considered all A levels but the wage premium for A-level mathematics was unique. Science, for example, did not have such an effect. The analysis was for 33-year olds and the authors said little about what happened to this cohort prior to 1991, and could not comment on what might have happened since.
The issue of causality is notoriously difficult. It is all too easy to make unjustifiable causal assumptions, especially, if they align with one's prior beliefs. In this case, the question of why this result is observed needs careful consideration, and the authors did discuss this. Did A-level mathematics have a screening or signalling role at that time? For example, did it tend to attract those with greater intellectual resilience and/or problem-solving skills and then later on signal those traits to employers? Alternatively, did the process of studying A-level mathematics result in those people acquiring some relevant knowledge and/or skillshuman capitalthat somehow enabled them to attain higher incomes 15 years later? Whether theories of screening or human capital (A. Weiss, 1995) offer the best explanations of the results is unresolved (see Adkins and Noyes, 2016, for a more in-depth discussion of the original research and these theoretical positions).
Dolton and Vignoles argued that 'A level mathematics is greatly valued by firms' (p. 10), which hints at the signalling value of the qualification. They also asserted that 'the skills provided by A-level maths are clearly correlated with workers' future productivity,' which is more of a human capital argument (i.e. the qualification 'provides'). Whilst the former is difficult to dispute, it is harder to justify the claim that A-level mathematics per se 'provides' profitable skills, although recent work by Attridge and Inglis has begun to explore this (2013). The theoretical explanation given by the authors matters less than the fact that policymakers have assumed a human capital argument, namely, that if a student is persuaded to study A-level mathematics they are likely to reap an economic advantage.

TRACING THE INFLUENCE OF '7-10%'
To research this case, we systematically collated and analysed a range of relevant literature from the turn of the century through to the present day. This included 57 relevant reports from government, think tanks, advisory bodies and learned societies. Given our interest in tracing the flow of ideas/people across the capillaries of the policy network, our method was to start from the present day and trace Dolton and Vignoles' 7-10% research finding, either through reference to the original research, clear references to secondary sources and/or other echoes of the work (e.g. 'it is well known that one earns more if…'). We gathered further relevant material including 20 published ministerial speeches (since 2010) and a range of associated news stories. This enabled us to identify 'hot spots' or nodes in this textual network of ideas and people, such as highly referenced sources or key interlockers and policymakers who regularly cite the research. This mapping of the flow of ideas during this period reveals how a small number of highly influential 'boundary spanners' seeded Dolton and Vignoles' research into public/political consciousness.
Upon publication of the original working paper in February 1999, there was a flurry of media coverage, for example: A-level in maths is key to high pay (Judd, 1999) Maths students count themselves better off (BBC, 1999) A-level maths equals money (McGavin, 1999) Despite these media stories, there is no evidence of there being any impact upon mathematics education policy at that time. 1 This might have been because the Curriculum 2000 reforms were well advanced but perhaps more importantly, the priority at the time was upon primary and lower secondary mathematics in the form of the National Numeracy Strategy (Brown, 2011;Brown et al., 2003). It is surprising that the research was not more influential at that stage as there was (a) a strong national policy emphasis on mathematics and (b) growing discontent about the declining numbers of A-level mathematics participants from academia and STEM in particular. Indeed, the same Labour government initiated the Roberts (2002) and Smith (2004) reports shortly thereafter, both of which put the spotlight on A levels.
Ten years after Curriculum 2000, under a new Conservative-led coalition government, Dolton and Vignoles' research resurfaced and became highly influential. The question is how and why this happened. Before answering this question, we trace the route through which this research smouldered out of site, through two influential 'boundary spanners': Professor Alison Wolf and Elizabeth Truss (who was not a government minister until near the end of this process). That is not to say that these were the only protagonists in this narrative (the special advisors (SPADs) and a range of other individuals were no doubt active in the background) but it is through their contributions that the research is reactivated.
Wolf, with her connections to education, economics and public policy, is an archetype of Ball and Exley's (2010) 'policy interlocker', that is, a key individual who 'straddle(s) sectors and policy fields and settings' (p. 152). Wolf referenced the Dolton and Vignoles research (Tikly and Wolf, 2000;Wolf, 2002) and has been repeatedly cited as the source of the A-level mathematics wage premium findings. Very few references in the policy literature refer back to the original research. For example, the Reform Group report cited Wolf rather than Dolton and Vignoles. This raises the question of whether the research would have become widely known without Wolf's interlocking role. There is evidence that the research had filtered into some schools prior to 2010 and was being used by heads of mathematics departments as they tried to influence students' A-level choices (Noyes and Sealey, 2011).
Prior to the Conservative-led coalition government coming to power in 2010, they commissioned a report on mathematics education (Vorderman et al., 2011). 2 In his introductory comments, the then Secretary of State for Education, Michael Gove, wrote that: Mastery of mathematics is key to success in the modern economy. The better educated a country and its citizens, the more likely they are to prosper. Few qualifications confer an economic advantage like mathematics qualifications (Vorderman et al., 2011, p. iii, italics added).
This hints at the influence of the original research on government thinking at the time. That said, it is important to note that there is a growing body of research pointing to the economic benefits of mathematics skills (Hanushek et al., 2015;Vignoles et al., 2010;Walker and Zhu, 2013). Gove's comments have a clear emphasis on economic success, prosperity and advantage and such a policy environment is conducive to the reification of research that signposts individual and collective economic benefit. His use of 'confer' is similar to the earlier 'provide' and suggests that it is the learning acquired through the qualification that matters rather than its signalling potential.
Subsequently, Elizabeth Truss had much to say on this topic. In a speech at the Institute of Education in March 2013, she said that 'People who take A-level maths earn on average 10% more than their peers' (Truss, 2013b) and, later that year, she explained that 'Maths, for example, is the only school subject which has been proven to add to earnings, by up to 10% at A level, even when every other factor is taken into account' (Truss, 2013a). In light of this, it seems entirely plausible that Gove's comments refer to A-level mathematics.
The notions of proof and completeness ('every other factor') represented here are erroneous given the caveats outlined in the original research and the nature of statistical modelling and observational studies. The models did not include 'every other factor', are dated, were for males, and have much greater variability at the individual level than 7-10%. The reference for the politician's use of the research can be traced back through the earlier Reform Group report and then to Wolf: Further mathematics qualifications also benefit individuals in the workplace. Alison Wolf, Professor of Public Sector Management at Kings College, London, has highlighted the economic returns on post-16 mathematics qualifications: 'Even after allowing for every other factor imaginable, people who took A-level mathematics earn substantially morearound 10 per cent morethan those who did not' (Kounine et al., 2008, p. 6).
Note that further mathematics in this context is advanced mathematics beyond GCSE study. Although the original research remained largely unacknowledged by Labour ministers, the incoming government of 2010 used this econometric study, together with comparative research (Hodgen et al., 2010) to drive a policy agenda that called for 'a renaissance in mathematics education' (Truss, 2012).
Although Truss was instrumental in bringing the research into policy consciousness, she did not remain in post for very long (21 months). Nonetheless, the research does not seem to have diminished in influence but continues to be cited in support of a number of policy programmes. For example, the current Secretary of State, Nicky Morgan, has argued that '… maths, as we all know, is the subject that employers value most, helping young people develop skills which are vital to almost any career. And you don't just have to take my word for itstudies show that pupils who study maths to A level will earn 10% more over their lifetime' (Morgan, 2014). The irony here is that the audience is presumably expected to 'take her word for it', even though the claim is problematic for the reasons discussed herein. Similarly, Nick Gibb, Truss's successor has also stated that 'those who do Maths A level will go on to earn 10% more'. Ball and Exley (2010) said that 'ideas have careers' and it seems that the career of this research idea is very much alive and active. The 7-10% finding has matured to a much neater 10% in the eyes of Truss, Morgan, Gibb and others. Additional features of the research have also disappeared or mutated, for example: • It seems to be time invariant • It applies only to mathematics (not computing) • It applies to all students, whether male or female • The inherent statistical variability has been cemented into a fixed 10% • Finally, the research compared A-level students who had, or who had not, studied mathematics. This is different from the idea above which seems to compare people in the population with or without A-level mathematics, irrespective of final qualification level achieved.
We now proceed to explore some of the plot lines of this particular researchpolicy drama.

DISCUSSION
The account above offers some insight into what Weiss (1991) calls 'the extraordinary concatenation of circumstances' by which research impacts policy. She suggests that research can shape policy as 'data, ideas or argument' and these can be seen in this case. More recently, Orland (2009, p. 136) identifies three conditions for research to be used in policymaking: rigour, relevance and usefulness. The case study highlights how key players, relationships and conditions have been important in the influence of this research on policy. Here, we attempt to synthesise the various points from the analysis into a typology of ways in which this research has been adopted but also adapted. Firstly, why and how did the research get adopted? We list six reasons, in no particular order.
(1) The main research findings are simple and/or simplifiable: Education policymakers are tremendously busy people, juggling impossibly full portfolios and facing problems for which the education research community offers little counsel. A piece of research, which is easy to remember, is attractive and easily communicated: A-level mathematics; earnings; 10%. Weiss explains that 'what distinguishes policy research as ideas is that most of the paraphernalia of research has been stripped away' (p. 311). In the same way, Ball and Exley (2010, p. 153) report how think tanks produce 'simple messages that can be easily understood by politicians, policymakers and the public'. (2) The research is persuasive: Although it is unlikely that any of the policymakers have ever read the original work (though their SPADs might have done so), its origins and provenance speak for themselves. In addition, the use of secondary data and statistical modelling ('policy research as data', Weiss, 1991) offers the desired scientific validity or, in Orland's terms, rigour. In this, and many cases, the fact that the findings seem to fit with common senseat least that promoted by the powerful STEM lobbyalso helps to make the research more compelling.
(3) Key connections are made: Even though the first two conditions have been satisfied, the research needs to be effectively inserted into, and flow around, key policy networks. In this case, the connection to the LSE, Alison Wolf, the Conservative Party, the Reform Group, Truss, Gove, etc. is easy to see. These interlockings are made temporally and spatially; this case is very London-centric. But there is also a degree of serendipity (Whitty, 2006), so whilst it is easy to see the career of the idea from today's vantage point, it was by no means clear that this drama would unfold as it did. (4) The research harmonises with policy ideology: Conditions 1-3 are arguably necessary but not sufficient. The research idea, however simple, persuasive and well communicated will not take hold unless the contextual ideologies and values are sufficiently well harmonised with the research ideas. In this case, the economic drivers that increasingly underpin a neoliberal, market-driven approach to education are well aligned to the research findings. (5) The implications of the research must be workable: No matter how simple and persuasive the research might be, parliamentarians can only make use of the research if there are mechanisms available to them by which it can be assimilated and recommendations enacted. In this case, accountability levers have been reconfigured to provide ways of reporting on the level of post-16 mathematics participation. (6) The research needs interested champions: This champion might be an individual or group who advocate or sponsor the research and for whom there might be personal or collective gain. That is not to say that policymakers do not have justifiable reasons for promoting research. Rather, our point here is that without such selfinterest, research would often not pass first base.
In addition to these six conditions for research adoption, research gets adapted when it meets policy priorities. This can either be intentional in the form of omissions and distortions or accidental through the 'Chinese whispers' that occur as ideas and people flow across policy networks. Four main themes can be identified from this case and we are confident that these are characteristic of the research-policy interface in other areas: (1) Decontextualisation: ignorance of the historical, economic and cultural context The policy discourse around this research does not acknowledge the historical context. Social research is temporally and culturally framed and losing sight of this increases the risk of misapplication.
(2) Partiality: only using the convenient parts of the research (a) The elision of mathematics and computing in the original research has been lost. Although this is not surprising, it could be considered a missed opportunity now that ministers are concerned to reestablish computing (i.e. programming) in schools. (b) Partiality can also consist in selective use of statistical results. The ranges reported in the original research have disappeared and the 'return' is now a fixed 10%, or 'around 10%' and reflects a bias towards a more politically expedient result. Moreover, there is a tendency to strip out the inherent uncertainty in any such statistical analysis, for example, confidence intervals. So, it is that 'the original research findings are reduced to a simple "story", qualifying statements are lost, and the conclusions are often stretched beyond the findings of the study' (Weiss, 1991, p. 311).
(3) Overgeneralisation: (a) To time: Nicky Morgan's claim about earnings over lifetimes is an example of overgeneralisation. All that can be said is that in 1991, amongst a small sample of men born in 1958, those who had completed an A level in mathematics or computing in 1975 were earning, on average, between 7 and 10% more than their A-level peers who had not taken mathematics. (b) To other subjects: Dolton and Vignoles' work identified A-level mathematics as having a unique effect. In particular, science A levels did not have the same effect (although it is likely that physics and biology behave very differently in this regard). However, this is inconvenient for promoting the STEM agenda. (4) Misinterpretation: Misinterpretation is a potential consequence of research being simple and simplifiable. In this present case, the notion of causality is a pertinent example. What cannot be implied from this research is that a young person aged 16 in 2016 who is persuaded to change their A-level choices to include mathematics on the basis of this research will, as a result, be earning 7-10% more than their non-A-level mathematics peers in 2033. Yet, this is the tenor of the politicians' position.
6. CONCLUDING REMARKS This case study shows that one cannot control how research is adopted and/or adapted once in the public sphere Orland's (2009, p. 117). Discussion of the 'distinctive and misaligned cultures' of research and policymaking helps to explain these processes in general terms. The empirical details of the case herein raise many questions for researchers and policy interlockers interested in the impact of research upon policy formation. One such question is whether there is a moral obligation for researchers to challenge the misuseintentional or otherwiseof their work. Perhaps, the answer to this depends on whether or not one was aiming specifically at such research impact.
The flows of people and ideas, even in this relatively simple, bounded case, are not easily understood and mapped. Indeed, it is quite likely that some of the most important capillaries of the policy networks relevant to this paper cannot even be accessed but are buried well below the surface of government, think tanks and other parts of the policy network (Ball, 2012). Incidentally, recent work commissioned by the DfE suggests that the premium on A-level mathematics is no longer clear (London Economics, 2015) though our own repeat study with the 1970 British Cohort Study challenges this claim .
Through careful analysis of this particular case, we have sought to show that direct and unadulterated impact of research upon policy is unlikely. Rather, research messages undergo manifold distortions and manipulations to suit a variety of purposes as they get 'whispered' from research paper to policy idea to government agenda. The research-policy nexus is complex and our capacity for understanding, let alone controlling, such nonlinear processes is not well developed as Cummings points out in the introduction to his 'thesis' and call for better education policymaking.
Although we understand some systems well enough to make precise or statistical predictions, most interesting systemswhether physical, mental, cultural, or virtualare complex, nonlinear, and have properties that emerge from feedback between many interactions. Exhaustive searches of all possibilities are impossible. Unfathomable and unintended consequences dominate. Problems cascade. Complex systems are hard to understand, predict and control. (Cummings, 2013, p. 1) So, more empirical research on this complex and unpredictable research-policy interface is needed, although it is methodologically challenging. Moreover, researchers need to be better equipped to navigate the network flows of people and ideas when designing external engagement and impact strategies. In this way, the chances of 'overlapping orbits' (Orland, 2009) between these two worlds are increased. Even where research can be shown to impact upon policy, such as is claimed in many REF case studies, it is less clear whether the impact was the result of (a) a well-designed 'impact strategy', (b) some serendipitous event or a seemingly random 'concatenation of circumstances' or (c) the (intentioned) agency of a critically important (i.e. powerful) policy interlocker or boundary spanner. Is it enough to hope for the best and let the research speak for itself or to presume the supportive workings of policy networks and their agents? The answer must surely be 'no'. Rather, for those interested in using research to catalyse better policy, new logic models of the complex research-policy-practice interface are needed to support the engineering of improved research engagement and use and impact planning. This paper speaks to this agenda.