Policy masquerading as science: an examination of non-state actor involvement in European risk assessment policy for genetically modified animals

ABSTRACT In 2013, at the request of the European Commission, the European Food Safety Authority (EFSA) announced a new risk assessment policy: Guidance on the environmental risks of genetically modified (GM) animals (‘Guidance’). This policy specifies the issues to be addressed in future risk assessments for GM animals. EFSA is the European Commission's scientific arm, responsible for food-related risk assessment. EFSA relies heavily on independent experts and consults non-state actors. Employing expert interviews and documentary analysis, the article explores non-state actor involvement in a traditionally expert domain through a case study. Analysis of EFSA's consultation demonstrates the inability of non-state actors to influence policy. The article argues that despite international legal obligations to develop risk assessment policy, the European Commission failed to recognize the Guidance as policy. When policy masquerades as science, unjustified restrictions are placed on non-state actor involvement and value judgements are cloaked from public scrutiny.

In 2013, the European Food Safety Authority (EFSA) announced the Guidance on the Environmental Risk Assessment of Genetically Modified Animals (the 'Guidance') (EFSA 2013a). This Guidance is a risk assessment policy (RAP), establishing the risk assessment framework for releases of genetically modified (GM) animals into the environment. As such, it addresses the scope of future risk assessments (the kinds of impacts deemed to be within/outside the scope), what counts as evidence and how much is needed, the interpretation of evidence, how uncertainties should be addressed, and how precaution should be applied (Millstone et al. 2004). The Guidance is aimed at applicants who want to release a GM animal into the European environment and for risk assessors conducting future assessments. Although no GM animal has been approved for release in Europe, experts estimate GM insects will be the first GM animals to be placed on the European market (Environment Agency Austria 2010). In line with this prediction, in January 2013 the British company Oxitec Ltd. submitted an application to release the GM Olive Fly for experimental release in Europe (European Commission 2013).
EFSA is an independent regulatory agency with a statutory obligation to provide scientific advice to the European Commission ('Commission') Directorate General for Health and Consumer Protection (DG SANCO) on food safety matters and take responsibility for the risk assessment and communication functions of risk analysis. EFSA's establishment in 2002 was part of a shift towards 'good governance' aimed to strengthen the independence of scientific advice, facilitate engagement and open up risk decision-making to public scrutiny. This shift was driven by a crisis in consumer confidence in European governance institutions following food and agricultural regulatory failures in the 1990s (Dreyer and Renn 2014). EFSA relies heavily on advice from ostensibly 'independent' scientific experts, but also has a statutory obligation to engage with stakeholders (EC Regulation No 178, 2002: Article 42). One way in which it meets this obligation is to hold public consultations ('consultations') that are open to all non-state actors (EFSA n.d.). 1 In contrast to the vast majority of consultation exercises, EFSA's consultations are science-based: traditional policy issues, including ethical and socioeconomic issues are outside its remit (EFSA 2012). The inclusion of non-state actors in the development of traditionally exclusive science-based RAP is noted by Dreyer and Renn (2014: 324, italics in the original) to be 'a true procedural innovation in the regulation of food safety at EU-level' . However, while there might be potential for effective nonstate actor involvement, research suggests in practice, EFSA may not be reaching that potential (Bengtsson and Klintman 2010;Borrás et al. 2007;Hartley and Millar 2014;Klintman and Kronsell 2010).
In a recent edited volume exploring the role of experts in decision-making processes, Ambrus et al. (2014a) asked: are experts advisors, decision-makers or irrelevant actors? The authors found experts 'may act as de facto decisionmakers, even if de jure the decisions are taken by others' and suggest this conclusion raises pertinent questions about how non-state actors are able to influence policy involving experts (Ambrus et al. 2014b: 15). Through expert interviews and documentary analysis, this article presents an empirical contribution to these questions by examining non-state actor involvement in a traditionally expert domain using a case study of EFSA's development of the Guidance. Overall, the findings call into question the quality, effectiveness and impact of EFSA's consultation, demonstrating the inability of non-state actors to influence policy. The article argues the Commission and EFSA failed to recognize the Guidance as policy, instead framing it as a technical document. This framing resulted in experts making policy and minimal non-state actor influence.

NON-STATE ACTORS AND POLICY INVOLVING EXPERTS
Since the growth in the use of technical expertise following the food safety crises in the 1990s, there has been a 'rhetorical shift' towards opening up scientific governance to a broader range of actors (Irwin 2006: 300). This shift translated into a significant rise in public engagement efforts and a burgeoning literature (Stilgoe et al. 2014). Traditionally, risk analysis has comprised of three distinct and linear stages: assessment; management; and communication. Public engagement efforts occur during risk management, where policy-makers determine risk acceptability and how it will be managed. However, calls for non-state actor inclusion in risk assessment, where scientific experts determine risks, have mounted in recent years (Borrás et al. 2007;Dreyer and Renn 2014;Dreyer et al. 2008;Millstone 2009;SCHER et al. 2013;Shepherd 2008). These calls respond to increasing evidence that risk assessment is not an objective, science-based process free from values, and therefore restricting decisions to scientific experts cannot be justified (Wickson and Wynne 2012). However, the role of non-state actors vis-à-vis experts in risk assessment is contentious, with some authors making a case for stakeholder involvement in developing terms of references for scientific opinions (Dreyer et al. 2008) and others clearly opposed to blurring the lines between assessment and management functions and the possible capture of independent and objective scientific processes and outcomes by political values (Gabbi 2007).
A growing area of the public engagement literature critiques the motivations and impact of efforts to open up science governance. Consultations have been identified as vehicles for building public trust rather than allowing participants to shape the decisions on which they are consulted (Klintman and Kronsell 2010;Macnaughten et al. 2005) and often found to have minimal impact on decisions (Petersen 2007). A number of typologies attempt to characterize the degree of influence public engagement exercises have on decision-makers. Arnstein's (1969) 'ladder of citizen participation' draws attention to the degree of power transferred during a consultation and hence to the degree of influence participants have on the decision-makers. The ladder consists of eight rungs: manipulation; therapy; informing; consultation; placation; partnership; delegation; and citizen control. The first two rungs describe non-participation or coercion; the second three describe tokenism or 'window dressing'; and the top two rungs describe a form of citizen control, where power is shared between participants and decision-makers. A few studies have examined EFSA's efforts to open up risk assessment to non-state actors with mixed results. EFSA's consultations were found to restrict the dialogue and powersharing Arnstein describes as necessary for citizen control (Dreyer and Renn 2014;Finardi et al. 2012;Hartley and Millar 2014). In general, this literature calls for further research on the quality, effectiveness and influence of EFSA's consultations (Bengtsson and Klintman 2010;Borrás et al. 2007;Dreyer and Renn 2014).
In addition to the literature on opening up risk decision-making, there is a well-established literature on the use of expert knowledge in decision-making which explores the way in which expert input is linked to actual policy decisions (Ambrus et al. 2014a;Millstone et al. 2004;Rimkutė and Haverland 2014;Schrefler 2010). Some research suggests experts influence policy through gathering, evaluating and providing knowledge to decision-makers who weigh this knowledge against other (political) factors and make a decision (Andresen 2014;Herwig 2014;Schrefler 2013;Siebenhüner 2014). However, the influence of experts may increase when non-state actors are involved in traditionally expert domains, as expert knowledge is prioritized over non-state actor knowledge in order to protect the scientific legitimacy of decisions (Hartley and Millar 2014;Waterton and Wynne 2004). Other research suggests experts play a far greater role in decision-making and in some instances experts become decision-makers. Busuioc (2014) suggests experts gain this privileged position by virtue of their highly specialized knowledge, whereas Ambrus et al. (2014b) argue experts may become decision-makers when they participate in the exercise of regulatory powers, such as the World Trade Organization and EU. However, Millstone et al. (2004) contend experts make decisions when policy-makers do not explicitly acknowledge the limitations of science for risk decision-making.
One policy area where tensions between non-state actor and expert influence over policy decisions are emerging is RAP. RAP is distinguished from risk assessment in risk analysis by the use of the term 'policy' to acknowledge the political decisions involved with determining the overarching risk assessment framework (Millstone 2009;Millstone et al. 2008). 2 RAP establishes the risk assessment framework, specifying the scope of risk assessment and how much of which types of evidence might be deemed variously necessary and/or sufficient for particular types of products or processes to be permitted, restricted or forbidden. While RAP should be informed by scientific considerations, those considerations on their own can never be sufficient. Determining RAP involves value judgements; therefore, it requires broader input. The Codex Alimentarius Commission ('Codex') states RAP should be established in consultation with nonstate actors (CAC 2007). Similarly, the Commission's Scientific Committees have advocated dialogue with non-state actors to reconceptualize non-state actors as participants who can inform and shape RAP rather than recipients of information (SCHER et al. 2013). In this dialogue, non-state actors could shape the initial framings, scope, timelines and depth of risk assessment at an early stage in the process (ibid.).

THE ORGANIZATIONAL AND INSTITUTIONAL SETTING
In 2007, Codex formally adopted the Working Principles for Risk Analysis for Food Safety for Application by Governments, committing its 186 members (including the EU) to develop explicit RAPs (CAC 2007), although evidence suggests the Commission is not following the Principles (Millstone 2014).
Codex commits risk managers to establish RAPs in advance of risk assessment (CAC 2007). The Commission has the remit for risk management and therefore responsibility for RAP development, yet despite the EU's commitment to Codex rules, in 2007, the Commission asked EFSA (the risk assessor) to develop the Guidance (EFSA 2012). EFSA refers to Guidance as Scientific Opinions (EFSA 2013a), yet the nature of Guidance is very different to that of Scientific Opinions (Pintado 2014). As RAP, Guidance establish the risk assessment framework for applicants and assessors, explaining the principles behind scientific risk assessment and specifying the information and data which industry must provide (EFSA 2014).
Under the General Food Law Regulation, EFSA has a statutory obligation to provide 'open and transparent public consultation' (EC Regulation No.178/ 2002: Art.9). An internal policy, EFSA's Approach on Public Consultations on Scientific Output, guides consultations that it defines as 'an effective exchange on a draft scientific output based on a decision of EFSA to seek comments from the public, namely the non-institutional stakeholders, which include academics, NGOs, industry and all other potentially interested and affected parties' (EFSA n.d.: 3). EFSA has responsibility to inform 'the right target groups' about the consultation in a timely manner and 'identify the stakeholders with relevant expertise and those who will be genuinely affected by the scientific output (producers, users) and those that have a general interest' (ibid.: 6). EFSA can trigger a consultation at three stages: (1) at the beginning of a process to help define and scope the issues; (2) early in the assessment stage, seeking information and data inputs;, and (3) following publication of a draft scientific output to ensure the clarity, completeness and soundness of the draft document (ibid.). However, EFSA's public consultations are science-based, thereby restricting non-state actors to provide only scientifically relevant information. In the case at hand, EFSA states it will 'consider all scientifically relevant comments . . .
[it] will not consider issues related to risk management . . . Ethical and socio-economic issues are also outside the remit' (EFSA 2012: 6).

RESEARCH DESIGN AND ANALYTICAL FRAMEWORK
EFSA's consultation on the draft Guidance is evaluated through expert interviews and documentary analysis. Data collection was achieved through analysis of: (1) documents publically available on EFSA's website (http://www.efsa. europa.eu/www.efsa.europa.eu/), including minutes of meetings for the Panel on Genetically Modified Organisms ('the Panel') and the Working Groups (WGs), expert reports, workshop summaries, the draft and final Guidance and all consultation submissions; (2) élite interviews with policy-makers in DG SANCO (n ¼ 2) and EFSA (n ¼ 4) and independent experts on the Panel (n ¼ 6/24), GM Insects WG (n ¼ 7/14), GM Fish WG (n ¼ 2/9) and GM Mammals and Birds WG (n ¼ 3/11). Participants' comments from the consultation and interview data were analysed through open coding to establish patterns and identify themes (Creswell 2014;Miles and Huberman 2014).
Availability of participants' comments and the Guidance before and after revisions, allowed for examination of the consultation's influence on the Guidance.
EFSA's consultation was evaluated against the normative criteria developed by Rowe and Frewer (2000). These criteria were refined by Rowe and Frewer (2004) and Finardi et al. (2012) and the appropriateness and validity of the criteria tested by Rowe et al. (2008) yielding positive results. These criteria are listed in Table 1 and presented in more detail in the Online Appendix. The next section presents the analysis with results summarized in Table 2. The following section discusses key findings and implications from the case and concludes with practical recommendations.

Task definition
The nature and scope of the consultation was clearly defined but contested. EFSA published the draft Guidance for consultation in June 2012 and invited interested parties to submit scientifically relevant comments via an electronic template on its website (see http://www.efsa.europa.eu/en/ consultationsclosed/call/120621.htm) for 10 weeks from 21 June until 31 August, 2012. By the close of the consultation, EFSA received 720 comments from 35 interested parties including 9 government departments/agencies, 4 university/research organizations, 1 industry, 11 public interest groups and 10 individuals (EFSA 2013b). The full list of participants is available in the Online Appendix. The call for submissions clearly stated the timeframes, requirements and that comments could only be considered if 'scientifically' relevant (EFSA 2012). Was the consultation open and transparent?
Sources: Frewer (2000, 2004); Rowe et al. (2008) This scope was challenged by participants. Five public interest groups argued debate was needed on risk management issues in advance. For example, the Soil Association stated: 'whilst it is correct to state that ethical and socio-economic issues are outside EFSA's remit, the issuing of draft Guidance before such issues are addressed is premature' (EFSA 2013b: 29). GM Freeze stated the Guidance was 'premature as it should follow a wide ranging debate across the EU as

Evaluation criteria Results
Task definition Instructions were clear and participants could understand how to participate, but scope was contested. Participants wanted to discuss a broader range of issues than the remit allowed for, particularly, political, social and ethical issues.

Structured dialogue
The consultation structure restricted full participation of certain non-state actors.
(1) Timing and response time deemed inadequate.
(2) Consultation allowed only one-way communication. Some participants called for further dialogue.
(3) RAP framework predetermined by experiences with GM plants. Some issues were excluded from the debate. (4) Participants were unable to make non-scientific comments.

Resource accessibility
Participation depended on knowledge, time and language resources. Participants had to possess scientific literacy and technical expertise to contribute.

Representativeness
The consultation was ineffective at capturing full range of non-state actors. Only 35 participants engaged and they did not represent the range of affected publics. Independence Some participants challenged the independence of the process and certain experts.

Early involvement
The consultation was held at the end of a long process shaped by experts who defined the issues before the consultation. However, experts and participants conceptualized the RAP's scope differently. Influence The consultation's influence on the RAP was minimal and confined to editorial and structural changes and clarifications. Those participants with scientific knowledge in line with EFSA's understanding of scientific knowledge had more influence. Individuals had little, if any influence. Transparency The way in which the consultation was used in revising the RAP is transparent.
to whether GM animals are acceptable' (EFSA 2013b: 47). Further, public interest groups, individuals and university/research organizations made numerous comments addressing risk management issues despite EFSA's clear instructions they could not be considered.

Structured dialogue
The consultation structure restricted full participation of certain non-state actors. The timing and response time were deemed by some participants and experts to be inadequate. For example, COGEM states: 'The overlap with the summer holidays limits the possibility . . . to submit comments and thus fuels public distrust' (EFSA 2013b: 133). One WG expert states 'It's not long enough because it's a huge document and it takes people a long time to go through it . . . EFSA should give people more time' (Expert #7, interview 2014).
The consultation only allowed for information to flow from participants to experts. EFSA published a summary statement explaining how comments were used but individual comments were not addressed. Some participants called for further dialogue. Two government agencies and two public interest groups requested a second consultation to allow for dialogue between experts and participants. For example, the German Federal Agency for Nature Conservation stated: 'We regard the present document as a first draft and would appreciate a second round of consultation after revision' (EFSA 2013b: 27).
Experiences with GM plants predetermined the RAP framework, excluding certain rules and concepts from debate. The environmental release of GMOs is governed by Directive 2001/18/EC and Regulation (EC) No 1829/2003, 3 developed prior to the emergence of GM animals. The Directive sets out the RAP framework, specifying the six environmental risk assessment stages and areas of risk to be addressed and establishing key concepts, such as comparative, step-by-step and case-by-case approaches, which were deemed non-negotiable for participants. Previous Guidance also shaped this Guidance, particularly the Guidance on the environmental risk assessment of GM plants (EFSA 2010) which establishes the structural framework (EFSA 2013a). For example, the table of contents headings in both Guidances are identical.
Experts shared a view that the environmental risks of GM animals would be similar to those of GM plants and considered the GM plants Directive and Guidance a legitimate framework for GM animals. One expert noted: 'the structure of the Guidance was really derived from the Directive' (Expert #3, interview 2013) while another commented: 'a lot of the ground work had been done already with the guidance for GM plants, so many of the issues that we discussed . . . had been discussed in the context of GM plants' (Expert #1, interview 2013). One expert noted the GM plants Guidance 'was quite useful because it gave us a template into which to fit information' (Expert #5, interview 2013). Most experts interviewed felt sufficient room existed for discussion and did not feel constrained by previous experiences with GM plants. However, one expert described the move from GM plants to GM animals as: 'a real struggle at times' (Expert #3, interview 2013). This expert explained the Insect WG had approached EFSA about this challenge: 'we had some discussion about [whether there was] any chance of changing the format to be more insect specific compared to the others and you know were pretty much told no and it had to comply with the form in the Directive' (Expert #3, interview 2013).
Lastly, the narrow scope of the consultation restricted the full participation of public interest groups and individuals, as noted in the task definition section above.

Resource accessibility
Participation depended on knowledge, time and language resources. Participants had to possess scientific literacy and technical expertise in the area of risk assessment of GM fish, mammals, birds or insects to contribute. In particular, the technical nature of the consultation limited the degree to which individual members of the public were able to participate. Only 10 individuals participated, and although they raised some concerns related to the science, for example about long term effects, risk and uncertainty, they did not articulate their comments in the scientific language used by EFSA.
EFSA provided no supporting information about animal biotechnology. However, owing to the early stage of the technology and secrecy surrounding animal research, participants would be unlikely to comment on the draft Guidance unless they worked in the field. This secrecy is exemplified by one expert: 'I mean just down the road at [name of facility removed] they have all these purple mice and . . . two headed mice, I think if the public knew some of these things they'd be absolutely horrified' (Expert #7, interview 2014).
The consultation was announced in English and the draft Guidance was provided in English only. Over half the participants (18 of the 35) and eight of the ten lay participants came from English-speaking countries (Ireland, Canada, the United States [US] and the United Kingdom [UK]). Only two public interest groups from non-English speaking countries (Agernova and Association Soleil en Tête) contributed in their first language. However, all experts interviewed believed language did not present a barrier to participation because, if one had the technical resources to participate, then one would have the language skills necessary as science is communicated in English.
Experts agreed that the technical nature of the consultation document presented the most significant barrier to participation for those without the necessary knowledge resources. One expert noted 'the average person does not have the scientific knowledge or expertise to be able to comment' (Expert #7, interview 2014).

Representativeness
The consultation did not reach the full range of affected publics. The 35 participants are listed in the Online Appendix organized by type and geographical location. Given the technical nature of the consultation, it is not surprising individual members of the public did not participate. However, significant public interest groups with an interest in GMOs, such as Greenpeace, did not participate. Other non-state actors missing from the consultation were farmers, even though the farming industry is likely to be affected by GM agricultural insects, or those who might be affected by the introduction of GM fish. Some experts expected more member states and public interest groups; others felt the consultation captured all those with an interest (competent authorities and industry). A few experts expected more public interest group involvement, as they considered GM animals to be controversial. Non-state actors networked with EFSA through the Stakeholder Consultative Platform, the GMO Network or the various policy networks around European food governance were more likely to be aware of the consultation and have time to prepare than those who had no previous experience with EFSA. EFSA officials and experts directly invited some non-state actors to participate. In fact, two WG experts were also members of the UK's Advisory Committee on Releases into the Environment, a consultation participant.

Independence
Some participants challenged the independence of the process of developing the RAP and of certain Panel and WG experts. For example, Food and Water Europe and Genewatch noted their concerns about conflicts of interest. In 2012, GeneWatch UK, Testbiotech, Berne Declaration, SwissAid and Corporate Europe Observatory published a document claiming some Panel and WG experts had undeclared conflicts of interest, particularly links to Oxitec Ltd., which develops GM insects (Genewatch et al. 2012). In March 2013, Genewatch filed a complaint with the European Ombudsman alleging EFSA had failed to address the conflict of interest issues raised with respect to several members of the GM Insects WG and that EFSA's rules were breached when two experts acted as participants (European Ombudsman 2013). A few experts raised concerns about interests capturing the consultation. One expert notes: 'I think in some cases it's not a genuine consultation . . . because people have an agenda in advance to the extent that they're not interested in the relative merits of any GMO release, they're against GMO release' (Expert #5, interview 2013). However, there was general agreement among experts interviewed that they could separate scientific facts from interests when considering the merits of individual comments.

Early involvement
The consultation was held at the end of a long process of policy development shaped by experts. Figure 1 outlines the RAP development process. It shows clearly the degree of expert involvement in this process and the timing of the consultation relative to expert involvement. This process is described in more detail in the Online Appendix. Participants conceptualized the RAP's scope differently to experts, yet holding the consultation at the end meant the scope was pre-defined by experts and was non-negotiable for non-state actors. Several government agencies, public interest groups and a university/research organization objected to the exclusion of amphibians, molluscs, crustacean, GM animals developed for pharmaceuticals and EFSA's narrow focus on animals released onto the market. Experts excluded issues some participants argued should be considered in the risk assessment, including traceability, socioeconomic impacts, co-existence and the precautionary principle. Primarily, concerns about the narrow scope of the risk assessment came from government departments/agencies, university/research organizations and public interest groups. In contrast, the industry participant, Oxitec, argued for a reduction in scope and the exclusion of indirect effects, unintended effects and efficacy of the GM animals.
Although EFSA can hold consultations at three stages in the process of Guidance development, it chose to hold the consultation at the end of the process, constraining the ability of participants to frame or challenge expert framing of the issues. To date, EFSA has not held a consultation at the beginning of the process to help define and scope the issues (EFSA official #1, interview 2014). The timing of the consultation was not discussed between the Panel, the Commission and EFSA, but was predetermined based on previous experience. One or two experts felt non-state actors had little to offer upstream and the value of the consultation was in providing clarification, therefore the timing was entirely appropriate. Others felt a genuine consultation was needed earlier to allow non-state actors to shape the whole process and EFSA was probably not the most appropriate EU body to engage non-state actors. EFSA officials acknowledged holding a consultation at the end of the process 'may be a little bit late. To some extent the concerns that the genuine public person on the street has ought to be part of what goes in at the beginning in terms of shaping the whole process' (EFSA #1, interview 2013).

Influence
The consultation had minimal influence on the final Guidance. The majority of revisions addressed structure, organization, definitions and consistent use of terminology, improved the explanation of concepts and increased clarity so applicants and risk assessors could better understand risk assessment requirements. Clarity and consistency were achieved by adding a new section covering the risk of pathogens, infections and diseases from GM insects (EFSA 2013a: Section 4.2.3) and expanding a section on risks to human health to include risks to animal health, specifically that venom or saliva from certain stinging or biting insects may cause localized or systemic allergic or toxic reactions in humans (ibid.: Section 4.2.7). Lastly, a new section (ibid.: Section 3.7) was added to provide further clarity on modelling.
In its response to the consultations, EFSA stated: 'Most of the general comments received were helpful and constructive, aiming at the improvement of the draft text of the Scientific Opinion. Stakeholders also provided very helpful suggestions for editorial improvements and clarifications' (EFSA 2013b: 4). EFSA officials and most experts interviewed felt the consultation's value was in improving clarity and consistency. One expert stated: 'I think one of the main things that came over was that some areas of the Guidance were not very clear . . . they said that they couldn't . . . clearly understand . . . what information should be supplied' (Expert #7, interview 2014). Another noted: 'it's more about the writing essentially, making sure that it's as clear as it could be' (Expert #2, interview 2013).
A few experts felt the consultation provided little value and had minimal influence. For example, one expert stated: 'In general I don't know what, how much influence those comments have on the final decision but it's probably minor' (Expert #4, interview 2013). Another stated 'In regard to my particular part there was almost nothing of relevance' (Expert #6, interview 2014). There was agreement the consultation made no substantive contribution. For example, one expert states: 'I don't think that the consultation necessarily threw up . . . any major issues that made you think well I need to go back to the drawing board' (Expert #5, interview 2013). Another noted: 'I can't recall any substantive changes' (Expert #8, interview 2014).
Participants had differing degrees of influence. Those with more specialist scientific knowledge in line with EFSA's scientific knowledge had more influence. As an EFSA official states: 'we see a lot of valuable comments from member state competent authorities . . . from the NGO side it's mixed feelings' (EFSA official #1, interview 2014). Following the consultation, the Panel and WGs reviewed the relevant comments and revised the document (EFSA 2013b). Although the Panel and WGs had responsibility for addressing comments, they only saw those EFSA's GMO Unit deemed scientifically relevant. One expert suggested the filtering 'eliminated the more extreme perspectives' (Expert #1, interview 2013). The vast majority of comments deemed scientifically irrelevant were made by university/research organizations, public interest groups and individuals. Those comments were deemed to fall outside EFSA's remit and addressed social, ethical and political issues such as animal welfare, labelling, public acceptance, traceability, legitimacy and consideration of alternatives. These comments had no influence as the experts did not see them. For example, one expert noted 'No, I didn't [see all the comments] . . . the secretariat would filter those comments in terms of what they deemed . . . scientifically relevant' (Expert #2, interview 2013).

Transparency
On 23 May 2013, approximately nine months after the consultation, EFSA published the 720 comments and provided eight pages summarizing the comments and explaining how they were addressed (EFSA 2013b).

DISCUSSION AND CONCLUSIONS
The results, summarized in Table 2, draw into question the quality, effectiveness and impact of EFSA's consultation in the development of Guidance on the environmental risks of GM animals and challenge Dreyer and Renn's claim that the inclusion of non-state actors in EFSA's decision-making is a truly 'procedural innovation ' (2014: 324). Evaluation against the Rowe -Frewer criteria shows participants challenged the consultation's boundaries and were unable to have their say, primarily owing to the timing and response time of the consultation, lack of opportunity for dialogue and exclusion of non-scientific comments, particularly as EFSA's remit does not allow it to consider nonscientific issues in development of Guidance (although Hartley and Millar [2014] observe EFSA has blurred the lines of its remit in the past). 4 The consultation was ineffective at capturing the full range of affected publics, resulted in an official complaint about the independence of experts and, more importantly, had minimal impact, particularly owing to the timing of the consultation at the end of a long, expert-driven process. According to Arnstein's (1969) 'ladder of citizen participation', EFSA's consultation was a form of tokenism, allowing non-state actors a (albeit limited) voice, but did not result in impact -there was no transfer of power from participants to policy-makers.

Policy masquerades as science
The Commission and EFSA failed to recognize the Guidance as irredeemably a policy document rather than a purely technical one and developing it is the Commission's responsibility. At interview, Commission officials stated: 'we wouldn't call it policy' (DG SANCO official #2, interview 2014). Another official stated: 'Maybe it would be good to clarify . . . it's not a policy' (DG SANCO official #1, interview 2014). Further, Commission officials stated the Guidance was the Panel's responsibility, not the Commission's nor EFSA's: 'it's not the Commission's Guidance and . . . we wouldn't even say it's . . . EFSA's Guidance, it's the Panel who adopt it' (DG SANCO official #2, interview 2014). Another official repeated: 'it's just a Guidance . . . not a legal document by the Commission' (DG SANCO official #1, interview 2014). The fact that it was not a Commission document did not, however, ensure it was a policyfree text.
A consequence of the failure to recognize the Guidance as RAP is that policy masquerades as science, raising the authority of experts and constraining the type of knowledge non-state actors are able to contribute. Given the historical prevalence of technocracy in the EU (Radaelli 1999) the results of this study (non-state actors had little influence on policy compared to experts) may be unsurprising. For example, in technocratic models of decision-making, science not only informs policy-making but is portrayed as if it provides a sufficient basis for policy-making (Millstone et al. 2008). In technocratic models, policy-makers are democratically accountable: scientists inform policy-makers and although their advice may be the sole basis of policy, it is the policymakers who decide and therefore make policy. However, in the case of RAP development for GM animals, the framing of the Guidance as a scientific rather than a policy document resulted in the Commission and EFSA relinquishing authority to scientific experts on the Panel -a group of experts with no democratic accountability. Experts not only informed policy, they made policy. The Commission triggered the development of the RAP and EFSA and the Commission negotiated the scope of the RAP, particularly in terms of which GM animals would be included (EFSA and DG SANCO officials, interviews 2014). Following this negotiation, EFSA instructed the Panel to develop the Guidance. The Commission was statutorily restrained from providing input into the process from this point on, although officials observed Panel and WG meetings. EFSA took a supportive role under the direction of the Panel when requested to do so. The Panel adopted the Guidance on 18 April 2013. On 23 May 2013, EFSA published the final Guidance on its website, officially establishing the RAP (EFSA 2013c). EFSA and the Commission provided no input or comment following the Panel's adoption of the Guidance. In this case, policy decisions were made by independent scientists without democratic accountability.

Expert vis-à-vis non-state actor involvement in policy
This study joins a growing number of cases suggesting experts move beyond influencing policy to actually making policy (Busuioc 2014;Lawrence 2014;Millstone et al. 2004). It supports Millstone et al.'s (2004) contention that experts make decisions when policy-makers fail to acknowledge the limitations of science for risk decision-making. In this case, the Commission, EFSA and the scientific experts on the Panel and WGs believe the Guidance is an objective, science-based document free of values. All those interviewed believed the Panel was best-placed to make the Guidance because Panel members were free of political values. However, RAPs are not objective scientific opinions, but policies involving value choices. By framing the Guidance as a scientific document, these value-choices were hidden from non-state actors and allowed experts to make policy. Participant frustration with this 'cloaking' of values was clear in the consultation and the issue has been raised frequently in the academic literature (Wickson and Wynne 2012).
The results of this study support Busuioc's (2014) assertion that regardless of the contentious normative debate about the role of non-state actors vis-à-vis experts at the interface between regulatory agencies and the Commission, the distinction between risk assessment and risk management is not being upheld in practice. The blurring of the lines between regulatory agency (EFSA) and policy-maker (Commission), where independent agency experts become policy-makers, results in policy decisions made by scientists without democratic accountability. The concern here is not that the political is shaping the scientific, but that the scientific is shaping the political and in doing so masking political choices being made by scientific experts. Although risk assessment is the responsibility of EFSA, development of risk assessment policy is the responsibility of the Commission, according to Codex rules to which the European Union (EU) is a signatory.
A reinvigorated role for non-state actors in shaping RAP need not result in a blurring of the lines between risk assessment and risk management as Gabbi (2007) fears. Rather, RAP needs to be removed from risk assessment (EFSA's responsibility) and placed into risk management (the Commission's responsibility). If the Commission takes responsibility for RAP development in the future, it may need to involve non-state actors and the wider public upstream in the policy process to allow them to examine and contribute to the value-judgment shaping: the scope of future risk assessment; what counts as evidence; how much evidence is needed and how it is to be interpreted; how uncertainties should be addressed; and how precaution should be applied. Deliberative mechanisms may be a preferable tool early on in the process if there is a genuine desire to understand non-state actor perspectives on RAP, particularly in cases of controversial technologies such as GM animals (Dreyer and Renn 2014). This case study highlights the lack of non-state actor participation in what has the potential to be a controversial issue, and suggests further research to explore the reasons why nonstate actors with an interest in GM animals, such as environmental groups , farming and consumer associations and public health advocacy groups, did not participate in the consultation.
This case study joins a growing number of cases where experts are found to be making policy. It suggests a comprehensive approach to studying this phenomenon is needed, involving comparative work to tease out the factors responsible. In addition, it points to a need to distinguish Guidance from risk assessments (or market authorization procedures) when examining independent regulatory agencies' scientific opinions. This distinction is not clear in the literature or in practice. Busuioc (2014) notes scientific opinions are one of the most blurred areas of agency responsibility, defining them as the outcome of specific risk assessments in the context of market authorization procedures. A quick search of EFSA's publications reveals over 100 Guidance documents developed over the last 10 years; 5 however, these policies have eluded investigation (Millstone et al. [2004] and Millstone et al. [2008] are notable exceptions). There is little recognition from Codex about the challenges of identifying RAP or information available to risk management agencies tasked with RAP development. Research is needed to reveal 'best practice' cases, where RAP has been conducted by risk managers) rather than risk assessors, to identify the process through which RAP was identified and the mechanisms employed for non-state actor involvement.
Biographical note: Sarah Hartley is a research fellow on the Leverhulme Trust programme, 'Making Science Public', at the University of Nottingham.