Are we all on the same page? A qualitative study of the facilitation challenges associated with the implementation of deliberative priority-setting

ABSTRACT Collaborative governance has given rise to decision-making methodologies promoting democracy, inclusivity and transparency. This is exemplified by deliberative priority-setting (DPS) that blends cost-effectiveness analysis with stakeholder deliberation. Little is known however, about the facilitation challenges when ‘technical’ and ‘social’ elements are combined in a methodology. This paper investigates the facilitation challenges of implementing a DPS project within the English National Health Service (NHS). Our study examines the relationship between facilitation and the effectiveness of DPS processes, highlighting the importance of knowledge management as facilitators seek to translate technical information, to enhance the deliberative experience and promote legitimate decisions.


Introduction
Over the last 30 years, policymakers and public managers have adopted various methodologies to define service priorities and allocate resources (McCafferty 2012). The late 1980s, for example, saw the rise of 'cost effectiveness analysis' (CEA), reflecting 'best value' imperatives of New Public Management (Garber and Phelps 1997). Participatory decision-making became commonplace in the late 1990s, reflecting a new era of collaborative governance (Ansell and Gash 2008;Fischer 2003;Fung 2015). These methods embrace deliberative techniques to engage stakeholders in more democratic decision-making (Ansell and Gash 2008;Doberstein 2016;Osborne, Radnor, and Strokosch 2016). Those advocating a deliberative approach are often critical of the reliance upon 'technical' data associated with CEA, which upholds expert opinion to the exclusion of other perspectives (Edejer 2003).
These transitions in policy decision-making can be seen with priority-setting methodologies that are concerned with determining the types and levels of public service provision in the context of finite financial and human resources and changing public expectations (Airoldi et al. , 2014Bruni, Laupacis, and Martin 2008;Cromwell, Peacock, and Mitton 2015). Although such decisions have traditionally been determined by policymakers on the basis of 'technical advice', contemporary priority-setting embraces deliberative principles. Specifically, 'deliberative priority-setting' (DPS) aims to combine the 'social' insight of stakeholders with 'technical' evidence through structured deliberation 2014;Campbell 2010).
Whilst deliberative techniques are regarded as the path to collaborative advantage (Huxham 1993), it is increasingly recognized that deliberation may have a 'dark side' (Nabatchi 2010). In particular, power dynamics between stakeholders may compromise inclusivity, and decisions may appear subjective or lacking in technical objectivity Hong 2015;Irvin and Stansbury 2004;Litva et al. 2002;Nabatchi 2010). To combat these issues, 'good' facilitation is regarded as the key to managing power plays and ensuring all stakeholders are able to influence decision-making (Fung 2007;Kaner 2014).
Whilst there is growing recognition of the importance of facilitation in participatory decision-making more broadly (see, for example, Hong 2015; Irvin and Stansbury 2004;Litva et al. 2002), there is limited understanding in the context of DPS. For example, we know very little about how 'technical' (e.g. cost and outcome data) and 'social' (e.g. stakeholder preferences) elements are combined in a deliberative context (Choi and Robertson 2013;Fung 2015). Our ethnographic study seeks to contribute to this gap by investigating the role of facilitation in the implementation of DPS methods and to understand how facilitation might influence the deliberative experience.
Our specific contribution is to highlight knowledge management as a particular facilitation challenge, specifically because DPS incorporates both 'technical' evidence and 'social' insight. We suggest that to enhance the deliberative experience and to promote equal and sustained stakeholder engagement, facilitators must translate complex, technical information, so that it can be accessed and understood by all. In drawing our conclusions, we offer practical recommendations on how to manage knowledge through effective facilitation.

The challenges of DPS
To explore some of the challenges associated with the implementation of DPS, we turn to the literature on collaborative decision-making, where we identify three prominent and linked themes. The first relates to the design and implementation of deliberative processes, drawing attention to the processes of decision-making (Bryson, Crosby, and Stone 2006;Fung 2006Fung , 2015 and how stakeholders are engaged and supported through these processes (Mansbridge and Parkinson 2012;Parkinson 2004). The second examines stakeholder engagement, highlighting the importance of stakeholder representation (Fung 2003(Fung , 2006 and stakeholder involvement in decision-making (Choi and Robertson 2013;Fung 2006;Robertson and Choi 2012). The third zooms in on the deliberative experience, for example, how power dynamics influence discussion (Choi and Robertson 2013;Martin, Giacomini, and Singer 2002;Purdy 2012) and whether stakeholders regard decision-making as inclusive, fair and legitimate (Fung 2015(Fung , 2006Johnston et al. 2011).
In this paper, we are theoretically interested in the interplay between these three themes, namely how the implementation of the deliberation process influences facilitation choices and, in turn, how facilitation might affect stakeholder engagement and the deliberative experience. To explore this further, it is important to consider what deliberation ideally entails. According to Campbell (2010), deliberation is 'a process of collective and procedural discussion where an inclusive and representative set of stakeholders consider facts from multiple perspectives, converse with one another to think critically about options, and through reasoned argument refine and enlarge their perspectives, opinions and understandings' (Campbell 2010, 10).
A positive deliberative experience therefore depends on the involvement and empowerment of stakeholders (Campbell 2010;Robertson and Choi 2012). However, this poses a particular facilitation challenge as heterogeneous stakeholders bring different interests, motives, resources, cultures and forms of social status to the deliberative encounter (Fung 2003;Robertson and Choi 2012). These differences often give rise to power imbalances and forms of social capital that can influence deliberation (Huxham and Vangen 2013;Purdy 2012). For example, many studies of collaborative decision-making highlight the continued influence of experts and professionals in shaping policy, despite claims to stakeholder engagement (Butterfield, Reed, and Lemak 2004).
Within the literature, it is well recognized that 'good' facilitation is necessary to promote stakeholder engagement, manage power imbalances and support effective deliberation (Airoldi et al. 2014;Ansell and Gash 2008;Doberstein 2016;Campbell 2010;Johnston et al. 2011). In general terms, facilitative leadership is regarded a precondition for successful and legitimate decision-making, with an emphasis on '. . . setting and maintaining clear ground rules, building trust, facilitating dialogue, and exploring mutual gains' (Ansell and Gash 2008, 554). These preconditions are commonly perceived as the path to the 'empowerment condition' (Gibson, Martin, and Singer 2005), whereby facilitators should seek to '. . .minimize power difference in the decision-making context and to optimise effective opportunities for participation in priority-setting' (2360).
Of specific concern to our study are the facilitation challenges of communicating technical information to stakeholders to promote fair and legitimate deliberation. It is recognized, for example, that facilitators working with DPS methodologies must ensure that both technical and social elements are equally considered and that all stakeholders can participate (Airoldi et al. , 2014Doberstein 2016;Campbell 2010;Johnston et al. 2011;Reagan-Cirincione 1994). As suggested by Mabin et al. (2001, 47), 'the very nature of multiple criteria problems is that there is much information of a complex and conflicting nature, reflecting different viewpoints and 'often changing with time'. As such, facilitators need to address two key factors.
First, data must be communicated clearly so that stakeholders can understand it (Berner, Amos, and Morse 2011;Callaghan and Wistow 2006;Campbell 2010). For example, Hong (2015) suggests it can be problematic when stakeholders lack knowledge and understanding of budgetary demands, and there are often trade-offs between inclusivity and knowledgeability. Second, as priority-setting is a political business, asymmetries of knowledge between stakeholder groups need to be recognized and, where possible, minimized (Airoldi et al. 2014;Berner, Amos, and Morse 2011;Callaghan and Wistow 2006;Campbell 2010). In particular, there is a tendency for professional groups to dominate decision-making, leading to outcomes that represent professional interests (Huxham and Vangen 2000;Purdy 2012;Warner 2006). As such, facilitation needs to manage the dynamics of power and knowledge.
To address these challenges, facilitators require technical skills to understand economic data about the opportunity and margin costs associated with different options (Purdy 2012); social skills to enable deliberation around competing priorities (Mabin et al. 2001); and analytical competencies to combine technical and social elements (Williams et al. 2014). Despite the popularity of DPS methodologies in public service decision-making (Sussex et al. 2013;Thokala and Duenas 2012), there remains a lack of understanding about the implementation of this methodology, in particular the relationship between effective DPS and facilitation. This paper contributes to this gap by examining the practical experiences of DPS from the perspective of stakeholders and facilitators. Specifically, we are interested in how facilitators manage knowledge throughout the deliberative process and how this, in turn, influences the deliberative experience. Our focus in this paper contributes to our broader understanding of what constitutes effective facilitation within collaborative decision-making, emphasizing the importance of knowledge management in the process (Campbell 2010;Cromwell, Peacock, and Mitton 2015).

The case study
The paper reports on the implementation of a DPS methodology called the 'Socio-Technical Allocation of Resources' (STAR) approach 2014;Health Foundation, 2015). STAR attempts to combine 'hard' technical data on service costs and outcomes, with 'soft' social insights of stakeholders. In practice, this involves communicating technical data, such as ratios of costs and health gains, through visual aids, referred to as 'value triangles'. The STAR guidance suggests that, in this form, stakeholders can more easily draw on technical data during deliberations. These deliberations occur through 'decision conferencing' workshops where stakeholders review data, discuss options and work together to prioritize services.
STAR is designed to be implemented through a series of linked stages. Initially, a preliminary 'set-up meeting' is convened, where the primary decision-makers (in this case, an NHS project manager, clinical leaders from a regional NHS Clinical Senate and Network and contracted data analysts) determine the focus and scope of the project (in this case, the reconfiguration of a regional cancer pathway). At this meeting, the 'areas' for prioritization (in this case, 'diagnosis', 'treatment' and 'survivorship') are identified, and the range of 'options' relating to these priority areas is selected.
These decisions determine the types of technical data required to inform these options and also the range of stakeholders to involve in decision-making (clinicians, patient and public representative, commissioners and volunteers). Significantly, STAR guidance emphasizes the importance of involving expert analysts to collate data and experienced facilitators to support stakeholder deliberation (Airoldi et al. 2014). It describes facilitators as needing to be objective and neutral, as having a good working relationship with data analysts and enabling stakeholder engagement with technical data (The Health Foundation 2015). The next step involves a 'kick-off' meeting, to introduce stakeholders to the methodology, outline priority areas and options, and present the technical data. STAR guidance suggests stakeholders should be invited to question and clarify the presented options and data before participating in decision conferencing. Decision conferencing is then organized through two or three workshops depending on the scale of the project. Here, stakeholders assess, score and rank service options, drawing on the technical data and their personal insight and experience. Scoring takes place sequentially at the individual, small-group and full-group levels, with the intention of reaching consensus through deliberation. The process ends with a 'task and finish' workshop involving the project team, clinical experts and analysts, where the outcomes of the process are identified to inform subsequent service change.
This case study is significant because the organization and delivery of healthcare exemplify the contemporary challenges of collaborative governance, where decisionmakers face growing public demand for quality and access, and also constrained financial and specialist resources. Within this context, DPS has attracted interest from policymakers and service leaders because of the potential to combine multiple stakeholder perspectives with data on clinical outcome and cost. Our study examined the implementation of the STAR approach to DPS within one region of the English NHS, where it was used by an NHS Clinical Senate and Network to prioritize decisions for the reconfiguration of a regional cancer care pathway. It should be noted from the outset that some of the activities observed during our study differ from those recommended above, as a consequence of local implementation choices by project leaders. Our analysis of this decision-making process pays particular attention to facilitation challenges, focusing on how the technical and social aspects of the methodology were combined during the stakeholder workshops. Importantly, the study authors are independent of both the STAR methodology and the project team implementing this decision-making process.

Data collection
Our ethnographic study investigated the implementation of the STAR methodology. Ethnography strives for a rich description and understanding of how people 'experience, interpret and structure their lives' (Burgess 1984, 3). Data were collected over nine months through a range of activities, including non-participant observations of pre-and post-workshop meetings between the project team (4 h); two stakeholder workshops (6 h per workshop); and the task and finish group (2.5 h). While the stakeholder workshops and the task and finish group were observed as integral stages in the STAR methodology, the pre-and post-workshop meetings enabled us to observe and understand the context of the implementation of the methodology by examining the project team's reflections and practical decisions about the decision-making process. These meetings took place before and after the two stakeholder workshops, with each meeting lasting for 1 h.
Observations examined how stakeholders and facilitators interacted during workshops, used technical data in ranking activities, communicated their experiences and arrived at consensus. The pre-and post-workshop meetings were observed by one researcher, whilst the workshops were observed by three researchers simultaneously, who recorded independent observations and reflective interpretations. It is important to acknowledge that the project set-up and the kick-off meeting occurred before the start of the research and were therefore not observed. To explore this preliminary stage, we rely on stakeholder interviews for insights into how the methodology was understood and applied within the context of this project.
Observations were complemented with semi-structured interviews with 31 stakeholders, of which four were interviewed more than once to capture their experience throughout the process (see Table 1 for details). Interviews were conducted at various time points during our nine-month study to capture the key stages of the process, specifically before and after the workshops. In addition, a focus group was convened with patients and carers who participated in the workshops. Interviews explored stakeholder experiences of deliberation, their interactions with technical data, the role of the facilitators and the quality of the process. All interviews and focus groups were recorded and transcribed verbatim. Documentary data was also reviewed, including project initiation documents, technical data packs and decision-analysis diagrams. The study received University ethical clearance and was approved by the NHS Clinical Senate.

Data analysis
Data were analysed following an abductive approach (Reichertz 2007;Richardson and Kramer 2006;Timmermans and Tavory 2012). Abduction is an iterative approach that seeks a pragmatic alternative to deduction and induction. Prevailing theoretical debates and constructs are used to sensitize qualitative enquiry, and emergent empirical observations are analysed in light of these sensitizing concepts to develop novel analytical heuristics and theoretical propositions. In practice, analysis follows a more traditional interpretative approach of open coding and constant comparison, but where comparison involves attending to not only the internal consistency of empirical codes, but also the relevance and similarity of emerging analytical explanations with existing theoretical constructions. Our abductive approach is depicted in Figure 1. In the first instance, all of the authors coded a sample of data to ensure analytical consistency. Through this process, we continuously related back to existing theoretical concepts as we looked for both confirmatory and novel explanations. Through this Two iterative process, ideas were refined and clarified to hone our analytical themes and contributions to theory. For example, our empirical codes 'emphasis on technical data' and 'dominance of analysts' were related back to existing theoretical debates about 'stakeholder engagement', which informed us about where our findings supported existing research and offered novel insight. In reflecting on how our empirical data related to the theory, we developed the analytical theme 'epistemological differences' (which we elaborate in our discussion below).

Findings
Reflecting our ethnographic approach, our findings describe the main stages of the DPS approach, beginning with an overview of the project set-up and the kick-off meeting, followed by a description of 'Workshop One' and 'Workshop Two'. We do not present findings from our observations of the 'task and finish' group in this paper as we wanted to focus on the stages of the process that involve stakeholder engagement. Through our descriptive account, we 'zoom in' on the role of facilitation, especially in the management of knowledge, and consider the impact upon the deliberative experience. It is worth noting from the outset that our findings present an account of a 'troubled' case.

Set-up and kick-off meeting
Project set-up involved the formation of the project team, comprising an NHS project manager and clinical leads from the regional Clinical Network and Senate, additional clinical specialists from across the region and the contracted data analytics experts. The project team defined the project focus, for example, the redesign of a cancer pathway,  together with the priority areas and related options, and also clarified the technical data requirements, for example cost and outcome data for each option. At the outset, the project manager explained how STAR required the involvement of specialist data analytics to help process technical data and facilitate the use of technical data in deliberations. The data analysts who were contracted were experienced in preparing and analysing complex healthcare data for the purpose of prioritysetting, and in this case study, they took responsibility for collecting and compiling NHS activity data for regional cancer care, such as diagnosis rates, referral times, treatment options, length of care, survival rates and costs. This became the primary source of technical information throughout the process.
During this preliminary stage, the project manager, in consultation with the clinical leaders, identified stakeholder groups to involve in the decision-making process. Clinical representatives were selected because of their specialist work in relevant cancer services, primary care or rehabilitation. Representatives were also identified from NHS commissioners, because of their responsibilities for contracting cancer care. Patient and carer representatives, as well as voluntary sector representatives were identified through a cancer survivor network, or established links with care providers. When identifying stakeholders, the project manager sought to balance stakeholder representation with the practicalities of bringing people with different experiences and concerns together to embark on a complex decision-making process: "the stakeholder group is very large and very different. . . So in terms of doing the scoring etc. it's very good to have that kind of stakeholder group. In terms of actually progressing the project it's actually much harder" (Project manager) In the 'kick-off meeting', the project manager reported that they introduced invited stakeholders to the overall purpose of the decision-making process. At this meeting, the data analysts also presented an overview of the STAR approach and used technical data to describe the current variations in cancer care across the region, which also served to justify the need to reconfigure services. Guided by the data analysts and project manager, stakeholders were asked to agree on the 'priority areas' of the care pathway, including entry to pathway, investigations and diagnosis, informing patients, treatment, and survivorship and follow-up. These priority areas provided the focus for deliberation in subsequent workshops, where, as described next, stakeholders were asked to consider service options within each area, for example, different referral methods, diagnosis test, or rehabilitation packages.
According to the STAR methodology (Airoldi et al. , 2014, stakeholders should have the opportunity to review technical data and deliberate the priority areas at this early stage. However, this process was largely perceived to be about 'information-giving', with little opportunity for stakeholders to discuss or question the priority areas and technical data: it did feel a little bit prescriptive in terms of "right, these are the things we need to look at and these options have been developed". It was sort of, "well, where did they come from then?" A similar issue was raised by some stakeholders who were concerned about the lack of genuine 'options' for the cancer pathway because of the highly specific nature of individual patient care: "the problem is in [reference to specific type of cancer] we don't feel there are those choices to make. You can't say "oh, shall we do curative or palliative care?" you need both of them and in lots of cases there isn't a number of options to choose from. . ." (Cancer network representative) Clinical stakeholders suggested that dividing the pathway into priority areas and option 'bundles' did not accurately reflect how the pathway worked and that subsequent deliberations were incompatible with the realities of cancer care: 'these options are not suitcase you can take off the conveyer belt, they are all part of a journey through the care pathway' (Clinical) "It isn't quite as simple as saying procedure A versus procedure B for a specific issue because some patients are going to have everything and some aren't" (Commissioner) Following the kick-off meeting, all stakeholders received a 'data pack' summarizing the costs and outcome data for the priority areas and options. This was compiled by the data analysts, with the assumption that stakeholders would 'do their homework' by reading the data to be 'well briefed' for the workshops.
As we discovered, however, many stakeholders did not complete this 'homework' because they struggled to understand the data. Public and patient stakeholders said they lacked the necessary expertise to process the data, to the extent they were discouraged from preparing for the workshops: "the data was very overpowering and I didn't want to ask questions because I felt daft" (Patient focus group participant) Professional representatives reported being familiar with this type of routine 'service data', but despite this, some questioned the validity and accuracy of data where they did not feel it reconciled with their professional experience: "unfortunately the data was incomplete because it clearly showed differences between different centres in terms of cost and terms of how people were being treated, but unfortunately they missed out quite a bit of the major expense which is chemotherapy and radiotherapy." (Clinician) The data analysts reflected on the challenges of compiling the technical data, suggesting that it was difficult to find a common language that was understandable and usable for all stakeholders: 'we tried to create a summary of the data analysis but frankly, it was impossible because the language and currencies [of the stakeholders] were so different' (data analyst) Although the data analysts acknowledged that some stakeholders struggled to understand the technical data, we found little evidence of any attempt to present this data in different ways (e.g. using visual aids) or to create opportunities for stakeholders to challenge the data, as suggested in STAR guidance .
Rather, it appeared that the data analysts began to dominate, even dictate, the decision-making process through their access to and control of the technical data and their ability to present particular views about the different service options. Furthermore, the lack of opportunity for stakeholders to shape the technical data, priority areas and options further suggested the dominance of the data analysts and the primacy of the technical data from the outset.

Workshop one
During the workshops, the data analysts contracted an expert facilitator to assist with the deliberative activities. This facilitator was known to the analysts and was experienced in Multi-Criteria Decision Analysis in healthcare. As with the kick-off meeting, the data analysts continued to dominate the process by, for example, using technical 'evidence' to frame discussions or even close down deliberation where the evidence suggested an option was too costly or indicated poor outcomes.
With regard to explaining workshop activities, most stakeholders reflected that the expectations and instructions were complicated and 'jargonistic'. Initially, stakeholders were organized into table groups of up to eight people, with representation from different stakeholder groups. Stakeholders were then asked to individually 'score' and 'rank' the presented options for each priority area, drawing on both the technical data and their personal experience, before discussing their scores on their table and reaching a consensus score. However, we observed that stakeholders were provided with little guidance about how this exercise should be undertaken. For example, it was assumed by the facilitator that stakeholders had appraised, or could use, the 'data pack' to complete this activity, but there was no explanation of what the data meant or how it should inform decision-making: '. . .if you actually went round and asked everybody, how many people would have actually understood the process. It would have been very low. . .I think it needs to be broken down even further and people need time to assimilate ideas and understand' (Cancer network representative) We observed how stakeholders struggled to reach individual scores, and many openly compared and copied each other. Reflecting on this activity, participants described how it felt like they were working under 'test conditions', and many had difficulties in reaching what they thought were the 'right' answers to the different options: 'I'd never done anything like that [scoring] before and it was just like a foreign language. I was looking around me trying to work out whether everyone else was doing it right and I felt like I was back at school in an exam' (Patient representative) During the scoring exercise, the facilitator recognized that some stakeholders were struggling with the exercise and used various analogies to explain how the technical data could be combined with personal data to inform the scoring. For example, they explained that 'when buying a car, we know that all cars basically do the same thing by getting you from A to B, but personal preference comes into the decision like colour and you have to weigh up the options to decide which one offers the best value for money'. Such analogies appeared to create a greater sense of engagement in the process: 'the analogies were. . .It's like in simple terms you've got to explain it even if you've got people who understand it because that's making sure we all get it' (Patient representative) During table discussions, there were further inconsistencies in how groups interpreted the task of reaching a 'consensus score'. Some used the opportunity to share individual scores and calculate an average, whilst others engaged in more open dialogue to reach a 'reasonable' score for each option. However, no table used the opportunity to discuss the issues influencing their decisions, and in subsequent interviews, we found that participants were generally unsure of the purpose of the group discussion: "on our table we weren't really sure whether we should add up all our scores and just calculate an average or whether we should be trying to change each others minds and fight for our own position" (3 rd sector representative) Significantly, table discussions were influenced by interpersonal and social dynamics, including variable levels of rapport, openness and communications. We observed, for example, how in some groups patients and carers were given limited opportunity to elaborate their views or qualify scoring decisions, being encouraged instead to focus on 'the task at hand' and not get sidetracked. In these groups, 'professional' voices appeared to dominate discussion, especially where specialists were more able to engage with 'familiar' technical evidence and marginalize patient perspectives. Reflecting on these observations, some patients said they were reluctant to share their views because they felt they were subjective or undervalued, and others felt embarrassed talking about personal experiences in the company of clinical expertise: "we obviously then discussed it as a group when there were differences and that's where your perspective as a patient got slightly changed because when you listen to it from the consultant's point of view and the administration point of view you thought then "Oh yes, perhaps the patient perspective's not quite as important as I think it is," (Patient representative) At other tables, professional stakeholders encouraged public representatives to speak up and responded positively to the contributions that others made, for example, by giving people the space to explore ideas and by openly thanking each other for their insights. Patients from these tables later described how they were 'listened to' and some clinicians claimed to be 'enlightened': ". . .particularly as a surgeon you do tend to focus on the curative treatment part of the pathway and I think it's been important to be thinking of other areas. . .I think the patient group as a whole has been quite useful for me" (Clinician) Following table discussion, the groups were asked to feed back their 'agreed' scores to the wider room. At this point, and in line with STAR guidelines, open deliberation should have taken place to discuss and determine the consensus scores for the priority areas. However, we observed how the data analysts, with the support of the facilitator, collected the table scores with limited discussion and then worked with the scores independently over lunch to produce 'value triangles' for the different service options. These visual representations were later presented and were generally well received by stakeholders: 'they allowed you to put some degree of economic cost to value it at. . .It was very striking. . . when you saw the value triangle it was pretty damn obvious what you should do because it did really focus your mind on which strategy was actually going to be most appropriate' (Clinical stakeholder) In accordance with STAR guidance, facilitators should use 'value triangles' as a visual aid to feedback, summarize and discuss the costs and outcomes for the different options. We observed, however, that this was not encouraged, with both the facilitator and data analysts closing down deliberations with phrases such as 'if you agree we will move on' (observation notes). We observed that some participants attempted to challenge the consensus score and value triangles, but the data analysts appeared reluctant to acknowledge competing views or change scores. As such, the workshop ended, like it started, with an emphasis on the technical data and quantitative scores, with the more subjective and social elements used in a limited way to validate or clarify measures, rather than to offer an alternative.
In summary, stakeholders were unclear about the purpose of the activities. While the workshops were presented as an opportunity to influence decision-making, it was clear that the analysts' primary expectation was that stakeholders would use the technical data to complete the scoring of the predetermined options. Despite the facilitator encouraging more open deliberation, there were variable levels of discussion amongst stakeholders, with some evidence of professional voices (in concert with the analysts) limiting deliberation. However, there was evidence of some good deliberation where stakeholders encouraged each other to share their views.

Workshop two
The second workshop was convened two months later, to complete the scoring for the remaining priority areas and options. We noted two significant procedural changes from the first workshop. First, the workshop was led by the expert facilitator, with far less input from the analysts, and second, table-level facilitators were introduced to lead deliberative activities. Table facilitators had different backgrounds and included the project manager, a data analyst, a cancer network representative and a clinical lead. Importantly, they were willing volunteers, rather than being selected on the basis of facilitation skills or experience, receiving a general overview of the STAR methodology, but no formal facilitation training. These marked changes in the overall approach were introduced because the project team recognized that there were problems in workshop one with facilitating stakeholder deliberation, combined with a preoccupation with the technical data.
Overall, table facilitation appeared to enhance the scoring activities. For example, facilitators were observed as helping participants link technical data to their own experiences and encouraging stakeholders to 'think through' their scoring. Where table facilitators appeared to embrace the principles of deliberation, dialogue was observed as more open, with fruitful discussion leading to consensual scoring. We found, for example, that some table facilitators created a space for marginal stakeholders to share their views and that these were recognized and valued, whilst simultaneously curtailing the dominance of the professional perspective.
In contrast, we observed how other table facilitators encouraged only limited deliberation between stakeholders, and seemed much more preoccupied with completing the scoring tasks within agreed time frames. In these cases, it seemed that they often turned to professional stakeholders to reach a quick decision because of their familiarity with the technical data and expertise in service delivery. At these tables, we observed little effort to debate scoring and, instead, table scores were reached by calculating an average based on individual scores. The differences observed between table facilitation appeared to stem from each facilitator's understanding and experience with DPS methodologies, rather than from the intrinsic characteristics of the table group.
"I understood it, I've seen it before and so I tried to check their understanding. They didn't really understand it, so then I had to do it like [facilitator] did. . .I went through it and kind of explained that, but then we kind of had a bit of a debate about what that actually meant because people perceived it as meaning different things" (Cancer network representative) Following the scoring exercises, we observed a final prioritization exercise in the form of a 'panel game', where three stakeholdersa patient, a commissioner and a clinical representativewere selected by the facilitator and asked to 'weight' the options that the wider stakeholder group had ranked as high priority. The facilitator asked the panellists to independently decide which options should be 'tackled first' and then justify and explain their view to see whether they could 'convince' the other panel members and the wider stakeholder group. One clinical representative from the wider stakeholder group commented on how it seemed 'silly to be guessing' at this stage, when the stakeholders had done all the scoring work during the workshops.
Although the wider stakeholder group was able to ask questions to the panellists during the 'game', they appeared to disengage from the process and merely observe the interactions between the panellists and the facilitators. Subsequent interviews suggested that stakeholders felt alienated by the experience, reduced to the role of audience member rather than participant. Others felt that this exercise was not given sufficient priority and were disappointed that the 'game' had not involved the whole room. By involving only three stakeholders, there was an impression that this had been detrimental to the ethos of both the methodology and also was against the 'spirit of the day'. This was also something that the project team reflected on: "I found it really frustrating because I don't think it held true to the methodology and I think it risked engagement" (Data analyst) "I thought it was a bit of a waste of time from my point of view because I didn't understand fully what was the point of it. . .you've got three people up there. You could pick three more people who'd give different options" (Patient representative) According to STAR guidance, this activity should be organized in a final 'prioritization' workshop, involving a more detailed and deliberative process with wider stakeholders. In this case study, however, it was added to the end of the second workshop to reduce the need for a further workshop and to more quickly conclude the decision-making process. As such, it appeared to be a tokenistic fulfilment of the prescribed processes, with limited purpose or contribution. A significant observation from the second workshop was stakeholder attrition, especially patients, carers and commissioners. Conversations with the remaining participants suggested that some stakeholders had decided to end their involvement because they saw the processes as endorsing predetermined decisions, whilst others again reflected on the difficulties in understanding the technical data and getting their voice heard. The attrition of patients and carers further amplified the input of clinical stakeholders, which aligned more closely with the technical data, and further marginalized the more experiential input of the remaining patients and carers. At a general level, some stakeholders openly questioned whether the appropriate people were involved throughout the process: "my overall impression of the whole thing, was a comment from a stakeholder who said "Listen, we're having these important discussions, but you've got the wrong people in the room.". . .what we do seems absolutely sensible to us, but clearly what other people do seems sensible to them and there are massive variations, but they weren't the right people to discuss those. It seemed like a really fairly surreal process." (Nurse) Interestingly, this interviewee went on to suggest that clinicians, in particular oncologists, were not sufficiently represented in the decision-making process. Other interviewees, including the project manager, felt that patients and carers might be involved in the process in different ways: 'I have been pondering the idea that patients and carers could attend a separate workshop and then we could bring all the views back together behind the scenes' (Project manager) In summary, the second workshop saw more deliberation, in part because of the additional table facilitators, but also a continued emphasis on the scoring process. Furthermore, the prioritization exercise appeared to alienate many of the stakeholders, whilst stakeholder attrition was a significant issue that adversely affected the overall process in terms of representation in the decision-making process, as patients and carers were marginalized over time, due to a lack of understanding from the outset.

Discussion
Our study investigated the implementation and practical experiences of a DPS project. Our aim was to understand how facilitation influenced the deliberative experience, focusing on the management of technical and social knowledge. In broad terms, our study reports a troubled case, with significant facilitation challenges, which we suggest led to variable stakeholder engagement and deliberation. These challenges are, to some extent, symptomatic of pluralistic decision-making, where stakeholders bring different resources (Purdy 2012) and perspectives (Ansell and Gash 2008). However, in line with the work of Campbell (2010), we suggest that where methodologies are designed to combine technical (objective) and social (subjective) forms of knowledge, there are particular facilitative challenges. While the challenges associated with facilitation are widely acknowledged in the literature, we focus on a number of interlinked analytical themes (see Figure 2) to highlight specific issues pertaining to knowledge management, before concluding with some tentative recommendations for the design and implementation of similar methodologies.
An overarching theme from our study highlights how the epistemological differences, inherent in a DPS approach can complicate decision-making. In the broadest sense, the technical and the social reflect distinct ways of 'knowing'. In our case, technical data in the form of costs and outcomes represented a more positivistic and scientific way of measuring and understanding the cancer care pathway and was  typically regarded by both the analysts and clinicians as more objective and rigorous. In contrast, the social insights of stakeholders represented a more subjective and experiential viewpoint, and despite offering unique views, these were often perceived by analysts and some clinicians as less valuable and difficult to reconcile with technical data. Although DPS explicitly seeks to combine these different forms of knowledge to promote holistic and legitimate decisions (Airoldi et al. , 2014, it emulates a methodological hurdle that has characterized social science research for at least a century (Campbell 2010). Namely, there were areas where the more objective and subjective ways of knowing could not be reconciled and where one form dominated the other, specifically where so-called deliberative activities were shaped by the generation of quantitative scores.
We believe that these epistemological differences both reflect and recreate underlying power and status inequalities that were evident throughout the process. Our findings also suggest that these differences were compounded by the poor translation of technical information. Whilst the wider literature recognizes that the heterogeneity of stakeholders challenges the power dynamics within a decision-making process, we found that knowledge and experience of the stakeholders directly impacted the deliberative experience (Purdy 2012) and some stakeholders were disadvantaged by their inability to understand and work with the data.
Focusing on the management of the technical information, and contrary to the methodological guidance, the observed analysts and facilitators failed to redress the power-resource-knowledge imbalance (Ansell and Gash 2008) and presented the data to stakeholders in a complex way. Whilst it is widely acknowledged that effective facilitation is premised on translating technical information so that it can be used by all stakeholders (Purdy 2012;Reagan-Cirincione 1994), we found little evidence of this and, importantly, there was very little opportunity for stakeholders to question and clarify their understanding of the technical data. This is in stark contrast with the wider literature that suggests stakeholders should be afforded the opportunity to question technical data, and use their own preferences and experiences to reach legitimate decisions 2014;Bruni, Laupacis, and Martin 2008;Campbell 2010). In this regard, the technical data was observed to sit alongside and dominate the deliberative experience, rather than informing the process. Furthermore, we observed that the value placed on the technical data and the dominance of the analysts throughout the process, combined with a lack of stakeholder understanding, ultimately 'crowded-out' stakeholder voice and 'the empowerment condition' was not achieved (Gibson, Martin, and Singer 2005).
A fundamental precept of deliberation is that stakeholders should feel empowered to work together through open, discursive exchange (Doberstein 2016;Nabatchi 2010;Parkinson 2003). As with other studies, we found that the asymmetries of knowledge together with power and status inequalities created limited deliberation, with deliberative activities dominated by the voices of professionals (Purdy 2012). The clinicians, in particular, found the technical data more accessible and had greater insights into the entire cancer pathway (Berner, Amos, and Morse 2011;Callaghan and Wistow 2006) compared with the more subjective lived experiences of cancer survivors. However, the power and status differences between stakeholders could have been reduced if the facilitators had worked to ensure all stakeholders understood the technical data and by managing the power and status inequalities more effectively. As such, the potential for deliberation was also limited by the approach to facilitation, with 'scoring' and 'ranking' exercises dominating the workshops, and therefore privileging the more technically proficient professional stakeholders.
Whilst the scoring and ranking exercises were intended to bridge the epistemological divide between the technical data and stakeholder perspectives, our study found little evidence that this occurred. Rather, because of the poor management of the technical data and the continued power and status inequalities between stakeholders and the ensuing limited deliberation, we ultimately observed a de-coupling of the technical and the social forms of knowledge. Reflecting on our reading of the literature, overall, the implementation and facilitation of the DPS approach lacked 'circular flow' (Campbell 2010). To achieve circular flow, a decision-making process should ensure that there is interaction between the information provided and stakeholder understanding of this information (Campbell 2010;Emerson, Nabatchi, and Balogh 2012). Furthermore, stakeholders must be able to check or clarify their understanding, to ensure that data inform stakeholder engagement, and ultimately influence the selection of priorities (Campbell 2010). The design and facilitation of our case study showed limited evidence of this. The requisite 'circular flow' (Campbell 2010) was also broken by a decoupling of the methodological principles of discovery, definition, deliberation and determination (Emerson, Nabatchi, and Balogh 2012), with very little iteration between these stages.
The facilitation of deliberative processes, especially in the management of different forms of knowledge, is a central concern for all deliberative decision-making processes. Effective facilitation contributes to, and ensures integration between, the three prominent aspects of deliberative decision-making, namely the design and implementation of deliberative processes, stakeholder engagement and the deliberative experience (Bryson, Crosby, and Stone 2006;Fung 2006;Mansbridge and Parkinson 2012;Parkinson, 2004). Specifically, we propose that attention to effective facilitation, including the management of knowledge in the design and implementation of decision-making processes, will foster stakeholder engagement and enhance the deliberative experience.
Our 'troubled' case study shows, however, the challenges of effective facilitation, which were rooted in the implementation of the decision-making process, resulted in reduced stakeholder engagement and undermined the deliberative experience. In particular, poor knowledge management negatively impacted the deliberative experience as deliberations were detached from, but dominated by, the technical data. In practical terms, the facilitators failed to ensure accessibility of the data and to effectively integrate the technical and social principles during workshop activities. As a result, they failed to redress the asymmetries of power amongst stakeholders. This may reflect the expertise of the facilitators and analysts and their familiarity with DPS approaches, which led them to overestimate the ability of stakeholders to work with the data.

Concluding comments: practical steps for the facilitation of DPS methodologies
In line with the principles of collaborative governance, DPS methodologies are becoming increasingly popular as policymakers try to balance technical costs and outcome data with stakeholder perspectives (Campbell 2010). The facilitation of effective deliberation is therefore a key concern for policymakers (Doberstein 2016;Mabin et al. 2001). While it is broadly acknowledged that 'good' facilitation empowers stakeholders, manages power imbalances and prevents the dominance of professional voices, there is limited understanding of what constitutes good facilitation in practical terms.
Through our analysis, we suggest that the management and reconciling of different types of knowledge is key to bridging the epistemological differences between the technical and the social, improving the translation of technical information, managing the power and status inequalities and thus enhancing the deliberative experience. A significant limitation of our study is that we did not consider how the decision-making process ultimately influenced service change, making it difficult to comment on the overall effectiveness of the methodology. However, our paper 'zooms in' on an aspect of the process that has received little empirical attention, namely the relationship between facilitation and the deliberative experience.
To manage knowledge effectively, we would suggest that facilitators consider the following practical steps when designing and implementing a DPS process. First, we would suggest that the project manager or an independent facilitator undertakes the role of translating and disseminating data, rather than the data being presented by the analysts themselves, who may take for granted the accessibility of the data. Linked to this, we suggest that recruiting facilitators with non-technical expertise might be beneficial as they would first need to work through the process of understanding the data themselves, which might improve their ability to more effectively translate the technical information for all stakeholder groups.
Second, data should be presented to stakeholders in the simplest form possible, using visual aids and layman's terms where possible (as per the STAR guidance) to ensure that it can be used in deliberation. Here, we would advise against the distribution of 'data packs' prior to workshops as this appears to overwhelm stakeholders. Stakeholders should be introduced to the data in a workshop where they have the opportunity to ask questions and clarify their understanding of the technical information and where facilitators can in turn check stakeholder understanding before moving on to the scoring and deliberation exercises. In facilitating this type of workshop, the power and knowledge differences must be anticipated by facilitators. Linked to this, we would suggest that it would be beneficial to brief stakeholders on the need to listen to each other and to acknowledge the potential power differential. Furthermore, we suggest that patients and carers could be briefed about the importance of their voice in promoting legitimate decisions, to encourage them to speak up.
Third, facilitators need to uphold the 'social' deliberative elements of DPS. To achieve this, we suggest that stakeholders are brought into the decision-making process at an earlier stage to enable them to shape the priority areas and the associated decision criteria. Moreover, we would recommend spending time explaining the approach in detail and providing some guidance about what open deliberation might entail. Linked to this, it is important to ensure that scoring activities do not overshadow the opportunity for stakeholders to openly deliberate and express their perspective. Taken together, we suggest that these recommendations may help address the challenges associated with knowledge management and preclude the problem of stakeholder attrition witnessed in our case. However, one final point for consideration is whether DPS is appropriate for the design or reconfiguration of complex services or pathways, involving multiple actors, decision-points and options, and where the target clients are highly variable. As such, DPS might be more suitable for more discrete, linear or simple choice domains.