Scoping the international impact from four independent national dermatology trials

Research impact describes whether and how research results in wider benefits to society beyond academic publication. Little is known about translation of clinical trial research into dermatological practice.


Introduction
Research impact is a topic that has grown substantially over the past 10 years. Traditionally, research impact was measured by whether an article was published in a prestigious journal with a high 'impact factor', or how many times it was cited using citation reports or altmetrics in a given timeframe. 1 The value of medical research is limited if it is cited only by other researchers in academic reports and not put into any form of action for patient or wider societal benefit. Even if research results are eventually translated into action, the time delay between publishing and implementing clinical findings has been estimated to be about 17 years, 2 which is clearly unacceptable and wasteful.
Greenhalgh et al. 3 suggested that impact is achieved when research results confer benefits to wider society in terms of health gain, economy or culture. The measurement of research impact is an emerging field, and requires a range of quantitative and qualitative methods such as payback frameworks, research impact frameworks such as the Canadian Academy of Health Sciences Framework, monetization models and societal impact assessment. In the UK, research councils determine research excellence of universities by undertaking periodic assessments of research quality and impact, which determine how much central funding universities receive. Whereas prestigious publications and infrastructure contributed heavily to determining excellence in older assessments, the 2014 research impact exercise introduced the concept of allowing universities to describe the wider impact of their research outputs. 4,5 During an exercise to evaluate a possible future impact case study on independent clinical trials led by our Centre of Evidence-Based Dermatology (CEBD), it was suggested that the authors should conduct a brief international survey of colleagues to evaluate whether the results of those studies were useful or not, and if so in what way. In this paper, we report the findings of this scoping survey.

Methods
We undertook a scoping survey, the responses of which were scrutinized for common themes and summarized narratively. We selected four of our national clinical trials 6-9 funded independently by Government (National Institute of Health Research) and charity sources (Action Medical Research and Cancer Research UK) that we anticipated would have significant international reach and relevance. The four trials are shown in Table 1, and dealt with interventions for basal cell carcinoma, cellulitis, bullous pemphigoid and pyoderma gangrenosum (PG). Three of the trials were published in general medical journals and one in a dermatology journal. [6][7][8][9] Three of the trials (PATCH, BLISTER and STOPGAP) were supported and delivered through the UK Dermatology Clinical Trials Network (UK DCTN), an independent UK charity dedicated to prioritizing and delivering national independent clinical trials in dermatology. 10 All four trials were open access.
We sought to achieve the highest response possible by including the request for feedback on the four trials in the body of a single, personalized email, the exact wording of which is provided online in Supporting Information. The email was kept short and deliberately open in order to avoid responder bias and to elicit themes that might not otherwise have emerged from more targeted questions. The wording was nondirective and included any form of change such as using a given treatment more, using a given treatment less or no change. One author (HCW) emailed international colleagues from 22 countries spanning 5 continents. For some countries, we wrote to > 1 colleague if we knew they specialized in a particular area (e.g. skin cancer) so as to cover all areas of medical dermatology included in our four exemplar trials. We included responses received from recipients as well as additional responses that they solicited from departmental colleagues or from elsewhere in their country. We did not seek to ascertain collective views through formal routes such as the national dermatological associations.
All responses were tabulated by one author (NKR), while themes were identified by another (HCW) and corroborated or added to by others (NKR, JRC and KST), and described narratively. No formal quantitative summaries are provided as they are not appropriate in the context of identifying key themes. In order

Results
In total, 33 email requests were sent to international colleagues from 22 countries across 5 continents, and a response rate of 100% was achieved. The full responses in relation to each trial by country are presented online in Supporting Information (Table S1).
The following themes were identified.

Assessing clinical trial impact in general
There was universal interest in the notion of tracking clinical trial impact to ensure clinical benefit. The importance of considering translating articles into other languages such as Chinese was also highlighted, as was the importance of searching beyond national guidelines or other key evidence sources such as UpToDate, and patient information sources such as DermNetNZ.

Treatment implications from the studies
The survey demonstrated increased use of study medications that had shown benefit in the trials.
This study has changed our practice in Colombia as it has become the first option before starting systemic corticosteroids for all patients (not only for some) in the majority of in-patient dermatology centers.
(BLISTER Trial impact, Columbia) In addition, treatments that had been previously used somewhat tentatively were now being used with more confidence as a result of the more robust evidence. There was also a lack of awareness, prior to the STOP GAP Trial, that ciclosporin might be a useful treatment for PG. I refer to this study a lot because ciclosporin is often forgotten as a good alternative for corticosteroids.
(STOP GAP Trial impact, The Netherlands) Use of the trial evidence to produce more robust estimates of the treatment response that can be incorporated into shared decision making with patients during consultations was also highlighted.
We cite your study in our decision tool for patients with BCC [basal cell carcinoma] and your data directly helps patients make more informed decisions.
(SINS Trial impact, USA) Trial evidence was also used in a modified way. For example, the BLISTER study intended to show that a strategy of starting treatment with doxycycline might produce benefit in all severities of pemphigoid, but some respondents felt that the data encouraged them to use doxycycline either as a steroid-sparing treatment after treatment with prednisolone had been started or just for milder disease only. I prescribe doxycycline frequently for mild disease and have used it as a steroid-sparing agent.
(BLISTER Trial impact, New Zealand) The potential impact of trials was dependent on the external validity of the studies relevant to specific countries Although these were pragmatic trials designed to reflect clinical practice in the UK, several aspects had an impact on the applicability of the results to other countries. For example, in many of the surveyed countries, cases of cellulitis were not dealt with by dermatologists, so it is unclear whether the trial findings were taken up by other clinicians using different care pathways in those countries.
There also appeared to be variations in the prevalence of dermatological conditions in some countries; for instance, conditions such as cellulitis are rarely seen in some countries such as Iran, Japan and Israel, for reasons that are unclear.
Variations also existed in whether study medications were available in some countries, e.g. oral penicillin was not available in Taiwan and there was a tendency to use more cephalosporins in Korea because of high rates of penicillin allergy.
Drug availability aside, there were variations in whether study medications were permitted to be prescribed for the target conditions in some countries, e.g. ciclosporin was not permitted for treating PG in several countries because of its high cost. In some countries, despite a treatment being available, it was not covered by health insurance, thus limiting the applicability of the results.

Other educational implications
Beyond the specific trial results, impacts on education and sharing of best practice were highlighted, such as papers being discussed and critically appraised in departmental journal clubs and online forums, serving an important educational purpose. I have also seen this paper discussed in online communities and journal clubs several times in Spain and internationally, and it has been the main guide for my therapeutic decisions since its publication.
(STOP GAP Trial impact, Spain) Our trials were also useful beyond the results, in that they served as exemplars of good trial design, registration and complete reporting. The use of videos on trial websites was also highlighted as an effective means of communicating complex high-level information in a short time and as a teaching aid. 11 This study was very impactful. I can clearly recall when it was presented in our journal clubs, including your breakthrough role in the video presenting the study! (BLISTER Trial impact, Israel) The mere act of surveying colleagues was also a stimulus to read the original research that they might have missed. This was particularly the case for the PG study that was published in the British Medical Journal.

Discussion
This brief international scoping survey has revealed new insights into the way that clinical trial research may or may not have an impact on dermatological clinical practice around the world. Our scoping survey exceeded our expectations in terms of revealing the diversity of impacts of our four independently funded clinical trials. In addition to simply documenting whether trial results had directly changed practice, a host of indirect effects such as using already available drugs with more confidence, using the trial results to improve shared decision-making and providing educational benefits such as journal club discussions were identified. Respondents clearly welcomed the concept of reflecting on how evidence had affected their clinical practice. The time course of the translation into clinical practice of the four clinical trials (published in 2013, 2015 and two in 2017) is much faster than the average of 17 years previously suggested, 12 but this could be because the trials were published in high impact journals, the study drugs were already available in most countries, and those surveyed may have been early evidence adopters. The open-access nature of our four trial publications may have also contributed to the reach of the studies. Open access publication is now being encouraged internationally by the Plan S coalition, but for authors not able to pay for open access, attention to partners/influencers, being clear about your message, and considering a range of channels including videos and social media may help to enhance dissemination. 11,13,14 The survey provides some assurance that the studies conducted by the Centre of Evidence Based Dermatology and the UK DCTN are studies that have wide international appeal and clinical significance. This may be because the trials all addressed key questions that had been identified in previous systematic reviews [15][16][17][18] and prioritized by clinicians and patients through the UK DCTN. 19 Two of the UK DCTN trials in this survey [penicillin for recurrent cellulitis (PATCH) and doxycycline for pemphigoid (BLISTER)] dealt with inexpensive medicines that had been used for many years by some clinicians, which were thought to be beneficial but had never been tested properly. The SINS study was the first independent comparison of topical imiquimod against the reference standard of excisional surgery, which provided stable estimates of treatment responsean active comparison study that industry would not usually tackle.
Study strengths include the wide global reach across 22 countries in 5 continents and the 100% response rate. The completeness and frankness of responses to all four trials was also appreciated. We did not publish a study protocol prospectively as we did not set up the survey as a formal research study; however, the results were so interesting that we felt it appropriate to share them more widely for educational purposes. We have also not attempted to report all of the items as suggested by standards for reporting qualitative research 20 because most were not appropriate for a simple survey. However, we did observe basic principles of analysing qualitative comments through thematic analysis that was corroborated by others. We also reported all of the responses verbatim, both positive and negative, so that readers can come to their own conclusion on what respondents actually said and meant. It might be argued that because all of the respondents were colleagues of one of the authors (HCW) they would be more likely to provide favourable responses. Against this notion, it should be noted that the original email gave permission for respondents to indicate if there was 'nothing much to report ª 2020 British Association of Dermatologists here' in terms of impact. Indeed, some responses did report that the trials had 'no or little' influence, also suggesting that the answers were frank and honest.
Pressures on researchers by universities to constantly generate research income and to 'publish or perish' 21 are likely to confer a type of researcher behaviour that assumes that the job is done once a funded study is published, and to move on to the next study instead of engaging with clinicians and service providers, policymakers and patients to implement the results. The late Alessandro Liberati 22 called this 'the butterfly behaviour of researchers, moving onto the next flower well before the previous one has been fully exploited'. Citations in guidelines are an easy way of tracking impact, but counts do not indicate how the study has been cited, and provides no evidence of direct patient benefit. The science of impact assessment and implementation science 12,23 in dermatology is very much in its infancy. We hope this report will stimulate others to invest more in translating research findings into practice, and to track the impact of research to ensure that patients ultimately benefit from it.