Core outcome sets in dermatology: report from the second meeting of the International Cochrane Skin Group Core Outcome Set Initiative

Results of clinical trials are the most important information source for generating external clinical evidence. The use of different outcomes across trials, which investigate similar interventions for similar patient groups, significantly limits the interpretation, comparability and clinical application of trial results. Core outcome sets (COSs) aim to overcome this limitation. A COS is an agreed standardized collection of outcomes that should be measured and reported in all clinical trials for a specific clinical condition. The Core Outcome Set Initiative within the Cochrane Skin Group (CSG‐COUSIN) supports the development of core outcomes in dermatology.

Results of clinical trials are the most important information source for generating external clinical evidence. The use of different outcomes across trials, which investigate similar interventions for similar patient groups, significantly limits the interpretation, comparability and clinical application of trial results. Core outcome sets (COSs) aim to overcome this limitation. A COS is an agreed standardized collection of outcomes that should be measured and reported in all clinical trials for a specific clinical condition. The Core Outcome Set Initiative within the Cochrane Skin Group (CSG-COUSIN) supports the development of core outcomes in dermatology. In the second CSG-COUSIN meeting held in 2017, 11 COS development groups working on skin diseases presented their current work. The presentations and discussions identified the following overarching methodological challenges for COS development in dermatology: it is not always easy to define the disease focus of a COS; the optimal method for outcome domain identification and level of detail needed to specify such domains is challenging to many; decision rules within Delphi surveys need to be improved; appropriate ways of patient involvement are not always clear. In addition, there appear to be outcome domains that may be relevant as potential core outcome domains for the majority of skin diseases. The close collaboration between methodologists in the Core Outcome Set Initiative and the international Cochrane Skin Group has major advantages for trialists, systematic reviewers and COS developers.
• The Core Outcome Set Initiative within the Cochrane Skin Group (CSG-COUSIN) was established to support the development of COSs in dermatology.
What does this study add?
• The second meeting of CSG-COUSIN took place in 2017 and included updates from 11 core outcome projects.
• Research gaps identified included the need to develop more guidance on how to appropriately define the focus of a COS, how to identify the core domains, how to best involve patients and which are the most useful decision rules within Delphi surveys when developing COSs.
• The meeting concluded that some common outcome domains may be applicable to dermatological diseases in general.

Background
Results of clinical trials are the most important information source for generating external clinical evidence for evidencebased medicine and care. 1 Threats to internal and external validity of clinical trials are well known and these limitations must be adequately taken into account when interpreting and summarizing trial results. 2,3 The increase in numbers of published clinical trials has revealed a further challenge that has received increasing attention during the past decades: the multitude and poor comparability of outcomes that are used and reported. 4 Noncomparable outcomes across trials investigating similar interventions for similar patient groups cause a number of problems for the interpretation and clinical application of trial results. The use of different outcomes across trials makes it impossible to compare treatment effects between studies. Even if the same outcome domain is captured in different trials (e.g. pain, clinical signs of disease severity), there are still diverse ways to measure this phenomenon or construct.
This problem occurs in all fields of health and medical care but also in dermatology: at least 20 different named outcome measurement instruments have been published to measure atopic dermatitis; 5 11 outcome measurement instruments have been identified for measuring aspects of vitiligo; 6 46 instruments for measuring repigmentation alone in vitiligo; 7 53 for measuring the clinical severity of psoriasis; 8 and 30 for measuring hidradenitis suppurativa in clinical trials. 9 More than 111 clinical scales are available for measuring skin ageing. 10 Different instruments measuring the same construct produce different numerical expressions that cannot be pooled in meta-analyses.
In addition, outcome measurement instruments themselves need to meet quality criteria including validity, reliability, responsiveness 11 and relevance to the target population. 12 Results of systematic reviews indicate that the reliability and validity of the majority of applied instruments in dermatology are not supported by adequate evidence. 5,6,[8][9][10] The choice of the best and most relevant outcomes is not only a challenge for trialists, but also for systematic reviewers. Systematic reviews should include all outcomes that are meaningful and relevant to clinicians, patients, the general public, administrators and policy makers.
In the 64 Cochrane Skin Group (CSG) reviews published up to January 2015, 402 outcomes were predefined by the review authors. Of these, 33% were not addressed in any individual trial. 13 The number of outcomes reported in the individual trials but not included by the systematic reviewers is unknown but probably much higher. This indicates that there seems to be a significant mismatch between outcomes considered important by Cochrane review authors (that include patients) and outcomes measured and reported in trials.

Core outcome sets
One solution to overcome these difficulties is standardization of outcomes and outcome measurements. A core outcome set (COS) is an agreed standardized set of outcomes that should be measured and reported, as a minimum, in all clinical trials in a specific disease or trial population. They consist of outcome domains and corresponding measurement instruments. 11 Domains are broader aspects or concepts of a disease indicating 'what' to measure (e.g. disease severity, pain). Measurement instruments are needed to measure the particular domain and indicate 'how' to measure (e.g. scales, classifications). 11,14 The Outcome Measures in Rheumatoid Arthritis Clinical Trials (OMERACT) initiative was the first group to systematically develop and promote core outcomes use.
Today, there is growing interest in COS development with a corresponding increase in the volume of methodological research and guidance. 11,[15][16][17][18] The Core Outcome Measures in Effectiveness Trials (COMET) initiative provides a platform for scientific exchange and supports methodological research in this area. COMET also hosts a database covering planned and published COS projects. 19 Recently, the Core Outcome Set-Standards for Reporting (COS-STAR) Statement was published. 17 However, compared with clinical trial methodology, the science and practice of COS development is still under developed and the field is continuing to tackle a number of fundamental methodological questions and uncertainties. The purpose of this meeting report is to summarize these challenges in relation to COSs in dermatology in order to identify and prioritize possible directions for future research and development.

Cochrane Skin Group -Core Outcome Set Initiative
The Harmonizing Outcome Measures for Eczema (HOME) initiative set out in 2008 to develop a COS for atopic dermatitis trials and was the first COS initiative in dermatology. 20,21 The HOME initiative developed the HOME roadmap 14 to be used as a methodological framework for COS development. In addition to other existing guidance, 17,22 the HOME roadmap is being used by many COS initiatives in dermatology. 9,23,24 Because outcome selection is so fundamental for clinical trials and systematic reviews supported by the Cochrane Collaboration, two of the report authors (H.C.W. and J.S.) established the Core Outcome Set Initiative (COUSIN) within the CSG in 2014. The CSG-COUSIN is an international multidisciplinary group that strives to support the development and to strengthen the quality of COS development in dermatology.
The inaugural meeting took place in 2015 in Dresden (Germany) within the CSG annual meeting. 13 CSG-COUSIN consists of a management team based in Dresden (Germany), a methods group and a number of disease-specific COS project groups. The management team coordinates CSG-COUSIN and provides organizational and technical support for the methods and project groups. Since the first meeting a homepage was launched, 25 a meeting report published, 13 newsletters prepared and visibility and awareness created (e.g. poster, flyer or presentations at dermatology conferences).
The methods group provides methodological support and internal peer review for CSG-COUSIN project groups, conducts methodological studies on outcomes research and COS development and sets up quality standards for COS development and implementation processes. The COUSIN group has also developed a practical guidance document on how to develop COSs based on the HOME roadmap. 26 CSG-COUSIN project groups work on the development and implementation of specific COSs in dermatology. Since inception, 14 COS development projects are now working with CSG-COUSIN. 25 In January 2017, the second CSG-COUSIN meeting took place within the 2-day CSG annual meeting at the Department of Dermatology and Allergy at the Charit e-Universit€ atsmedizin, Berlin (Germany) hosted by two of the report authors (J.K. and A.N.). The entire first day was dedicated to CSG-COUSIN topics, the second day covered methodological topics of the CSG in general.

Aims of the meeting
The primary objective was the presentation and discussion of the current status of COS development in the different COS project groups in order to share learning of how to overcome common logistical and methodological hurdles. Groups were requested to present their current work and achievements but most importantlyto identify challenges and problems.
Additionally, the meeting aimed to present and discuss current standards of COS reporting and quality criteria/quality assurance related to COS development so that the work of the group could be aligned to the latest relevant research in the field. Furthermore, the meeting aimed to strengthen the cooperation between clinical researchers, trialists, methodologists, COS developers and systematic reviewers involved in Cochrane reviews. Based on the identified problems and opportunities, a work plan for the next year was to be developed.

Meeting participants
Dermatologists, methodologists, systematic reviewers and researchers with an interest in evidence-based dermatology and COS development attended. The majority were from the CSG. Patient representatives were present and participated in the discussions. However, there was no special form of patient involvement at this first day of the CSG annual meeting. Patients always participated actively in the individual COS development groups.

Meeting content
After an introduction, a keynote lecture by Jamie Kirkham from the COMET group, and critical reflection on the development of CSG-COUSIN, 11 individual COS groups presented their current work status (Table 1). Each presentation included a summary of what has been done so far, preliminary results and challenges. The identified challenges were discussed extensively with the whole group. The discussion was led by a moderator and emerging issues documented on flip chart papers visible for all. During this process, overarching methodological challenges that were relevant for COS development in general were identified, and these are summarized below.

Health problem and population
The definitions of the health problem, target populations, healthcare setting and likely interventions are crucial first steps in COS domain development, 14,17 yet it is not always clear how this should be done. For instance, it was discussed whether separate COSs should be developed for children and adults and for induction and maintenance treatments for people with chronic skin diseases. Do COS domains for melanoma stage 1 differ from other melanoma stages? Is a COS for nail psoriasis justified or is it just a subset of patients with psoriasis in general? Do different types of interventions (e.g. repositioning vs. special support surface for pressure-ulcer prevention) in pressure-ulcer prevention address different outcomes, requiring intervention-specific COSs? On the other hand, it was argued that for some skin conditions, different interventions usually have the same aim thus justifying identical domains. The overall question of when to split a skin disease or treatment into subgroups or when to treat this as one entity was discussed.

Domain identification
In the early stages of a COS development project, all possible disease domains must be identified first. 14,26 In addition to qualitative approaches, systematic literature searches are another way for identifying domains. It was discussed whether the consideration of published clinical trials is sufficient for domain identification. When choosing core outcomes from existing clinical trials, other important domains may be missed. In accordance with current methodological guidance, 16 it was agreed that looking at published clinical trials is necessary but not sufficient. A discussion arose that other publication types (e.g. qualitative studies) need to be considered. It was agreed that the domain identification should not only be influenced by the assumed or known existence of measurement instruments.
A major challenge for nearly all groups was how to extract and/or to define domains based on the literature. Methodological guidance on how to develop COS domains is available 11,17,18,22 but an unsolved problem is deciding how broad or narrow a domain should be. Are all clinical signs of a cutaneous disease considered together as a domain or is each sign (e.g. erythema, scaling, inflammatory lesions) a domain? When in the process should what be summarized by whom? Is 'skin ageing' a true domain? Moreover, it is unclear how many domains should be included in the subsequent Delphi study and how many outcome domains should be included in a COS. Slightly different definitions of 'domains' and 'outcomes' in existing methodological frameworks further contribute to uncertainty.
Conceptual difficulties regarding domain definition and identification also exist in established methodological frameworks 27 and they may be context-or discipline-specific. Discussions showed that the level of abstraction of core domains in dermatology is not clear. In addition, there appear to be outcome domains that may be applicable for the majority of skin diseases. Furthermore, guidance is needed for the timing of outcome assessment of these domains.

Instruments
Heterogeneity generally concerns variation between different outcome measurements used. However, in some diseases (e.g. nail psoriasis) wide variation within outcome measurements with the use of many different versions of the same outcome measurement has been detected. These different versions should be mentioned as separate instruments in the process towards COS development. The methodological appraisal and selection of measurement instruments in general was regarded as challenging. One main reason seems to be that widely used instruments often do not meet criteria for good measurement properties. 11 Development of new instruments is a major, time and resource intensive task that is not easy to accomplish. This led to the unanswered question of what to do with domains for which appropriate instruments are missing.

Decision rules during the Delphi rounds and disagreement between stakeholders
The Delphi technique is considered as the current methodological standard for outcome domain and outcome measurement instrument selection and prioritization prior to further face-to-face consensus work. Although methodological guidance is available, 22,28 the predefined consensus criteria and scoring system rules were discussed in more detail. Currently, five-, seven-and nine-item scales ranging from one (not essential/important) to nine (absolutely essential/ important) are widely used to measure agreement between Delphi study participants in COS projects. Decision rules are often based on cut-offs (e.g. a certain proportion of responses between seven and nine). 21,29 This format is based on the RAND/UCLA appropriateness method 30 and also proposed by GRADE. 31 However, although the RAND/UCLA method proposes a number of stricter and relaxed rules for determining agreement and disagreement, these are not applied in current COS initiatives. RAND/UCLA proposed agreement and disagreement rules were designed for nine-member panels only, 30 whereas in COS Delphi groups, the number of participants is much higher. Therefore, using strict thresholds to decide whether COS domains are kept or left out is arbitrary. 32 This procedure also questions the usefulness of the full-information content that is obtained from the entire one to nine scale. It is always recommended to use the full range of information from rating scales otherwise they are not needed in that specific format. 33,34 Consequently, the Core Outcome Set for Congenital Vascular Malformations (OVAMA) group used dichotomous questions in a consensus meeting after completing three Delphi rounds by simply asking participants whether they think each domain should be included or not. This approach was considered as a possible alternative to the current standard of the nine-item scale method used by the majority of COS developers. However, a dichotomous approach may also be associated with loss of information that might be valuable for discussion should the consensus process involve a consensus meeting.
Closely related to decision rules, the question of how to deal with disagreements between stakeholder groups, especially physicians vs. patients, was discussed. Examples were presented where there was complete disagreement between both groups and possible solutions were explored. The vitiligo outcomes initiative has encountered a difficulty in achieving consensus among stakeholder groups on how best to measure repigmentationone of the essential outcomes. One idea was that disagreements might be solved in a structured face-to-face discussion. The way in which results are presented are likely to influence subsequent decision making. The question arose of whether patients should have a veto on choosing a particular domain if the patient perspective is considered to be the most important.

Patient involvement
Involving patient representatives during the COS development process was regarded as important. Patient and carer involvement is crucial for domain identification for example. Guidance on how to involve patients in research in general 35 is available and how to involve patients and service users using qualitative COS development methods is emerging 16 but there was uncertainty on how best to ensure meaningful patient involvement. Possible options include using existing patient groups (e.g. COMET's People and Patient Participation Involvement and Engagement working group). Premeetings and patient training sessions before participation in meetings and Delphi studies were recommended.
Patients may find it especially difficult to understand the concepts within eDelphi studies. Face-to-face meetings with patients were considered to be empowering for patients. Online meetings using advanced webinar software may enable easier participation without the costs and burden of travelling. A general concern was whether involved patients are sufficiently representative for a whole patient group. Patients are usually highly selected, for example they must be willing to actively participate, they need to speak English, must have the possibility and willingness to travel, or must be familiar with online technology. This leads to a systematic exclusion of particular patient groups. Overall, there is a difference between involving patients in the COS development project as partners or as participants (e.g. in a Delphi study).

Are there common domains within dermatology?
It was clear from the presentations that there were common outcome domains between different skin diseases [e.g. physical signs and symptoms, global severity assessment by a physician/healthcare professional or satisfaction with treatment as was included in Core Outcome Set for Hidradenitis Suppurativa (HISTORIC) and Core Outcome Set for Acne (ACORN) outcome selection]. Recently, OMERACT proposed a conceptual framework of core areas for outcome measurements in intervention studies 36 and the idea was proposed that there may be dermatology-specific outcome domains that are applicable to the majority of COSs of clinical trials in skin diseases. The possibility of creating a long list that covers all possible domains and from which each group could make a selection from was also discussed. This proposal will be further explored and will become the subject of a future CSG-COUSIN project.
Funding COS development work is not generally funded by public funding bodies. A lack of appropriate funding was regarded as one important cause for comparably slow progress in many COS development initiatives. Some delegates mentioned that they had been successful in obtaining funding for PhD students to work on COS studies. Generally, it was believed that industry may also have an intrinsic interest in funding the development of the most relevant COSs but identifying an appropriate funding model that was free of possible bias was unclear.

Conclusions and outlook
COSs are needed to improve evidence-based dermatology and patient care. Therefore, it is important to address this topic appropriately using high-quality methods. Inappropriately developed and published COSs are potentially no better than no COS. 26 CSG-COUSIN exists to support and to promote high-quality COSs in dermatology. All meeting participants and groups expressed their interest and need for continuing interaction and discussions.
The close association between COS development and the CSG has many advantages. Systematic reviewers must consider COSs once they exist. Even if a COS is not available CSG-COU-SIN provides the platform to connect systematic reviewers with COS groups. COS development groups are strongly advised to liaise with Cochrane review authors to ensure their insight into published outcomes and trials can be utilized when developing COSs. In order to implement this, the CSG editorial base and the CSG-COUSIN will develop better links between CSG review authors and COS groups before starting work. Further collaboration exists for instance with other groups interested in developing patient-centred outcomes such as The International Dermatology Outcome Measures (IDEOM) group 37,38 in HISTORIC. 32 During the meeting, more questions than answers were raised. We are also aware that some issues such as the challenges of COS implementation were not addressed. 39 Patient and public involvement could have been stronger and more structured at our meeting, which is something that will be addressed at the next CSG-COUSIN meeting. COS development is a complex and challenging task. Established methodological frameworks exist 11,16,18,36 but some steps and decisions during the process are more subjective than others. Further standardization seems to be one way to establish quality standards. One attendee asked whether COSs are reproducible, that is, whether different groups using identical information would come up with similar domains and instruments. Although it would be extremely challenging to do such comparisons it is not impossible and it might answer the question of how robust current COS development methods are.
One main conclusion of the meeting was that the CSG-COUSIN methods group needs to develop requirements for the development of high-quality COSs in dermatology. Based on existing guidance, 14,17,18 such practical standards will include blueprint protocols, internal peer review and standardized interaction between COS groups and corresponding Cochrane review groups. These processes and documents will be developed by the methods group by the end of 2017 and introduced at the next CSG-COUSIN meeting in Amsterdam in January 2018.
Methodological guidance and standardized procedures throughout the different stages of COS development 14 have been identified as a critical prerequisite for CSG-COUSIN to meet their primary aim of developing high-quality COSs. The other short-term goal for CSG-COUSIN to complete prior to the next meeting in 2018 is to better integrate the development of high-quality reviews within the CSG with COS development through the CSG-COUSIN collaboration.
CSG-COUSIN is not externally funded and relies on the voluntary work of the people involved. CSG-COUSIN is an international group with a clear and exclusive focus on developing COSs in dermatology according to high methodological standards and is firmly embedded with the international CSG that produces high-quality systematic reviews of primary research. We invite interested researchers, clinicians, methodologists, patients, payers, industry and regulators to participate and to contribute to this exciting new initiative in dermatology and we welcome proposals from groups wishing to develop COSs in skin diseases not currently being developed within CSG-COUSIN.