Development of international learning outcomes for shelter 1 medicine in veterinary education: a Delphi approach 2

27 Shelter medicine is a veterinary discipline of growing importance. Formally accepted as a 28 clinical specialty in the USA in 2014, the practice of shelter medicine worldwide is 29 expanding. As a topic in veterinary pre-registration (undergraduate) education, it is frequently 30 used as an opportunity to teach primary care skills, but increasingly recognized as a subject 31 worthy of teaching in its own right. The aim of this study was to use a Delphi consensus 32 methodology to identify learning outcomes relevant to shelter medicine education. 33 Shelter medicine educators worldwide in a variety of settings, including universities, non- 34 governmental organisations and shelters were invited to participate. Participants were initially 35 invited to share shelter medicine teaching materials. These were synthesised and formatted 36 into Learning Outcomes (LOs) based on Bloom’s taxonomy and organised into five subject- 37 specific domains. 38 Participants were then asked to develop and evaluate the identified LOs in two rounds of 39 online surveys. Consensus was determined at >80% of panellists selecting “agree” or 40 “strongly agree” in response to the statement “please indicate whether you would advise that 41 it should be included in a shelter medicine education program” for each LO. In the second 42 survey, where re-wording of accepted LOs was suggested, preference was determined at 43 >50% agreement. 44 Through this method, 102 agreed LOs have been identified and refined. These LOs, as well 45 as those which did not reach consensus, are presented here. These are intended for use by 46 shelter medicine educators worldwide, to enable and encourage the further development of 47 this important veterinary discipline.


51
Shelter medicine is one of the newest disciplines within veterinary medicine, recognized as a 52 specialty by the American Board of Veterinary Practitioners in the USA in 2014 (1). 53 Internationally, this field is widely practiced and of increasing importance. 54 Shelter medicine describes the veterinary care and management of unowned animals in 55 shelters or other rehoming establishments. The practice of shelter medicine is wide-ranging 56 and includes many aspects of physical and behavioural health of animals in shelters, 57 epidemiology, population dynamics, high quality clinical decision-making, resource 58 management and reducing companion animal homelessness (2, 3). 59 Shelter medicine is an integral element of small animal veterinary teaching in many 60 universities, encompassing a variety of underpinning principles of veterinary medicine (4, 5). 61 The inherent value of teaching these topics combines the opportunity for practice and 62 application of many aspects of small animal primary care (6). However, shelter medicine is 63 itself increasingly recognized as independently worthy of study and practice. Student interest 64 and engagement has been a further driver of inclusion of shelter medicine in education (7). 65 Worldwide, pre-registration (undergraduate) teaching in shelter medicine shows considerable 66 differences between veterinary faculties. A lack of consensus regarding key concepts of 67 shelter medicine within veterinary curricula could explain these inconsistencies. However, it 68 is increasingly recognized that more inclusion of shelter medicine within veterinary education 69 is desirable (7-9). 70 The concept of constructive alignment (10) sets out parameters for curriculum development 71 that focusses on linking the teaching methods and content to what the learners need to know 72 and be able to do. Integral to constructive alignment is the development of appropriate 73 learning outcomes that the students should be able to achieve at the end of a period of study. 74 Furthermore, the method of study, student activities, learning materials and assessment 75 should be finely tuned to ensure students achieve their learning outcomes. Bloom's taxonomy 76 of learning outcomes (11, 12) considers student activity and achievement in three domains: 77 cognitive (knowledge), affective (attitudes) and psychomotor (skills). 78 The Delphi technique is a recognized method of harnessing expert opinion to reach consensus 79 where absolute evidence is scant or lacking (13). There is a simultaneous exploration of both 80 similarities and differences of opinion. Contributors give their opinion anonymously, which 81 reduces the bias of other methods of collaboration and allows for parity in consideration of 82 different ideas (14). 83 The Delphi technique uses a focus group to garner expert opinion on a given topic. The 84 method collates, sorts and summarises responses to questions in a survey and re-presents this 85 information to the contributors for further feedback, thus bringing the opinion on the subject 86 matter to a consensus (15, 16). 87 Successful use of the Delphi technique has been established in various aspects of curriculum 88 and professional development in veterinary and other medical fields (17)(18)(19)(20). There is 89 precedent for its use in successful development of learning outcomes in both undergraduate 90 and postgraduate veterinary education; specifically anchoring the outcomes with sound 91 educational principles (21,22). 92 The aim of this study was to identify learning outcomes relevant to shelter medicine 93 education using the Delphi consensus methodology. 94

Materials and Methods 95
Participants 96 survey was distributed via an email invitation and link on the 7 th of January 2020, a reminder 182 email sent on the 24 th of January 2020 and the survey closed on the 31 th of January 2020. 183

Analysis of Delphi Round 3: Survey 2 184
For the accepted but reworded LOs, a greater than 50% majority agreement was used to 185 select which of the two versions to include in the 'Accepted LO' list (26). For LOs with less 186 than 80% from Round 2, and New LOs suggested in Round 2 a greater than 80% agreement 187 placed them in the final 'Accepted LO' list. LOs which did not reach greater than 80% 188 consensus in this round were rejected from the final LO list. 189

Ethical Considerations 190
All participants in the workshop, respondents to email requests for teaching material, and 191 online surveys gave permission to use their responses. This proposal was reviewed by the 192 ethics committee at the University of Nottingham School of Veterinary Medicine and Science 193 proposal # 2100 170815. 194

195
Delphi Round 1: Acquiring Learning Outcomes 196 Forty-eight respondents initially agreed to participate in the study. Panellists were located in 197 Europe, Australia, Asia and North and South America. They described holding a variety of 198 roles, including academic, shelter practitioner and non-governmental organisations. Most 199 were involved in education of veterinary students, qualified veterinarians, or both. Some were 200 involved in education of student veterinary technicians or nurses, or of shelter staff. 201 Of these panellists, ten contributed shelter medicine teaching materials from programs in 202 Europe, Australia and the USA. These materials consisted of a mixture of course aims, course 203 materials and learning outcomes, which were rationalised and sorted as previously described 204 and organised as an online survey consisting of a total of 133 learning outcomes (Table 1). 205 These were divided across five domains: 206 Agreement was considered achieved at >80% of panellists selecting "agree" or "strongly 224 agree" (*Some of the LOs with <80% agreement were consolidated or split for the Delphi 225 Round 3). 226 Delphi Round 3: Survey 2 227 Twenty-four panellists responded to Survey 2. Eight were in the USA, 15 in Europe and one 228 in Australia. Respondents were invited to select words to describe the institutions they 229 worked within. Common descriptors included academic (11), shelter (9), non-governmental 230 organisation (5) and government (1). 231 For the second round, the twelve agreed LOs that had had their wording queried were given 232 to the panel with both original wording and an amended version. Two original learning 233 outcomes were identified as containing duplication, so were offered condensed into one. All 234 the resulting twelve amended versions were accepted as preferred to the originals. 235 The 38 learning outcomes which did not reach consensus in Round 1 were condensed into 34 236 LOs, re-worded in response to comments from the panellists and re-presented. Of these, 4 237 reached 80% consensus and were accepted, while the remaining 30 did not reach agreement. 238 Therefore

Physical Health
Overarching Principles Identify the differences between population and individual animal medicine and the points where they may conflict. Design and analyze preventative health care protocols to minimize the risk of shelteracquired disease and prepare animals for a timely outcome. Assess physical and behavioral health and management of overall population. Design procedures for animal intake including initial assessment, preventative health care, and urgent care. Utilize evidence-based medicine to determine an optimal treatment plans for individual animals including costs, staff training, resources, and other constraints and understand why an optimal plan does not always mean reported "gold standard" in practice. Describe the importance of evidence-based decision making in the shelter setting. Create adoption releases and protocols that allow clear communication and expectations regarding the animals' health and needs. Demonstrate understanding and knowledge of pathophysiology, risk factors, clinical signs and diagnosis of infectious diseases common in the shelter environment.

General Medicine
Create appropriate treatment plans for shelter animals that take into context the mission and resources of the shelter. Diagnose and treat common conditions found in shelter animals. Assess physical and behavioral health of individuals and formulate a plan for short-and long-term interventions and long-term prognosis. Demonstrate proficiency in taking a history in the shelter setting and creating a problem list most effective for approach and diseases in the shelter. Devise treatment options for patients including different pain management protocols and considerations for time for treatment. Conduct medical rounds in a shelter setting identifying, treating, and monitoring animals for response to treatment. Justify treatment plans and protocols using evidence and appropriate logic. Surgery Describe pros and cons of spay and neuter from a medical, behavioral, and populationbased perspective. Demonstrate proficiency in shelter surgical and anaesthetic procedures including sterilizations and paediatric spay and neuter utilizing appropriate asepsis and tissue handling. Describe different surgical techniques and options and compare to the evidence in the literature such as pedicle ties, scrotal neuters, miller's knots, etc. Devise and utilise appropriate and humane multimodal analgesia protocols for routine surgeries.

Infectious Diseases and Outbreaks
Synthesize information regarding appropriate vaccine handling and protocols and apply in the shelter environment. Create a plan for outbreak management for various types of diseases in the context of a particular shelter's resources and discuss implications of staffing, training, and oversight. Recognize and respond to common infectious diseases in the shelter including isolation and quarantine. Identify elements relevant for evaluating infectious disease risks.
Evaluate aspects of the population, husbandry and biosecurity likely to be important in affecting the risk of infectious diseases in individuals and in the population. Demonstrate the uses and limitations of diagnostic tests, including concepts such as sensitivity, specificity and predictive value. Understand how properties of a pathogen affect its potential routes of transmission. Describe the place of management strategies such as foster care in infectious disease prevention. Utilise appropriate handling, anesthetic, surgical and post-operative management practices for surgical neutering of free roaming animals being returned to their habitat. Euthanasia Recognize methods, laws, guidelines, and management surrounding euthanasia and how to create a humane plan for a patient to provide euthanasia in a timely fashion. Describe appropriate practices for humane shelter animal euthanasia including proper training, staff, and procedures and describe the regulations surrounding this practice. Explain how different euthanasia policies may affect decision making in the shelter environment. Discuss euthanasia decisions in the context of multiple stakeholders with different opinions.

Standards and Assessments
Read and utilize the ASV Guidelines for Standards of Care in Animal Shelters Identify other online resources that can be utilized by shelter veterinarians. Identify spay and neuter training programs and research regarding the practice.

Sanitation and Biosecurity
Review protocols for cleaning and sanitation that take into consideration the population and specific disease risks such as parvovirus or ringworm. Recognize zoonotic diseases, discuss risks to animals and people and develop risk management plan. Use appropriate personal protective equipment and biosecurity principles when conducting examinations. Describe any specific local regulations related to common zoonotic diseases such as notifiable or occupational diseases. Facility and Housing Design Discuss housing recommendations for individual animals based on consideration of species, age, health, intake status, and behavior. Discuss the role that housing and shelter design plays in animal health and welfare in a shelter setting, including preferred types of housing, space allocation recommendations, and impacts of appropriate versus inappropriate housing on animal health, welfare, and live release. Describe how facility design and housing can reduce stress in animals' disease transmission, protect behavioral and mental well-being, and increase efficiency for staff.

Behavioral Health
Overarching Principles Describe and interpret feline and canine body language. Discuss the factors that influence an animal's behavior. Distinguish different professional roles and accreditations in behavioral intervention including shelter veterinarian, veterinary behaviorist, clinical behaviorist, and trainer. Describe the relationship between stress and disease.
Determine protocols for reducing stress and promoting positive welfare. Evaluate and prioritise the sometimes-conflicting welfare needs of an animal in a shelter (behavioral and physical health, short term vs long term welfare, individual vs population).

Population Behavioral Health
Assess the impact of housing and environment on behavior and welfare. Use appropriate behavioral and welfare information to make appropriate outcome and placement decisions within various sheltering models including discussion of humane euthanasia and public safety. Determine strategies to improve individual and population behavioral health and welfare and describe the advantages and limitations of each. Identify opportunities to promote good behavioral health including socialisation and learning, recognising sensitive periods for behavioral development, behaviorally sensitive handling, appropriate environment, and training techniques. Individual Animal Behavioral/Psychological Health Discuss available methods and tools for behavioral interventions in a clinical setting in the context of resources within a shelter. Discuss the benefits and limitations of tools and methods for assessing behavior. Discuss the methods and tools for assessing behavioral health as an element of quality of life assessment. Participate in behavioral modification, enrichment, and socialization for shelter animals. Create plans for behavioral wellness and treatment for individual shelter animals including specific enrichment, management, training, behavioral modification, medication, rehoming, and education strategies. Demonstrate methods of low stress handling including maximizing animal and handler safety and minimizing overall stress. Discuss, and as appropriate select, medication that may be used as short-term (e.g. sedatives) and long-term (e.g. psychopharmacological) intervention to ensure handler safety and aid in the management of behavioral problems in the shelter. Apply the principles of behavioral first aid to ensure handler safety and avoid worsening problem behaviors. Determine appropriate immediate intervention for a behavior problem to ensure handler safety and avoid worsening problem behaviors. Contribute to the development of shelter animal behavior and welfare diagnostic and treatment plans using a problem-oriented approach that maximize behavioral health while minimizing impact to physical health.

Shelter Management Understanding Companion Animal Homelessness in Context
Discuss drivers for pet acquisition, retention, and relinquishment dynamics and risk factors for animal homelessness. Assess potential in-community risk factors for animal homelessness.

Options for Care
Describe the role of the veterinary professional in animal sheltering organisations. List the laws applying to animal sheltering in your community and evaluate their impact on different shelters types. Describe programs designed to maximize shelter live release rates including managed intake, alternatives to admission, removing barriers to adoption, redemption facilitation, rescue/transfer programs, return to field, open selection, owner requested euthanasia, etc.
Describe the considerations of transport programs and discuss guidelines and use of 'best practice' in different contexts.

Care of Shelter Staff
Discuss how shelter administration can support shelter staff who are at risk for compassion stress and fatigue. Discuss stakeholder priorities in the shelter including management, volunteers, public, staff, and veterinarians.

Monitoring and Evaluation
Describe the different ways that animals enter and leave the shelter facility. Describe the concept of capacity of care and apply in the context of different shelter facilities. Identify methods of measurement and evaluation for assessing programmatic success of interventions.
Recognize the importance of animal flow and length of stay and their impact on different shelters' 'capacity of care'. Discuss the format of general operations and flow through and execute pathway planning to decrease length of stay and promote live outcomes. Interpret statistics commonly used to describe shelter population dynamics, assess the availability of this data at different shelters, and apply to procedural recommendations or interventions.

Emergency, Outbreak and Disaster Response
Describe different principles of emergency and disaster response.

Public Health, Community Medicine, and Public Policy Animals and Public Policy
Perform welfare assessment of animals, especially within the shelter setting. Describe the role of shelters and veterinarians in investigating, resolving, and providing care for animal affected by cruelty, neglect, and hoarding and interfacing with human agencies to provide comprehensive and effective interventions. Describe the management techniques, literature, and pros and cons of different approaches to different types of stray/feral populations of domesticated animals. Describe different types of programs to prevent relinquishment to shelters including safety net programs and accessible car. Describe and apply the five welfare needs in a shelter setting and discuss the concept of positive welfare and a good life, beyond the five welfare needs. Public Health Discuss concepts of zoonotic disease and responsibility of the shelter to prevent their spread, considering the shelter as a workspace and a public place to visit/acquire pets. Apply prevention techniques to avoid physical harm in humans when handling animals (dog bites, cat bites/scratches, etc). Describe risks to staff engaged in shelter or community medicine and how to prevent exposure of disease particularly to high-risk groups. Demonstrate knowledge of national legislation and regulations pertaining animal sheltering and welfare, use of disinfectants and zoonotic diseases. Describe knowledge of legal rules and regulations pertaining to reportable diseases.

Animal Cruelty and Investigations
Describe how to recognize animal cruelty and describe the role of the veterinarian.
Investigate how to find laws for your area for animal cruelty and the role of the veterinarian. Develop plans for an animal hoarding situation including an understanding of implications of different types of interventions on the owner and management of the individual animals. Be familiar with the scientific literature on forensics of animal abuse. Describe the link between domestic violence and animal abuse, and how to engage with humane investigations. History Discuss overall trends in stray populations and provide examples of historical interventions (failures and successes) and interpret recent trends surrounding these issues.

Shelter Medicine Principles Communication
Summarise the key concepts of shelter medicine to veterinarians in other disciplines. Demonstrate tolerance for conflicting ideas and an openness and flexibility to accept change. Develop communication skills that develop good relationships with team members, pet owners & fosterers, including cultural sensitivity, compassion, respect, and the importance of prioritising the human animal bond and animal welfare. Recognize personal bias and develop skills to productively discuss difficult or controversial topics. Recognize the challenges of working in a multidisciplinary team in a calm and collaborative manner whilst prioritising animal health and welfare. Recognize employment opportunities that exist for veterinarians in shelters and discuss the types of experience and perspective needed for success in the positions.

Ethics
Recognize responsibility for the welfare of the community and advocate for the ethical principles of the field. Define common ethical approaches and their impact in decision making in animal welfare.

Evidence-Based Veterinary Medicine
Appraise evidence and apply the EBVM approach in both individual and population management strategic decisions, protocols and procedures. Describe and utilize principles of antimicrobial stewardship and articulate why lack of resources does not justify inappropriate use of antimicrobials. Describe husbandry and basic management of exotic, wildlife and farm species including legal and ethical restrictions on treatment. Discuss principles and methods of necropsy, tissue handling and collection of diagnostic samples. Evaluate HQHVSN (High quality high volume spay and neuter) programs for efficacy in addressing community need, efficiency in utilization of resources, and adherence to Veterinary Medical Care. Guidelines for Spay Neuter Programs

Behavioral Health
Explain the underlying learning theory (including classical and operant conditioning, habituation, counterconditioning and desensitisation) used in behavioural modification techniques. Compare the use of reward-based training, and training that applies punishment. Create a behavioral plan for shelter animal in the home including communication with the adopter or fosterer about how to manage it.

Shelter Management
Describe the organisation of different types of animal shelters, including non-profit, animal control or other government entities. Compare the mission, community and wider impact and resources of the different shelter organisations. Describe a typical management structure in an animal shelter organisation. Examine needs for training of shelter and veterinary staff, adopters, and volunteers. Interpret raw shelter data and identify complicating factors for collection, retrieval, and analysis. Investigate shelter level hypotheses and research using relevant shelter data and interpretations. Propose resource reallocation needed in an outbreak situation Recognize the need for appropriate work-life balance strategies supporting shelter staff wellbeing. Discuss the veterinary implications of developing pet friendly housing.

Public Health, Community Medicine, and Public Policy
Discuss cultural and legal differences in animal management locally, nationally, and internationally and identify how general shelter concepts can be applied effectively in these settings. Create a productive plan for advocating for animals within organizations and communities. Recognize the features of the human animal bond and how it impacts people in the community and through the shelter. Describe animal bite prevention programs. Develop skills to identify and respond to emerging diseases including a surveillance plan for reportable diseases Understand basic disaster preparedness and response concepts. Engage in one-health concepts and organization and coordination with human and environmental interventions. Discuss impact of different types of trade (meat trade, live markets, etc.) on animal welfare and public health and complicating importance in other cultural contexts. Define and interpret common epidemiologic and statistical terms including incidence, prevalence, rate, odds, risk, relative risk, etc. Recognize the importance and diversity of the human animal bond. Recognize the link between animal cruelty and interpersonal violence. Describe the possible roles a veterinarian can play in humane investigations. Complete a SWOT (strengths, weaknesses, opportunities, threats) analysis of the shelter's current position / progress in relation to historical trends.

Shelter Medicine Principles
Design, rationalise and communicate possible treatment plans including recording of concise clinical notes. Develop appropriate techniques for communication within the shelter including facilitating teamwork. Work effectively as part of a veterinary team demonstrating calm and collaborative behavior. Discuss the role of the veterinarian in complex decision making e.g. depopulation, treatment of the population versus the individual and confinement. Discuss the use in shelter medicine of some of the basic frameworks for understanding and instigating human behaviour change. Table 4: Learning outcomes not accepted following Round 3. 254

255
This study has utilised a consensus-based Delphi approach to identify and refine learning 256 outcomes which are relevant to shelter medicine education worldwide. This represents an 257 evidence-based approach to developing this nascent discipline with a grounded, inductive 258 strategy. The resulting learning outcomes can serve as a tool for those developing veterinary 259 curricula for academic programming relevant to all graduates (27). It has been suggested that 260 involving more veterinarians in shelter medicine, and equipping them with the tools to 261 practice this discipline has the potential for an enormous positive impact on animal welfare 262 (2,8,9,28). 263 These learning outcomes will be relevant both as a foundation to those intent on a career in 264 shelter medicine, and to the many veterinarians entering into primary care practice where 265 shelter medicine will form a component of their work. should feel empowered to include and interpret them in the context of their own needs. 291 Additionally, within this context, the LOs were focussed on the cognitive domain of Bloom's 292 taxonomy, and those deemed highly focussed on affective or psychomotor domains were 293 excluded (with the exception of two LOs considered by the authors to be fundamental to 294 shelter medicine, and unlikely to be seen elsewhere). This decision was made due to the 295 extensive crossover with primary care which these represented; however, this highlights an 296 area for future exploration. 297 Additionally, further discussion regarding the value of defining learning outcomes for first-298 day skills learned through shelter medicine and learning outcomes for shelter specialists is 299 warranted. This was initially intended to be part of the present study. However, through the 300 process of this research, considerable geographical variation emerged concerning whether 301 specific skills and learning outcomes were regarded as appropriate to pre-registration 302 education, and which were more specialised. It is possible this reflects aspects of veterinary 303 education (which is a primary degree in the UK but postgraduate in the USA and elsewhere). 304 It may also be influenced by well as how integrated shelter medicine is into general practice, 305 and its stage of development as a distinct specialty, in different locations. In the light of this, 306 it was considered that developing a core list of Shelter Medicine LOs should be considered an 307 essential first step. As shelter medicine continues to develop as a specialty in more areas 308 across the world, the need for consideration of universal international learning outcomes at 309 the specialty level would be especially helpful. 310 Limitations 311 Participants were included based on self-identification as a shelter medicine educator. The 312 United States is the only country that so far has a recognized veterinary specialty in shelter 313 medicine and therefore, given the international nature of this study, panellists were drawn 314 from a variety of unstandardized contexts within veterinary shelter medicine education. This 315 strategy was chosen in order to represent as broadly as possible the scenarios in which these 316 LOs might later be utilised. This is consistent with similar Delphi exercises in which expert 317 status has been defined by being embedded within the practice being explored (20,25,30). 318 Although the geographical spread of the panellists initially recruited was relatively diverse, 319 the eventual participants were very heavily skewed towards Europe and the USA. It is likely 320 that this affected the final choice of learning outcomes. It is possible that providing the 321 surveys in languages other than English may have helped capture more diverse opinions; 322 however, this was not considered logistically feasible within the constraints of the project. 323 Similar studies have utilised a formal thematic analysis in order to incorporate free text 324 comments into shaping learning outcomes (21, 31). In this study, free text responses were 325 scant, and therefore panellists' comments were discussed among the research team and 326 utilised in a less formalised manner. 327 328 The Delphi methodology is described in the literature as a tool to generate discussion (13). It 329 is recognized for its democracy and its scope in reaching a consensus, and for harnessing 330 expertise in the face of logistical challenges such as geographical spread (16). The final LO 331 list is intended to establish foundations of the ever-developing shelter medicine field within 332 the curriculum. This study has documented the panellists' areas of interest and honed the list 333 of learning outcomes, all at different levels of Bloom's taxonomy, covering many facets of 334 this new discipline. The learning outcomes generated are intended as a resource to be utilised, 335 shared, developed and adapted, and should not be regarded as a final pronouncement, but 336 instead a foundation for the basis of continued re-evaluation and re-development. 337

338
A set of LOs, developed and evaluated by agreement by shelter medicine educators 339 internationally has now been collated and can be used to embed shelter medicine teaching 340 within pre-registration curricula worldwide. The LOs can be adapted in any educational 341 setting in any country with confidence that colleagues believe the subject material to be of 342 value. Shelter medicine can be taught alongside and integrated with other disciplines to help 343 enhance the field and further develop the practice of cross-cutting themes. This international 344 study has emphasised the breadth and depth of this important discipline, which is of value 345 and necessary in its own right, and not simply an addendum. 346