Engineering education for sustainable development: using online learning to support the new paradigms

Abstract This paper explores the experiences of three academic members of the University of Nottingham Open Online Course (NOOC) and Massive Open Online Course (MOOC) team, comprising an engineer (tutor), an engineering education specialist (facilitator) and a specialist in higher education pedagogy (convenor). The paper explores notions of what makes for effective teaching of sustainability within a multidisciplinary online context, and the extent to which this experience has impacted upon personal behaviours and attitudes to sustainability, from an Engineering Education for Sustainable Development (EESD) perspective. Team members’ experiences are further supported by findings from student and learner evaluations of the NOOC and MOOC courses. Key findings of the research include (a) Interdisciplinarity is a strength of the course (b) The course can lead to genuine change in the understanding of sustainability; (c) Teaching sustainability online is different; (d) Involvement in the course impacts upon teaching practices; (e) Cultural and disciplinary diversity within both the course team and the student cohort is a major contributor to the overall sustainable development learning experience in the NOOC/MOOC. It is hoped that the findings of the study will pave the way for engineering and non-engineering educators to explore the potential of integrating sustainability within the modules they teach, via online teaching and learning means.


Introduction
The engineering profession continuously evolves as it responds to environmental, social and economic demands (Carew & Mitchell 2001). While built upon possession of 'a broad spectrum of artistic knowledge, craftsmanship and management skills' (Kastenhofer et al. 2010, 44), in recent times the profession has shifted towards a more specialised and technically focused vocation. The dawn of the industrial engineering era is said to be the root of this shifting perspective, which has also signalled a requirement for engineers to conform to industrial legislation and responsible practices towards the environment and society (Kastenhofer et al. 2010). These 'responsible practices' require engineers to develop their sustainability literacy (knowledge, competences, values and attitudes related to sustainable development) and competency as the profession faces up to the environmental, economic and societal concerns of the twenty-first century. That the profession is trying to do this can be attributed to its adaptability in 'adjusting its accustomed approach' (Carew and Mitchell 2001, 1). The adjustments being made are resulting in a paradigm shift from development engineering to sustainable development engineering (Thom 1996;Clift 1998;Mitchell 1999).
To play their part in the creation of sustainable engineering technology and to become change agents for sustainability, today's engineers need to be equipped with the knowledge, skills and values necessary to contribute positive solutions in mediation of the impact of climate change and global warming. There is recognition that 'the need to educate the engineer of the 21st century differently -or more precisely, more strategically -is essential to the endurance of the profession' (Galloway 2008, 5) within a higher education context that is shifting towards Engineering Education for Sustainable Development (EESD). Leal (2009) notes that universities have a responsibility towards their students, faculty and staff to develop not only skills for success in a globalised world, but also positive attitudes towards environmental issues which can be added to these skills for the longer term benefit of society. This responsibility cannot be over-exaggerated considering the evidence for the negative influences of university graduates on the ecosystem (Corcoran and Wals 2004). Higher education must prepare its graduates to embrace sustainable development, as observed by Cortese (2003) and Leal (2009). The sector's commitment to sustainable development is evidenced by the Higher Education Sustainability Initiative (HESI) which emerged from the Rio 2012 summit. The HESI action plan addresses five areas which are seen to encompass both the contribution and the responsibility of higher education. The articulation of the five indicates a holistic approach that connects sustainability to all disciplines and positions it as a core competency for graduate-level employment: • Teach sustainable development concepts as part of the core curriculum for all, developing employable graduates with sustainability literacy; • Encourage research, knowledge exchange and innovation; • Model sustainability throughout all operations and campuses; • Work in partnership to support sustainable local community-building; • Share learning through international frameworks and report regularly on progress and challenges (http://sustainabledevelopment.un.org/index. php?menu=1073) While HESI and other initiatives are positive, it is important to scrutinise the learning environments in which EESD operates. If learning occurs in an educational system based on educational views that 'sustain unsustainability' , then this learning can itself contribute towards the continuing production of graduates with unsustainable behaviours and attitudes, state Davis and Cooke (2007) and Jucker (2011). Also fundamental is the vision and culture of the institution, and its readiness to embrace change that is transformational (Sterling 2004). Several studies have urged higher education to 're-think' not only its curriculum, but its pedagogy too, in particular by the application of systemic thinking to course design and delivery (Capra 2000;Martin and Murray 2010).
In 2013, the University of Nottingham (UK) accepted the challenge to 're-think' education for sustainability as part of an initiative to offer our own students and staff, but also a wider audience, the opportunity to study sustainability via a collaborative, peer-supported online course. We embarked upon an online learning initiative that resulted in two courses: the Nottingham Open Online Course (NOOC), Perspectives on Sustainability, which was accessible to our students and staff, and the Massive Open Online Course (MOOC) Sustainability, Society and You which was accessible to anyone with an internet connection. This paper explores the experiences of three academic members of the NOOC and MOOC team: one engineer (tutor), one engineering education specialist (facilitator) and one specialist in higher education pedagogy (convenor). The paper explores notions of what makes for effective teaching of sustainability within a multidisciplinary online context, and the extent to which this experience has impacted upon personal behaviours and attitudes to sustainability, from an EESD lens. Team members' experiences are further supported by findings from student and learner evaluations of the NOOC and MOOC courses.

Nottingham's NOOC and MOOC in context
The University of Nottingham is well known internationally for its commitment to sustainability, demonstrated in teaching and learning initiatives, through environmentally responsible research and across the campus operations. Since its inception in 2010, it has remained in the top two of the UI Green Metric World University Ranking, being placed first for three consecutive years (2013)(2014)(2015). There have been year on year increases in recycling and recovery across the University and a significant reduction in waste per student. Carbon reduction projects have resulted in 11,000 tonnes of carbon savings and an absolute reduction in emissions against the backdrop of an expanding estate. Expanding transport choice, including a link to the city's light rail network have enabled more active travel and, with the recent introduction of workplace parking levy, less reliance on the car for travelling to the University. In addition, 11 major capital projects have achieved BREEAM excellent or better, more than any other University (The University of Nottingham 2016).
Some undergraduate teaching programmes, particularly some located within the Faculty of Engineering have strong environmental content, whilst many courses allow students to take optional modules in the field of sustainability to complement their core degree. As part of the Higher Education Academy's Green Academy 'Curriculum for Tomorrow programme' the University created a small team including academics, professional service staff and students who, supported by, sustainability and change management experts, helped to develop an action plan to embed sustainability within the curriculum (The University of Nottingham 2016).
The NOOC is a cross-campus online course which aims to 'develop holistic understanding of the values and principles associated with sustainability and to equip students with some of the knowledge and understanding required to make sustainable decisions' . As of December 2014, the course has run twice with 1,650 student and staff enrolments (838 in first iteration, 812 in the second). These students and staff are drawn from across the University of Nottingham's campuses in the UK (47%), China (43%) and Malaysia (10%). The course is designed to allow for flexible and self-directed learning. Participants can follow as much or as little of the course as they wish, but there are incentives to complete a range of learning activities to earn either credits or a certificate of achievement.
The MOOC 'provides an introduction to the values and principles associated with sustainability and will equip you with some of the knowledge and understanding required to make sustainable decisions in your personal and professional life' . As of summer 2014 it has run twice, initially as an 8-week course (January-March 2014, with 8,729 enrolments) and then as a 6-week course (June-July 2014, with 4,370 enrolments). Those enrolling were drawn from across the globe. Participants can choose to purchase a certificate of completion as long as they have completed 50% of all the 'steps' in the course. A step can be a reading, a video, an activity, or a peer review.
The courses are based upon the same content, with slight nuances for different audiences. For example, there will be more material specifically related to Nottingham in the NOOC. Both NOOC and MOOC are multidisciplinary in nature, involving areas such as engineering, business, education, the arts and languages, social policy and health. They were developed by a university team of academics with expertise and interests in sustainability, working with online learning specialists to use course resources within an effective online learning structure. A cross-university team of tutors and facilitators from the University's UK, Malaysia and China campuses provided facilitation and support, and engaged with the learners daily.
The courses have been well-received within the institution and sector. In 2014, the NOOC was shortlisted for a Guardian University Award, while the MOOC was 'Highly Commended' at the Green Gown Awards, sponsored by the Environmental Association of Universities and Colleges. Completion rates have been good. For the NOOC, an average of 25% of students enrolled met the requirements for the award of credit towards the University's co-curricula skills award, The Nottingham Advantage Award. Over 70% of all students enrolled completed at least one activity or one week in the course, and thus were eligible for the award of a badge. The MOOC has seen an average of 47% for social learning' , higher than the average for most MOOC-style courses of 25%. The term 'social learners' refers to those participants who have engaged with the course to the extent of posting comments and completing activities; they have gone further than simply accessing content. Evaluations of both NOOC and MOOC by participants have been positive, both in terms of the online learning experience, and in relation to the academic content. For example, Nottingham students on the NOOC told us that they liked: The fact that it was offered on all campuses so I could interact with fellow university students and hear what things are like on their side of the world.
The variation in tasks-different medias and new techniques of analysis I had not used before e.g. SWOT.
Being able to choose what assessments to complete, and being able to fit it in around when I had time.
A repeated comment from the MOOC learners was that the course inspired them to change their personal practice, for example: This has been an inspirational course and I intend to switch to Good Energy as soon as I move into my own home in a couple of months.
Further examples of qualitative evaluation are included within the analysis below.
The design of the NOOC and MOOC is based upon a triple-layered understanding of sustainability (sustainability as content; sustainability as pedagogy; and sustainability as process). Each course builds its learning community over 6-10 weeks by taking learners on a journey from simple conceptualisations of sustainability to critical thinking and ethical judgement. The core idea is that individual learners can 'make a difference' both by their own positive actions, and via their potential agency as catalysts within their own communities. Activities and tasks, carried out both online and offline, encourage learners to audit their own footprints (waste, water, energy, food), to connect sustainability to issues of social justice and to develop skills of advocacy via the collaborative production of learning resources (e.g. bookmarking, posters, critical reviews).
Before examining what makes for effective teaching of sustainability within a multidisciplinary online context, and the extent to which this may impact upon personal behaviours and attitudes to sustainability within an EESD lens, it is first necessary to understand the context within which education for sustainable development (ESD) and EESD operate. A review of current debates on the integration of ESD and EESD within the higher education context therefore follows.

Current debates on the integration of ESD and EESD within the higher education context
Education is a significant resource, both positive and destructive (Schumacher 1973). For higher education, delivering sustainable development literacy through the curriculum is claimed to be the most significant contribution that a university can make to the global sustainability agenda. Unfortunately, this contribution is under-developed (Martin and Jucker 2005). As Sterling (2001) notes, education as a whole must become 'sustainable education' if sustainability is to be infused throughout the core business of every university -its teaching and learning. Sterling (2004, 50) asserts that: Sustainability does not simply require an 'add-on' to existing structures and curricula, but implies a change of fundamental epistemology in our culture and hence also in our educational thinking and practice. Seen in this light, sustainability is not just another issue to be added to an overcrowded curriculum, but a gateway to a different view of curriculum, of pedagogy, of organisational change, of policy and particularly of ethos.
At present, most higher education is seen to add to unsustainability as it does little to address the 'whole and interdisciplinary approaches, the issue of ethics and the fact that sustainable development is an evolving field, as factors that impede the implementation of sustainable development. Mulder and Jansen's (2006) study on integrating sustainable development in engineering education at Delft University of Technology (Netherlands) revealed academic culture and organisational issues as barriers to carrying out sustainable development initiatives at the university. Cultural issues cited included the fear of external forces threatening academic freedom, the race for scientific credentials, preservation of strong disciplinary borders and expertise areas and resistance to curriculum change. Organisational issues included time, the availability of resources and personnel, political processes within departments and being attuned to the demands of industry stakeholders.
In another study on implementing sustainable development for engineers at the University of Technology, Sydney (Australia), Bryce, Johnston, and Yasukawa (2004) found that the faculty structure and a narrow curriculum were stumbling blocks. There was a failure to promote understanding of the social and environmental contexts for engineering practice. In addition to these issues, the university's pressure on faculty to win research funding inadvertently led to the establishment of discipline-centric research groups instead of interdisciplinary and transdisciplinary research clusters (which are seen to be important in sustainable engineering research). The findings of this study support those of Velazquez, Munguia, and Sanchez (2005) and Mulder and Jansen (2006), indicating discipline centrism, organisational structure and the refusal to shift away from areas of expertise as common hurdles to the implementation of sustainable development in engineering and non-engineering programmes.
Strategies and recommendations to counter these challenges are seen as important measures in smoothing the transition to sustainability in institutions of higher learning. In 2012, a study on 'Turnaround Leadership for Sustainability' in higher education in Australasia, North America, the UK and Europe, was conducted by the University of Western Sydney, in partnership with The Australian National University and the Sustainable Futures Leadership Academy. The study was conducted in the wake of 'a complex, interlocked and rapidly unfolding set of sustainability challenges underpinned by social, cultural, economic and environmental developments' (Scott et al. 2012, 1) faced by the higher education sector. 'Turnaround Leadership' noted the need for higher education to take a leading role in developing future leaders who possess the skills to manage the challenges of sustainability effectively. The study outlines 10 strategies for the systematic implementation of sustainable development in higher education institutions: person' -'spirit, heart, head and hands' (Sterling 2001;12). Sterling goes on to note that educational reorientations have given education a mechanistic paradigm, in which it is (a) 'still informed by a fundamentally mechanistic view of the world, and hence of learning, (b) largely ignorant of the sustainability issues that will increasingly affect all aspects of people's lives as the century progresses and (c) blind to the rise of ecological thinking which seeks to foster a more integrative awareness of the needs of people and the environment' (Sterling 2001, 13).
Pedagogies related to sustainable development centre upon principles that are associated with sustainability and pay importance to learner's beliefs, interaction, engagement and active creation of knowledge, which are all fundamental sustainable development learning principles. For engineering education, for example, this means using Sterling's (2001) notion of 'sustainable education' to foster a holistic learning experience which is 'meaningful, engaging and participative, rather than functional, passive and prescriptive' (Sterling 2001, 27). A holistic engineering education curriculum will be a possibility only if sustainable development becomes an essential component of the university's engineering education curriculum, culture, policies and teaching and learning practices.
There has been much debate on the integration of ESD and EESD within the higher education context. Ryan's (2011) study, for instance, identified challenges in addressing ESD in the curriculum. These were: creating an integrated understanding around ESD; developing interrelated approaches across the university; and positioning of ESD within the university's teaching and learning functions to enhance teaching and learning. A study conducted by Martin, Dawe, and Jucker (2006) on embedding sustainable development in higher education in the UK revealed four obstacles: an overcrowded curriculum; irrelevance of sustainable development as perceived by academic staff; lack of staff awareness and expertise; and a lack of institutional drive and commitment. Similar obstacles are observed by Valazquez et al. (2005) and in Down's (2006) study of Jamaican trainee teachers, where staff scepticism, students' course expectations, content versus actual course with ESD input, policy absence and syllabus constraint, were cited as some of the challenges facing attempts to make sustainable development 'mainstream' .
Besides these factors, Wells et al. (2009), in their case study of the role of academia in regional sustainability initiatives in Cardiff University in Wales, UK, discovered that there was little engagement of universities in regional sustainability initiatives. Leal (2009) found additional barriers in institutional sustainability policies, staff and student mobility, need for staff training and sustainability issues with research. Sterling (2011) cites the lack of content boundaries, the use of holistic job descriptions that do not clearly support sustainable development initiatives and apprehension of criticism from more authoritative or influential staff, faculty and groups (Brinkhurst et al. 2011).
Lozano's (2010b) study of Cardiff University's (Wales) adoption and diffusion of sustainability in its curricula indicated that there is a tendency for the university to address sustainable development as 'a 'portfolio' , leading to 'compartmentalization, over-specialization, and reductionism' (643). This has resulted in academic schools excelling in their individual areas and in a specific facet of sustainable development. Lozano notes that if sustainable development is integrated 'as a concept, in and among the different disciplines and schools, and tailored to their specific nature' (643), universities could become more 'balanced, synergetic, trans-disciplinary and holistic' (643) enabling their graduates to become competent sustainability change agents. Lozano (2010b) also recommends sustainability reporting as a strategy for universities to gauge their environmental, economic, social and educational impact on sustainable development. Sustainability reporting is viewed as a useful strategy for universities to undertake in communicating sustainable development initiatives to their stakeholders.
Waas, Verbruggen, and Wright's (2010) study focused on definitions and features of sustainable development research at universities. Twenty-two characteristics of university research, defined as either content or process, were identified from three sources of data: sustainability in higher education literature; sustainability in higher education documents, and reports and workshop findings from the University of Antwerp (Belgium). Process characteristics included action-orientedness, management of the environment, safety and security, transfer of data through varying means and to varying groups, i.e. students, the general public and policy-makers, multidisciplinarity, and interdisciplinarity and the need for university research to be reviewed by society. Content characteristics included 'coherence of economic, environmental and social aspects, long term perspective (inter-generational links), global north-south distribution aspects dealing with uncertainties' (Waas, Verbruggen, and Wright 2010, 633). Mickwitz and Melanen's (2009) study analysed the co-operation between Finnish academicians and local decision makers in the development of sustainability indicators. They found that the joint production of knowledge was significant and supportive of the development of purposeful sustainability indicators. Researchers at the University of British Columbia identified seven ways in which sustainable development could be 'infused' throughout the institution: (1) the infusion of sustainability in all decisions, through the incorporation of sustainable development in the university's vision and mission statements, goals and processes; (1) acknowledge the distinctive challenges and complexity of education for sustainability (EfS) leadership (2) sharpen the focus and understanding of EfS as it applies in higher education (3) context counts: ensure organisational integration and system alignment to support EfS and its leaders (4) track and improve EfS programme quality more systematically (5) put in place the right incentives (6) engage the disengaged and the institution's senior leadership (7) apply the key lessons of successful change management in higher education (8) focus on the change-leadership capabilities identified in this study (9) review EfS leadership position descriptions, selection processes and succession strategies in the light of the study's findings (10) apply the most productive approaches to leadership learning identified in the study to the professional development of EfS leaders (Scott et al. 2012, 2) Brinkhurst et al. (2011) conducted a study to explore environmental sustainability and organisational transformation at universities across North American universities. These universities characterise their sustainable development initiatives (campus operations, financial and administrative planning, policy, curriculum and research that facilitate environmental changes) as either top-down (by university administrators) or bottom-up (by students), instead of focusing on the role of the 'institutional middle' (340), namely the faculty and staff often most involved in the development of such initiatives. The top-down and bottom-up approaches have their own set of challenges, including the creation of leadership gaps as a result of dependency on individual advocates who are substituted fast. Another challenge is the lack of awareness of the functioning of the university. This leads to awareness-raising initiatives rather than policy or planning changes. Top-down challenges include the lack of support from the university community (who may not feel represented or consulted in decision-making). The complex nature of university governance and a perceived lack of institutional support are some of the barriers of the bottom-up approach. An adverse effect of both approaches may be the creation of barriers to effective and long-term campus change. Given these challenges, the engagement of the institutional middle is seen as an important tool for achieving sustainability of the university (Brinkhurst et al. 2011), although this also has its difficulties. These include resistance from uncooperative superiors or project partners, shortage of time, lack of authority, disempowerment as change agents within a bureaucratic institution, heavy workloads, (indeed those who are more likely to feel that it is not related to their discipline). (Christie et al. 2015, 658) This paper describes one initiative to break down these barriers and straddle disciplinary boundaries within an online course. Looking through an engineering education perspective and a pedagogic lens, it offers one means of facilitating holistic learning and joint production that may support the development of sustainability literacy and competency both for engineers and also for other university stakeholders.

Methodology
Mixed methods research designs are commonly used for evaluation, social and behavioural research and for educational research (Johnson and Onwuegbuzie 2004). While authors like Tashakkori and Teddlie (1998) consider mixed methods as a research methodology with philosophical assumptions, other mixed methods proponents like Greene, Caracelli, and Graham (1989), Creswell (2003) and Creswell and Plano Clark (2007) focus on the methods used for the collection and analysis of data.
This research adapted a triangulation mixed methods research design, in which quantitative and qualitative approaches are used to collect different but complementary data to address the research aims of the study (Morse 1991). The triangulation design is usually employed when the researcher wishes to collect and analyse 'concurrent but separate' quantitative and qualitative data (Creswell and Plano Clark 2007, 64). Upon data collection, the data-sets are analysed separately. The results of both data-sets are then merged by combining the separate results in the interpretation and analysis.
For this study, three members of the course team: one engineer (tutor), one engineering education specialist (facilitator) and one specialist in higher education pedagogy (convenor); separately generated lists of research questions that aimed to explore our learning from the experience of working on the course. We then collaboratively refined these questions into an agreed list of five topics that were used as prompts for reflection in individual interviews: (2) the promotion and practice of collaboration across disciplines; (3) institutional change for the promotion and practice of interdisciplinarity and transdisciplinarity, (as a disciplined focus approach was seen to stifle creativity and innovative problem-solving opportunities, and did not allow faculty members to teach outside their departments, given the structure of the university); (4) creating a focus on personal and social sustainability through the reduction of work load, reconfiguration of timetables, added community involvement in teaching and improved job security for sessional lecturers, i.e. contract-based non-faculty lecturers with teaching responsibilities; (5) coordinating planning and assessment strategies with the university's academic plans, policies and implementation strategies, and using sustainable development indicators in evaluation criteria and performance indicators; (6) integrating teaching, research and service, instead of overemphasising on peer-reviewed publication and research, and undermining the importance of teaching excellence (7) encouraging and supporting transformative and transdisciplinary undergraduate learning through student-centred learning, collaborative group work, increased interaction between students and lecturers, reflective and active learning (Moore 2005).
In discussing partnerships for the advancement of sustainability, Australia's (TNEP 2016), a collaborative not for profit sustainability project comes to the fore. TNEP, which is made up of QUT, Griffith University, Adelaide University, ANU and Curtin University is an example of a successful model of institutional cooperation between sectors such as education, research and development of policy towards the development of innovations for sustainability.
The empirical research cited above indicates what little evidence we currently have for what makes for effective teaching and learning for sustainability. It is more common to identify the challenges than to report on successful solutions. There is, for example, a consensus that higher education adds to unsustainability where it does not prioritise holistic learning experiences built around the joint production of knowledge. Yet university structures mitigate against initiatives that straddle the boundaries of disciplines or cross the barriers between knowledge producers and consumers. The research gap is clear: Few studies have sought to identify teaching academics' conceptions, beliefs or experiences of EfS in higher education (Cotton et al. 2007), and even fewer from a nationwide, multidisciplinary perspective encapsulating teaching academics not particularly associated with EfS

Interdisciplinarity as a strength or weakness of the course
The course was originally built around five e-books that were disciplinary specific (geography, engineering, business, education, arts and humanities). 1 For the convenor, it was the overlap between these that 'nudged' the team into deciding that there should be one interdisciplinary course rather than five separate offers (Speight and Morgan 2014). Subsequently, the range of disciplines involved was expanded with colleagues from Philosophy, Sociology and Social Policy, Nursing and Mental Health, Chemistry, Economics and Built Environment contributing to the 2nd version.
Interdisciplinarity was a means of creating connections. One way was in reaching out to as broad a population as possible by relating sustainability to 'non-obvious' as well as obvious disciplines: We consciously worked from the position that sustainability was of direct relevance to everyone and every discipline. (convenor) This was important. In 2011 focus groups had been carried out with student leaders on all three campuses of the university (Speight and Morgan 2014). These had suggested, in line with the literature cited above, that the chief barrier to the integration of sustainability values into teaching and learning were disciplinary; students and tutors in 'non-obvious' disciplines struggled to see the direct relevance of sustainability to their work/study (Emanuel and Adams 2011;Clifford 2013, 2015;Sivapalan 2015). In response, the course used the 2010 work of the Centre for Sustainable Futures at the University of Plymouth which had explored the connections between disciplines such as Drama, Music, Theology and Social Work with sustainability (Jones, Selby, and Sterling 2010).
Interdisciplinarity also triggered debate. While the construction and delivery of the course brought together people from different disciplines, they did not always share the same views: The tutor, facilitator and convenor separately responded to these prompts and produced a reflective account of our experiences. We then came together to discuss our reflections, using a simple coding system to highlight the key points made. Following this, we referred back to a series of existing data-sets collected during, and at the end of the course to identify the strongest areas of shared learning as indicated not only by members of the course team but also by students.
One of the challenges in triangulating the different sources was showing due regard for ethical issues around ownership of data and consent to use information submitted as part of a teaching and learning rather than research process. Students who completed the course feedback survey did so with the knowledge that their views would be used to inform subsequent version of the course and in assessments of the value of the online format in learning for sustainability. In the few cases where we used data drawn from blogs or other course activities, we contacted the individual authors to secure consent for anonymised use.
The range of data sources used and their relationship to each other is outlined in Table 1.
The findings of the study are discussed in the section that follows.

Discussion
The discussion of the findings of this study is as presented below. The findings are discussed in accordance of the five key themes identified, i.e. Interdisciplinarity as a strength or weakness of the course; Changing views on, or understanding of, sustainability during the development and delivery of the course; Teaching Sustainability online; Notions of change within teaching practice as a result of involvement in the course/other sustainability projects; Impact of different cultural backgrounds of participants, tutors and facilitators on the overall sustainable development learning experience in the NOOC/ MOOC. Each of these dimensions was strengthened in successive iterations of the course. The first version was less interdisciplinary than the second, the second built on evaluation to enhance learning, the second and third provided more opportunities for direct engagement between learners and course team.
For the tutor, the exposure to different disciplines and perspectives provided a deeper appreciation of alternative viewpoints but it did not lead to change. For example, the inclusion of Corporate Social Responsibility (CSR) within the course seemed at odds with a holistic approach: I see sustainability holistically -it's more about 'who you are' rather than 'what you do' . In particular, I don't like the idea that sustainability is an optional extra or something that can be bolted onto a corporation's strategy. I think that's why the whole CSR thing grates so much with me. (tutor) This tutor worried that the online sustainability course would be seen as the solution to ensuring students developed sustainability literacy, rather than a starting point for something more effective: I think that many aspects of sustainability have to be experienced rather than taught, and that's very difficult to do within a traditional undergraduate programme. It's also something that is hard to achieve from the perspective of a lecturer of one module, with limited influence over the whole course. Having one 'sustainable' module isn't going to compensate for nine 'unsustainable' modules. (tutor) But there was evidence that the course experience led to changes in students' understanding of sustainability between the start and finish of the course. For example, in a separate piece of work we analysed the blogs written by a small group of students (37) active in both weeks 1 and 10 of the first version of the course (Speight and Bowman 2015).
This demonstrated that the greatest shift in complexity of student understanding of sustainability was demonstrated by those students who during week 1 gave either a broad and non-specific response to the question, or provided a very limited response. By week 10, they had shifted to being able to provide qualitatively different examples of concrete and/or abstract issues relevant to sustainability. For example, a student in week 1 responded to a request to describe sustainability with a paraphrasing of the Brundtland definition: 'Sustainability is concerned with the past, the current and the future' .
By week 10 the student was able to demonstrate individual thinking as follows: I have learnt that sustainability is a massive, intricate issue that cannot be solved by an individual alone, but will take many people all over the world working together in all walks of life trying to meet various goals in order to become truly sustainable. Issues like sustainable water and energy have impacts on the sustainability It was very interesting to hear how staff and students from outside of engineering defined sustainability, which led to some debate when putting the course together. (tutor) I didn't always agree with what was being raised in the discussions. (tutor) It was also interesting to see participants interacting across their disciplines of study or professions when discussing the questions posed in the weekly learning activities. As a facilitator, I found this enlightening, as I too was able to learn beyond my own discipline of education. (facilitator) The fact that the course was online, meant also that the learner demographic was more varied than might be expected on a campus-based course. Students appeared to value this: I have a new approach of learning things by online classes and I have the chance share my ideas with other students from other countries. (student evaluation) Perhaps the best thing of this course is its flexibility available for students of different specialities and backgrounds. (student evaluation) Alongside the stimulus of interdisciplinarity were differences of culture and context, allowing for a rich sharing of knowledge and experience. People shared personal rather than professional experiences, and in so doing pushed the course team to think again about the curriculum choices we had made: Having voices from different cultures and nationalities challenged some of the assumptions that were made; for instance, it was all too easy to assume that everyone would be familiar with using the kitchen appliances that were being discussed. (tutor) This 'rich sharing' and challenging of assumptions came through in discussion of 'appropriate technologies' -where the course benefited not only from input from engineers, but that this input came from engineers trained and working in very different contexts.

Changing views on or understanding of sustainability during the development and delivery of the course
While members of the course team felt that they had developed understanding of sustainability already, they felt that the course experience helped them to refine or articulate this. The convenor felt strongly that the course had a 'triple-layered' conceptualisation of sustainability running through it: • Sustainability as content (topic focused); • Sustainable pedagogy (formative assessment, interactive and active learning); • Sustainability as process (use of open educational resources, student-generated content; use of evaluation to inform design).
way, we positioned ourselves as co-learners rather than as experts.

Teaching sustainability online
By the second iteration of the course, there was a clear focus upon the learning journey of students. The content was reorganised and augmented by new material from different disciplines and the activities and discussions were reviewed and re-structured so that learning deepened from week 1 through to week 10 (Table 2). Weekly activities were mapped against learning outcomes and transferable skills, ensuring that 'learning' was more central than 'knowing' . For example, students began the course by writing blogs about their current understanding of sustainability and commenting upon the blogs of their peers. They moved on to collaboratively constructing a visual history in which they found and captioned images telling the story of the developing sustainability agenda.
In the middle weeks of the course they carried out waste audits, measured their water footprint, and produced a SWOT analysis of an organisation's sustainability strategy. In the final weeks they prepared poster presentations to prompt informed behaviour change.
The third version of the course attracted more students and students from a wider range of backgrounds and contexts. This was because it was offered as a MOOC on the FutureLearn platform. While the content of versions 2 and 3 was more or less the same, there was some editing for tone and relevance to an international group of professionals, students and engaged members of the public. In the MOOC we had engineers in practice, in retirement and in training. There was more opportunity to use their knowledge and expertise and so the role of the tutors and facilitators became more supportive and less instructional: of a food supply and it is difficult to meet all of the demands in a sustainable manner without some sacrifice along the way. The thing I found difficult to comprehend was the sheer intricacy of the sustainability web, How one thing affects another and how difficult it is to find a balanced middle ground of all the things that need to be taken into account. Those students who already had a developed understanding in week 1 showed a reinforcement rather than development of their knowledge in week 10.
The tutor was concerned that sustainability had to be experienced rather than taught. This concern was addressed within the development of the course by the construction of assessed tasks firmly located in personal action. One of the most significant learning experiences was the waste audit. This pushed students into reviewing their own behaviour and making practical changes to daily routines. It helped students to realise that personal action, although not easy, was possible: For me or most people, learning and understanding the sustainability is easy. But it is difficult to carry out the sustainable actions in the life. This is a serious problem that knowledge we have learnt online is not applied into the life. For example, the amount of paper wasted is related to us closely..….. Maybe we should consider how to persuade people to act in a sustainable manner and contribute to a sustainability society together. (student waste audit report) I will certainly think more about recycling and when buying food next year when I'm in my accommodation.

(student evaluation)
The course is great, because it let me know I can do something for sustainability. (student evaluation) This emphasis upon personal action was extended to the course team in that we all made personal pledges ourselves at the end of the MOOC versions of the course to change something about our own behaviours. In this before scheduled classes ('flipped classroom'). This is supporting more in-depth and informed work within the classroom. (convenor) There was a feeling that models for online learning could be adapted for use in face to face teaching, for example, in managing student queries: one design principle of NOOCs is to make each action count for as many students as possible. On the basis of this I instituted a departmental policy that all student queries be posted and answered in Moodle discussion forums. (reflective blog written by a course tutor) The course materials were also available for reuse and that helped some tutors to introduce a stronger sustainability dimension into the curriculum of the modules they were teaching: in an Undergraduate module, 'Land and Landscape in Modern British Culture' I have made use of materials from our Arts and Humanities e-book. Sustainability has become a topic of discussion, linked to planning legislation, notions of 'rural England' , preservationist principles and the surviving evidence of past land use. (convenor) there was an interesting debate about rubbish cultures which has enhanced my understanding of how localism shapes resource-use solutions; I've incorporated that into some work I'm doing … I've also adapted some of the debate on how best to wash up in a task for secondary school students. (staff feedback questionnaire) The Professional Communication Skills module has traditionally centred on teaching undergraduate student the knowledge, skills and values necessary to develop and deliver effective technical and persuasive presentations, within their area of study. Since I started teaching the module, I have introduced sustainable development and sustainability communication as priority contexts in which student presentations should be developed, presented and also assessed. (facilitator) The impact upon teaching practice could be seen first, in the greater take-up of technology-enabled learning in other contexts, and, secondly, in the specific adaption and reuse of course resources and exercises for other modules.

Impact of different cultural backgrounds of participants, tutors and facilitators on the overall SD learning experience in the NOOC/MOOC
The course consciously explored different worldviews and how these affect our understanding of sustainability: for example, in a debate about anthropocentric and ecocentric worldviews. In versions 1 and 2 of the course this was discussed fairly uncritically with students willing to categorise themselves as one or the other. In the 3rd, MOOC phase, the diversity of learners encouraged a more critical debate with comments about the limitations of these categories, the spaces in between, other worldviews and predominance of Western-centric thinking when it comes to sustainability. Our use of social media expanded as we looked for ways to show this much larger group of learners that we were responsive to their needs. We introduced 'Wordles' as a quick and easy way of summarizing discussion threads, and we established 'Scoop-It' and 'Delicious' bookmarking sites to enable easy access to learner blogs and suggested resources. (convenor) The overt course design of the NOOC and MOOC increased awareness of planning, executing and assessing teaching and learning, monitoring student participation and learning engagement in online courses. Experience of this was seen as highly valuable: I am in the midst of developing an interdisciplinary sustainability elective module, incorporating flipped, blended learning, hangout sessions and webinars with counterparts in the UK and Australia. I have my NOOC and MOOC facilitating experience to thank for the confidence to develop this elective. (facilitator) Levels of student engagement varied but this was recognised as a positive dimension of the online format: my involvement taught me that NOOCs and MOOCs provide an opportunity for a cohort of students to develop a unique set of student-generated learning resources, working together on a learning journey to create something that is owned by the whole group. It also taught me about the diversity of learning styles ('lurkers' etc.) and how these can all be accommodated within an online framework. (reflective blog written by a course tutor) To deal with such large numbers, the team had to focus upon developing learner autonomy and critical thinking; students were guided into collaborative learning with and from their peers. Throughout it, there was an emphasis upon learning from each other which helped students to understand the value of peer review and peer assessment.

Notions of change within teaching practice as a result of the involvement in the MOOC/other sustainability projects
Team members commented that their involvement in the course had impacted upon their teaching practice. This included the ability and confidence to make more effective use of the university's virtual learning environment (Moodle), using it to structure student learning in between face to face sessions: I'm quite a traditional 'chalk and talk' lecturer in many ways, although I do try to liven up lectures with stories, drama and so on, but that's another tale. The MOOC experience has made me think about teaching outside of the lecture theatre more, perhaps using Moodle, but I do struggle to fully utilise the online tools that we have available, mostly due to lack of time to develop courses and unfamiliarity with the 'bells and whistles' that could be put to greater effect. (tutor) I am setting up discussion fora, Q + A, wikis etc and requiring student to complete certain tasks online the impact upon the course team of the experience of working, learning and interacting online with a diverse community of learners in a MOOC-style course designed to explore sustainable development in an interdisciplinary context, with a particular focus on engineering. It is perhaps surprising that the effects on the course tutors and facilitators have arguably been more pronounced and certainly more memorable than the effects of faceto-face interactions in more traditional teaching contexts such as physical classroom spaces.
We suspect that the combination of the topic (sustainability being so more than just a dry theoretical subject) and the cultural diversity of the participants had much to do with this, allowing as it did, a rich sharing of knowledge and experiences. It was also a change from normal practice to work together in a large enthusiastic team to develop 'flipped classroom' course materials and to deliver these jointly rather than just being a solitary 'sage on the stage' . Blurring the lines between personal and professional experience challenged the dualistic approach to work and life often expressed in Western cultures, and encouraged the incorporation of different worldviews and attitudes towards sustainability. It is indeed rare that delivering a course challenges the teachers' assumptions as much, if not more so than it does the students. Note 1. Available as free downloads from http://www. nottingham.ac.uk/open/ebooksandibooks.aspx

Disclosure statement
No potential conflict of interest was reported by the authors. Michael J. Clifford, PhD, is an associate professor at the University of Nottingham's Faculty of Engineering. His research fits within themes such as appropriate technology, composites processing, dynamical systems, fluid mixing and engineering education for sustainable development. He has published extensively in these areas. He was a tutor on the It was clear from the MOOC, with its older learner demographic from many different countries, that attitudes towards sustainability and understanding of it were culturally specific. Members of the team found themselves challenged to rethink their own assumptions about where the power lies to find solutions: Perhaps sustainability is viewed as more affordable in the West, but even here there is a perception that things like organic vegetables are only affordable for the middle classes. Participants from poorer countries seem to demonstrate more resilience to things like lack of electricity, clean water and so on -the things we tend to take for granted. I think there's also a lot more creativity and 'survival instinct' demonstrated in coming up with simple but effective solutions, such as using plastic bottles for skylights and so on. (tutor) For the non-Western facilitator, while negotiating the Western-non-western perspectives debated and discussed was fairly manageable, there was recognition that the scope to explore alternative worldviews was under-developed:

Notes on contributors
Much of the cultures and religious practices of the non-western participants and the indigenous communities present within the non-western countries advocate and promote ecologically sustainable values and practices. Interestingly enough, not much discussions were centered around this on all versions of the module. (facilitator) This indicated one of the weaknesses of the course in that its construction and delivery was Western-centric and that, despite the presence of large numbers of Asian students in the early versions and the globally diverse group of learners on the MOOC versions, this perspective was strongly entrenched. In the most recent version of the course (November-December 2015), we tried to address this by reviewing language and content both for inclusivity and to openly acknowledge the predominant perspective of the course creation team. We also introduced a new section on appropriate technologies. Content will be reviewed again before the next run, which is scheduled for academic year 2016-17.

Conclusion
From earlier research (Speight and Bowman 2015), we could see that the open-access study materials in question -the NOOC, 'Perspectives on Sustainability' and the MOOC 'Sustainability, Society and You' , enabled students to undergo paradigm shifts in their understanding, moving beyond sustainability literacy to bring about behaviour changes in their personal and professional practices. While many other studies have investigated the efficacy and impact of online learning on students, few have considered the effects of creating and sustaining an online course upon tutors and facilitators.
In this study, we have employed a series of mixed methods (and rather informal) methodologies to explore