International Journal of Evidence Based Coaching and Mentoring
2024, Vol. 22(2)

Reflections from the Field

Evaluation of a MOOC in Coaching for Learner-Centred Conversations

Arti Maini (Medical Education Innovation and Research Centre, Department of Primary Care and Public Health, Imperial College London)
Zoe Moula (Medical Education Innovation and Research Centre, Department of Primary Care and Public Health, Imperial College London)
Celine Esuruoso (Medical Education Innovation and Research Centre, Department of Primary Care and Public Health, Imperial College London)
Victoria Collin (Medical Education Innovation and Research Centre, Department of Primary Care and Public Health, Imperial College London)

Introduction

Over the last few decades, the field of education has experienced a paradigm shift towards learner-centred educational approaches, drawing on adult learning concepts such as self-directed learning and experiential learning, with students taking on increasing responsibility for learning (Colet, 2017). The field of coaching also draws heavily on adult learning concepts (Cox, 2006) and is based on the premise that people are resourceful and know themselves the best. The coach’s role is to support their clients to identify and work towards goals that are important to them. Training both educators and learners in person-centred, solution-orientated coaching approaches may be beneficial to support development of a learner-centred mindset.

Achieving this training at scale however presents challenges in terms of the resources required, including financial commitment by the institution and the time investment required by the institution as well as by training participants themselves (Harding et al, 2018). We therefore wanted to explore whether a learner-centred mindset could be developed through coaching skills training via an online learning platform that is potentially more cost-effective and less resource-intensive than traditional face to face methods, and increasingly accessible to people in this age of technology.

Massive Open Online Courses (MOOCs) are accessible on a large scale globally (Daniel, 2012), reaching people with diverse disciplinary and cultural backgrounds (UK Dept for BIS, 2013). MOOCs typically utilise a range of learning formats to engage learners, including videos, articles, interactive quizzes and self-assessments (Hendricks, 2019).

Research on MOOCs is growing in parallel with their increased popularity. Previous evaluation studies of MOOCs in relation to learners’ knowledge, confidence and application of learning in practice have been promising (Hossain, 2015; Goldberg 2015; Sneddon 2018; Berman, 2017). However, further evaluation and research is needed to explore the degree to which MOOC completion influences learners’ subsequent practice (Foley et al, 2019) and approaches to improve MOOC design for culturally diverse participants (Zhu et al, 2021).

Imperial College London developed a globally accessible Massive Open Online Course (MOOC) in coaching skills for learner-centred conversations in partnership with the global learning platform EdX. The MOOC aims to promote exploration and application of coaching approaches in educational contexts and facilitate development of a learner-centred mindset.

The MOOC consists of four modules each of 1-2 hours duration, covering the key principles of coaching approaches in education, creating the conditions needed for an effective learning relationship, applying coaching approaches to conversations with learners and using coaching approaches in feedback conversations. The course includes a range of learning activities and approaches including instructor-led videos, role-play videos, asynchronous discussion forums and interactive quizzes.

The design of the MOOC format and activities were informed by constructivist design principles and Self-Determination Theory, with a view to maximising intrinsic motivation and engagement of participants. Constructivism theory posits that learners construct new understandings and knowledge, integrating with what they already know (Piaget, 1971). Self-determination theory (Deci & Ryan, 1985) suggests that intrinsic motivation is increased when a person’s needs of autonomy, competence and relatedness are fulfilled, and we aimed to build in opportunities addressing these needs throughout the MOOC.

This paper describes our evaluation of the MOOC where we aim to explore the following questions:

  1. Does the MOOC effectively support participants to develop a learner-centred mindset, and if so, how?
  2. Is the MOOC experienced by participants as culturally inclusive?
  3. Is the MOOC perceived by participants as relevant and useful to a range of education contexts, and if so, how?

Methods

Approach to evaluation

A mixed methodology (post course survey and semi-structured interviews) was used to evaluate the impact of the MOOC and the experience of undertaking it.

We considered the Kirkpatrick model (Kirkpatrick, 1959) an appropriate tool to guide the evaluation of the current study. We selected three levels from the Kirkpatrick model to evaluate the MOOC, namely reaction, learning and behaviour.

Participant recruitment

All learners who enrolled in the MOOC (via the EdX platform) and who were aged 18 or over were invited through the EdX platform to take part in the MOOC evaluation study and were provided with participant information via the MOOC platform. Informed consent was sought from those who choose to participate in the study (indicated through an online checkbox). Pre-course and post-course surveys were accessible through the MOOC which directed participants to an online Imperial-hosted Qualtrics survey. We also sought expressions of interest from course participants to be interviewed and sought their informed consent (again through an online checkbox) for the research team to contact them by email with further information if they were happy to be interviewed. A subset of these participants who expressed interest were then invited to take part in a semi-structured interview to explore in depth whether and how the MOOC impacted their learning and behaviour in practice after completing the course. Purposive sampling was used to recruit and interview 16 participants comprising different genders, age groups, professional backgrounds and from a range of countries to maximise diversity.

Pre-MOOC survey

Prior to starting the MOOC, participants were asked to complete a voluntary questionnaire on Qualtrics. Questions pertaining to demographics, including gender, age, ethnicity, country of citizenship were asked, as were questions relating to their prior MOOC experience and motivations for taking the course.

Post-MOOC survey

On completion of the MOOC, participants were again asked to complete a voluntary survey which asked for their feedback of the course. Respondents were asked to evaluate aspects of the course such as overall learning experience, clarity of instructions, relevance of tasks, feedback from the tasks, user friendliness of digital platform, clarity of course content and engagement with other learners on a Likert-type scale (1 poor- 5 excellent) as well as through free-text comments. In addition, respondents were asked to complete a number of validated measures (described below).

Intrinsic Motivation

The Intrinsic Motivation Inventory (IMI) (Plant & Ryan, 1985) was utilised to assess aspects of intrinsic motivation. Six subscales consisting of two questions each were selected relating to the constructs of interest and enjoyment, perceived competence, pressure or tension, perceived choice and value and usefulness. The wording of the items was slightly modified to better fit the MOOC context as per Martin, Kelly and Terry (2018). Respondents were asked to indicate their level of agreement on a Likert scale (1-3 not at all true, 3-5 somewhat true and 5-7 very true), with higher scores indicating greater agreement. The mean Cronbach’s alpha for the subscales was 0.75, showing an acceptable level of internal consistency.

Critical Reflection/Transformative Learning

Four items from the Reflection Questionnaire (Kember et al., 2000) were taken to measure critical reflection. Critical reflection refers to a higher level of thinking and a shift of perspective or transformative learning (Mezirow, 1998). Respondents were asked to rate on a Likert scale their level of agreement (1 disagree- 5 agree) to 4 statements relating to critical reflection such as “this course has challenged some of my firmly held ideas.” A higher score indicates a higher level of transformative learning. The Cronbach’s alpha for internal consistency was deemed acceptable at 0.89.

Cultural Inclusivity

To assess how well the MOOC catered to a culturally diverse group of learners, respondents were asked to evaluate the MOOC in relation to cultural inclusivity by indicating on a 10-point scale (1 to very poor- 10 very good) (Taheri, Hölzle & Meinel, 2020). Two additional open questions were also asked to identify aspects that respondents did not feel aligned with their cultural beliefs or understanding.

Data Analysis

Items were reverse coded as necessary prior to mean scores being calculated for all scales and subscales, which were then used in the analysis. Data analysis was conducted using SPSS version 27 (IBM). Descriptive statistics are provided.

Content analysis of free text comments relating to cultural inclusivity questions

Two questions relating to cultural inclusivity were included in the survey:

  • Are there aspects of the course that you felt didn’t align with your own cultural beliefs/understandings?
  • Are there aspects of the course that could be changed to better align with your own cultural beliefs/understandings?

The data were cleaned prior to analysis, with null statements such as N/A, none etc being removed. Two researchers (LS and AC) then undertook a conventional content analysis owing to a lack of theoretical framework established from the existing literature. Due to the overlap in the questions, they were combined for analysis. LS coded the statements using an inductive approach and developed a coding scheme which was used by a second researcher (AC) to independently recode the data. Through discussion with each other and the wider team, these codes were then re-examined and revised before being sorted into categories.

Semi-structured interviews

Semi-structured interviews were conducted approximately 3-6 months post-MOOC completion to explore in-depth the impact of the MOOC on participants’ learning and behaviour relating to coaching skills for learner-centred conversations as well as identifying factors perceived as barriers or facilitators to learner-centred conversations in educational settings. Development of the interview questions was informed by constructivist theory. Interviews lasted up to 1 hour and were carried out remotely through Zoom calls. All interviews were recorded and transcribed verbatim by an independent transcription service. Two researchers developed a coding framework and then undertook analysis of the interviews independently using the agreed upon framework. Both researchers analysed the interviews thematically, while also writing a short research ‘memo’ outlining the key themes and initial reflections on the data. This analysis and memos were then discussed with the wider authorship team and summaries were produced for each theme.

Results

From March 2020-January 2021, 9906 learners from 146 countries enrolled in the MOOC.

Pre-MOOC survey

1509 enrolled learners responded to the pre-MOOC survey. Respondents were predominantly female (64.9%) and were over 18 (97.5%). Participants held citizenship from 108 countries. Countries with the highest levels of citizenship were the United States (n=282), India (n= 148) and the United Kingdom (n= 116). Just over a third of participants (35.7%) reported having English as their first language. 21% identified as a student, 63% as an educator and 5.6% as a medical educator or medical student.

Further respondent characteristics details are provided in Tables 1 and 2.

Table 1: Demographic information for respondents

Variablen  Percentage  (%) 
Age 18 or over 1471 97.5 
Under 18 31 2.1 
Missing 7 0.5 
Gender Female 980 64.9 
Male 512 33.9 
Non-binary/gender queer 8 0.5 
Missing 9 0.6 
Ethnicity Any other ethnic group 128 8.5 
Arab 76 5.0 
Caribbean 24 1.6 
African 35 2.3 
Any other Black/African/Caribbean background 33 2.2 
Chinese 55 3.4 
Bangladeshi 5 0.3 
Pakistani 21 1.4 
Indian 161 10.7 
Any other Asian background 155 10.3 
White and Asian 52 3.4 
White and Black African 8 0.5 
White and Black Caribbean 9 0.6 
Any other Mixed/Multiple ethnic background 114 7.6 
English/Welsh/Scottish/Northern Irish/British 127 8.4 
Irish 26 1.7 
Gypsy or Irish Traveller 2 0.1 
Any other White background 428 28.4 
Missing 53 3.5 
English as first language Yes 534 35.4 
No 963 63.8 
Missing 12 0.8 
Prior MOOCs undertaken None, this is my first one 778 51.6 
Less than 5 496 32.9 
Between 5-10 157 10.4 
More than 10 70 4.6 
Missing 8 0.5 
Occupational Role Student 315 20.9 
Educator 946 37.0 
Medical Educator/Medical Student 84 5.6 

Table 2: Respondents’ country of citizenship

Country of citizenship n Percentage (%) 
Algeria 3 0.2 
Argentina 20 1.3 
Australia 22 1.5 
Austria 5 0.3 
Azerbaijan 4 0.3 
Bahamas 1 0.1 
Bahrain 1 0.1 
Bangladesh 2 0.1 
Barbados 2 0.1 
Belgium 11 0.7 
Bolivia 1 0.1 
Brazil 49 3.2 
Brunei 1 0.1 
Bulgaria 2 0.1 
Cambodia 1 0.1 
Cameroon 1 0.1 
Canada 54 3.6 
Chile 8 0.5 
China 21 1.4 
Colombia 20 1.3 
Congo (Democratic Rep) 1 0.1 
Costa Rica 4 0.3 
Croatia 1 0.1 
Cyprus 2 0.1 
Czech Republic 5 0.3 
Denmark 4 0.3 
Dominican Republic 3 0.2 
Ecuador 9 0.6 
Egypt 16 1.1 
Estonia 1 0.1 
Ethiopia 1 0.1 
Fiji 1 0.1 
Finland 3 0.2 
France 22 1.5 
Germany 37 2.5 
Ghana 1 0.1 
Greece 4 0.3 
Guatemala 2 0.1 
Haiti 4 0.3 
Honduras 1 0.1 
Hungary 3 0.2 
Iceland 26 1.7 
India 148 9.8 
Indonesia 24 1.6 
Iran 5 0.3 
Iraq 1 0.1 
Ireland (Republic) 17 1.1 
Israel 2 0.1 
Italy 19 1.3 
Japan 18 1.2 
Jordan 6 0.4 
Kazakhstan 2 0.1 
Kenya 2 0.1 
Korea South 4 0.3 
Kuwait 1 0.1 
Laos 1 0.1 
Latvia 2 0.1 
Lebanon 8 0.5 
Libya 2 0.1 
Lithuania 3 0.2 
Malaysia 20 1.3 
Mauritius 1 0.1 
Mexico 32 2.1 
Moldova 2 0.1 
Mongolia 3 0.2 
Morocco 15 1.0 
Mozambique 1 0.1 
Myanmar (Burma) 8 0.5 
Namibia 1 0.1 
Netherlands 19 1.3 
New Zealand 4 0.3 
Nicaragua 2 0.1 
Nigeria 7 0.5 
Norway 3 0.2 
Oman 2 0.1 
Pakistan 14 0.9 
Peru 8 0.5 
Philippines 84 5.6 
Poland 9 0.6 
Portugal 12 0.8 
Romania 9 0.6 
Russia Federation 14 0.9 
Rwanda 1 0.1 
Saudi Arabia 15 1.0 
Singapore 15 1.0 
Slovakia 2 0.1 
South Africa 13 0.9 
Spain 29 1.9 
Sri Lanka 1 0.1 
Sudan 2 0.1 
Suriname 3 0.2 
Sweden 6 0.4 
Switzerland 4 0.3 
Taiwan 3 0.2 
Thailand 19 1.3 
Trinidad & Tobago 3 0.2 
Tunisia 4 0.3 
Turkey 22 1.5 
Turkmenistan 1 0.1 
Uganda 3 0.2 
Ukraine 12 0.8 
United Arab Emirates 10 0.7 
United Kingdom 116 7.7 
United States 282 18.7 
Uzbekistan 4 0.3 
Venezuela 3 0.2 
Vietnam 12 0.8 
Zambia 1 0.1 
Missing 8 0.5 

Reasons for taking MOOC

Respondents were asked their motivations for undertaking the MOOC by ticking all that applied. 82.1% (n= 1239) stated that it was for professional development reasons, followed by personal development (65.9%, n=995). Over a fifth of learners stated that this was for curiosity (26.2% n= 396) and 17.7% (n=267) to support wider or formal study. 11.1% (n=167) stated it was for a change of career and 3.2% (n=48) was because they were considering applying for a degree at Imperial College London. 2.8% (n=43) opted for a ‘other’ reason. These reasons included it was mandated or advised by their work to take part, interest in or pursuing a coaching career, alleviation of boredom during Covid isolation/quarantine requirements, or to improve or enhance existing skills.

Post-MOOC Survey

1521 respondents answered at least one question on the post-MOOC survey. As the level of missing data was >5% Multiple Imputation was utilised using the fully conditional specification model with 5 iterations (for a justification for using this approach see Van Ginkel et al, 2020). Owing to minimal differences found between the original data and pooled dataset, only the original data is reported.

Evaluative Statements

Percentages of those rating aspects of the MOOC as poor, average, good and excellent are provided in table 3. Those who stated ‘don’t know’ were removed prior to analysis. As can be seen a higher percentage of respondents rated the clarity of the course content and instructions as ‘excellent’ (62.1% and 63.6% respectively). 94.7% of respondents rated the overall learning experience as either excellent (50.6%) or good (44.1%). Engagement with others and feedback from tasks were rated less favourably (33.4% and 45.1% excellent, 25.8% and 10.3% poor/average).

Table 3: Evaluative ratings

  N  Percentage  
Overall learning experience1475Poor0.7
Average4.5
Good44.1
Excellent50.6
Clarity of instructions throughout the course 1471Poor0.4
Average3.2
Good32.8
Excellent63.6
Relevance of the tasks1439Poor0.8
Average4.7
Good42.1
Excellent52.4
Feedback of the tasks  1267Poor2.8
Average7.5
Good44.6
Excellent45.1
User friendliness of digital platform  1442Poor1.4
Average0.5
Good37.6
Excellent56.0
Clarity of course content  1463Poor 0.4
Average3.6
Good33.9
Excellent62.1
Engagement with other learners  1205Poor7.4
Average18.3
Good40.8
Excellent33.4

Intrinsic Motivation Scores

Means were calculated for the six subscales used from the Intrinsic Motivation Inventory. As can be seen in Table 4, subscales relating to the courses value and usefulness and the amount of interest and enjoyment scored the highest levels of agreement (‘very true’) whilst the subscale relating to pressure and tension scored the lowest levels of agreement (‘not at all true’). The subscale of relatedness scored lower than most of the others although this was still rated as being ‘somewhat true’.

Table 4: Intrinsic Motivation Inventory: Subscale Means

 IMISubscaleNMean (S.D) 
Interest/Enjoyment 15206.02 (1.19) 
Perceived Competence 15165.82(1.16) 
Pressure/Tension 15172.35 (1.28) 
Perceived Choice 15135.95 (1.32) 
Value/Usefulness 15126.27 (1.11) 
Relatedness 15074.35 (1.86) 

Transformative Learning- Critical Reflection

Overall respondents demonstrated a high level of agreement with the statements relating to transformative learning, with 66.1% to 77.7% agreeing (definitely agree or agree with reservation) with the statements (see figure 1). An overall mean score was calculated for critical reflection from the four items, M=3.85 (S.D=1.02). 

Figure 1: Respondents level of agreement on a five-point scale assessing transformative learning

Cultural Inclusivity

Respondents rated the MOOC high for cultural inclusivity, with a mean score of 7.89 (S.D= 1.85, n=1405). Percentages for the level of agreement are provided in figure 2.

Figure 2: Respondents level of agreement on a ten-point scale assessing cultural inclusivity

Content analysis of free text responses to cultural inclusivity questions

The content analysis identified the following four categories:

  • The majority of participants felt the MOOC aligned well with their cultural beliefs
  • The MOOC was widely perceived to support development of a learner-centred mindset
  • Some participants desired greater representation of diversity within the MOOC (including cultural and ethnic diversity, mental health, disability, neurodiversity)
  • Some participants felt coaching principles may be less aligned to cultures with stricter hierarchies

Examples of responses relating to hierarchical contexts included:

“learners in my culture/ certain cultures expect to be taught more didactically/ instructively”

...in certain cultures, students want explicit guidance and instruction from their teachers. with a strong respect for hierarchy. And [the coaching] model may be not easy to apply under these circumstances.

“the coaching style may be incompatible with the culture where I come from”

Results for semi-structured interviews

Demographic data for interview participants:

16 participants in total were interviewed. These included 7 female participants, 8 male participants and 1 participant who identified as non-binary. The participants were drawn from 14 countries. The age range was from 25-66. Eleven participants identified as educators and three identified as students.

The following key themes emerged from the interview data:

  • Engagement with MOOC
  • Learning from the coaching course
  • Application of coaching approaches
  • Challenges to using coaching approaches
  • Addressing challenges
  • Meaning-making

Engagement with MOOC

This MOOC launched at the beginning of Covid-19 pandemic with the associated transition for many people to online working and learning, and some interview participants viewed this as an opportunity to learn new things whilst spending time at home. Most participants had prior interest and/or knowledge of coaching or similar areas, such as motivational interviewing and viewed this course as an opportunity to consolidate and build on prior interest, knowledge and career aspirations. This course was seen as an opportunity for continuing professional development, especially since it was free to access without geographical restrictions. As a participant commented: 

It's so difficult for us living in Argentina to have access to these new theories because, what is normal in the US, for us is like 20,000 years in the future.

Engagement was enhanced by the short length of the course. It could be completed within approximately 4 hours, which enabled some participants to even complete within a single day. Participants where English was not their first language commented that the facilitators’ clear and slow delivery of the videoed sessions kept them engaged and increased their confidence that they could complete the course. The role-play videos of conversations between tutors and learners were highly valued by most interview participants who viewed these as relevant for them when considering real world challenges of how best to support student-centred learning. This led to reflection and learning:

I could see in the roleplay… I was reflected back in the negative aspect, that sometimes I didn’t give students the time they deserve.

Learning from MOOC : Development of a coaching mindset

The greatest learning benefit according to most participants was the development of a learner-centred mindset. Participants described how the MOOC supported development of a mindset shift in them, whereby they came to view learners as resourceful and able to generate solutions to issues, and consequently the value of educators adopting a facilitative approach.

The concept of asking questions was itself very new to me, especially thinking of students or learners as a resourceful person, trusting the person, and ask questions…leading the students to find new things by themselves.

Reflecting critically on the role-play videos prompted some participants to reflect on their own experiences of teaching, learning and feedback conversations and the impact these have had on them, and encouraged them to adopt more empathetic approaches.

I've always tried to be empathetic with my students, but with this course, I thought about myself when I was a student and the feedback I got from my teachers...I felt so bad when they spoke to me like that, like the bad videos you showed...and it took me back there to think about the bad things I experienced...and of course, to think about myself now that I'm on the other side and I don't want my students to feel bad.

Participants also noticed a desire to become less judgemental and more empowering towards other people: 

…it's good to help them find the solution for themselves, not to give advice, like, oh, I think you should do this or you should do that. But to embrace them, like help them find the solution. But help them, not us, like being superior to them, like I'm here and I know more.

I’ll tell them at the end, hey, just remember, guys, when you’re in here, I did nothing, you did this. So, I would hope there would be some sense of pride or accomplishment.

Learning from MOOC: Development of a coaching skillset

Participants described gaining a range of new skills, in particular, being more aware of the impact of the language used (e.g., directive versus non-directive language), forming more open and constructive questions, and becoming active listeners. More specifically, participants reflected on previous experiences where aspects of communication, such as the tone of voice or body language, impacted on their conversations.

Following the course, participants felt better able to form open, non-directive, and constructive questions, and saw the value of this, for example: 

even if they come in defensive, I feel like I can diffuse it faster than I would have been able to before, which I think is definitely down to how my questioning has changed… and I noticed a difference in the quality of the answer I was getting.

So now [the students] are not dependent on me, [they are] more independent on themselves to answer their own questions. And that was very inspirational.

Participants also noticed that their active listening was substantially improved following the course, and they described becoming mindful of circumstances where they were not fully present, and how that had impacted their relationships and conversations. As a participant mentioned: 

I feel that I’m judging a lot less than before… my listening is more active. I’m being less judgemental… Right now, I feel that very, very, very strong…  I have memories of listening at somebody and I was not paying attention… that’s a huge change

Another theme that emerged related to the development of reflexive skills:

It's made me reflective of myself. This idea of being present…makes you really question yourself…makes you also think about the consequences of what you've just done or how that particular session has gone…it's made me think a lot more about my actions and words.

Some participants became more aware of how changing the way in which they verbally expressed things improved their interactions with other people, while also encouraging reflection and problem-solving thinking:

… even though I would tell them what was right, people would take it personally. It was too strong, like someone was telling you exactly what to do. Now I think more that I just give you an idea, and I give you a thought, and then you can choose what to do with it…. so I think just changing the way that I say things has helped me.

Application of coaching skills

Participants described applying the coaching skills they had learned within a range of contexts, including educational contexts, managerial work contexts (through improving work relationships and productivity), clinical contexts (for those working in a clinical capacity where coaching skills enabled more person-centred approaches to be adopted) and personal contexts (through improved relationships with family and friends, progress relating to their own goals and emotional regulation).

Within educational contexts, participants noted that they were engaging differently with learners and felt better able to support learners to reflect and learn. They described having more collaborative and empowering conversations and were able to experiment and tailor these conversations better to learners:

My teaching now that I look back at it changed. I no longer tell students what they need to do […] And in the first two minutes, they usually struggle as students usually struggle but then they actually came up with really good ideas.

Participants also reported feeling enthused about the value of learning coaching skills and thereby promoting it as a useful skill for others to also learn.

Challenges in using coaching approaches

A key challenge highlighted by several participants in applying coaching approaches in educational contexts related to expectations of learners due to hierarchy and cultural contexts:       

if [some students] meet somebody who’s actually saying, I’m going to talk to you on the same level…Sometimes they don’t actually want that... In their world that’s not how it works for them. And it’s not a system that they like.

Other challenges noted included financial, workload and time restrictions, the need for preparation, a desire to want to offer advice, the need to build confidence in coaching holistically and in building rapport and trust, active listening and asking coaching questions. Some commented on the challenge of convincing others of the value of a coaching approach. Many participants recognised the need for ongoing practice to maintain and further develop the skills they had learned.

Addressing challenges

Participants also described their thoughts about how the challenges could be addressed, highlighting the importance of expectation-setting, patience, generating conditions for trust to be established and awareness of their own limits:

You have to assure them that they're going to get somewhere, they're going to discover things. But in that I am not going to tell you everything. I'm not going to give you from A to Z, everything that you need to know. We'll be discovering it together.

You only need that patience to keep on pushing because you never know what will come out tomorrow…you just give them time.

Sometimes people need more space and time to show their vulnerability.

When something is beyond you and you find you’ve tried and you cannot manage, you can make a referral. 

Meaning making

Several participants described the impact on them personally of undertaking the MOOC coaching skills training course.

Some participants noted an increased self-awareness and ability to self-regulate emotions:

[Coaching] helps you connect into the things in yourself that are important…Because I’m more aware about what pushes my buttons and why, I think I’m able to self-regulate a little bit.

Participants also described a sense of satisfaction from being able to inspire students:

What makes me more happy in this journey is how I inspire my students. As I receive feedback, both from parents and students, it’s my trophy.

Participants also described their perceptions of coaching as important to teaching and supporting students navigate an increased level of challenge:

Coaching is so important to teaching now…the COVID environment…has meant that there needs to be much more outreach by teachers to students to help them through…very, very difficult situations.

Some participants described a role for coaching in promoting values-based learning:

[Geopolitically], I've never seen values-based learning as important as it is now…getting young people to really understand…their responsibilities, civic responsibilities, and their commitment to democratic principles and norms and civil behaviour…One of the [reasons] I got into teaching was to try and help develop good citizens. Now it's become urgent. Coaching has a lot to help to do that.

Discussion

This coaching MOOC was well-received by participants, with evaluation findings suggesting that the coaching skills training through an interactive online medium enabled participants to develop a more learner-centred mindset and apply coaching skills learned in practice to enhance educational interactions as well as interactions more generally.

Our evaluation findings contribute to the evidence base relating to the potential for MOOC completion to influence subsequent practice. The ability to influence practice through a MOOC format may enable training at scale, addressing some of the resource challenges identified by Harding et al (2018).

Participants scored highly on the intrinsic motivation scale, suggesting that the MOOC may have contributed to addressing participants’ needs of autonomy, competence and relatedness, thereby facilitating their self-determination and intrinsic motivation.

From a constructivist perspective (Piaget, 1971), participants constructed meaning from testing and de-constructing their pre-existing knowledge and experiences and integrating this with the new skills and ways of thinking while reflecting on and adapting this learning within their individual contexts.

Participants scored highly on the critical reflection/transformational learning scale, suggesting that the MOOC had prompted participants to reflect critically. Thematic analysis of interview data suggested that reflection appeared to be an essential part of activating prior knowledge, elaborating prior knowledge, and learning in new contexts, for example through participants reflecting critically on the role-play videos which led to a development of a greater sense of empathy and a non-judgemental and empowering approach towards learners.

Another example of this relates to applying coaching in more hierarchical educational contexts. The majority of those responding to the post-MOOC questionnaire rated the MOOC content as culturally inclusive. However, the semi-structured interview and post MOOC questionnaire data analysis highlighted that greater power imbalances between educators and learners posed both opportunities and challenges to implementing coaching approaches. For those working in these more hierarchical cultural contexts, highlighting the value of coaching approaches enhanced development of a learner-centred mindset and skillset for many participants. However, participants recognised that there is a need to balance this with the challenge of developing a culturally inclusive model of coaching accessible across different cultures, where a learner centred approach and flattened coach/ learner hierarchy may not be the norm for either coach or learner. Nagalia et al (2010) highlight how social hierarchy shapes the expectations that clients have from the coach and coaching, requiring coaches to flex their coaching style to suit the social context of their clients. In our study, some solutions articulated by participants when coaching in hierarchical contexts include clear expectation setting and recognition of the need for greater levels of patience to build trust over time.

Content analysis also highlighted the need to further develop the MOOC to be more representative of diversity, for example through increased representation of issues relating to mental health, neurodiversity, disability and cultural and ethnic diversity. The need to better reflect diversity within learning materials and approaches is increasingly recognized (Guo & Jamal, 2007, Mayhew & Grunwald, 2006). We aim to address this further as we continue to develop this MOOC.

From a constructivist position, knowledge is socially and culturally constructed and therefore, the researcher and the researched are inseparable (Mann & MacLeod, 2015). As such, the researcher influences the questions asked (and the way they are being asked) based on their previous beliefs and experiences. Power hierarchies between the researcher and the researched also influence interactions within the context of the interview (Aléx & Hammarström, 2008). In the present study, the researcher who conducted the interviews (ZM) was from a UK-based higher education institution that is internationally recognised; in fact, the Imperial branding was one of the main motivators to start this course for many participants. However, most participants were based in countries where coaching has yet to be formally developed or recognised by the community. It is possible that this power imbalance may have increased the likelihood of participants sharing aspects of their coaching experience or practice they are ‘not good at’ (as a form of inferiority), rather than the aspects that are working well. For example, one of the participants admitted that:

Before we started the conversation, I felt quite nervous to start my conversation with you. I have had insomnia for a whole night

It is therefore important to recognise that due to unequal power dynamics, the picture we have presented of how the non-directive coaching approach highlighted in the MOOC is applied internationally might not have captured adequately the aspects of coaching that are working well in other countries. It is also possible that participants tended to share things that worked well in this course, and less likely to share the parts of the course that could have been improved from their perspective.

However, our team has an interdisciplinary background which helped at the stage of data analysis to ensure that all participants’ voices were heard and considered in the interpretation of the results. The lead author (AM) is an accredited coach with international experience and expertise. ZM has a background in the arts and humanities with expertise in public health and health research (but without prior experience of coaching). CE is a newly qualified medical graduate and has trained in coaching skills.

We acknowledge the inherent bias that the lead author of this study (AM) also led the development of the MOOC. Given this conflict of interest, AM was not involved in data collection, including conducting interviews with selected participants, and only had access to anonymised data for the purpose of data analysis. There may also be a selection bias inherent in this study in that participants who choose to undertake this MOOC may already have an interest in learner-centred approaches within educational settings.

In order to ensure participants’ anonymity, we did not ask for any identifying information, and therefore did not have a way of linking their pre-course answers to their post-course answers. This therefore has implications on the generalisability of the data, as the demographics provided by participants prior to the course are not necessarily indicative from those post-course. However, scrutiny of the analytical information contained in the MOOC platform suggests that for some key demographic information at least it is comparable. Future research using a similar approach could embed a unique identifier across the pre and post-course survey, allowing for a more comprehensive analysis. Our evaluation findings suggest that the MOOC on Coaching for Learner Centred Conversations has been effective in supporting participants to develop a learner-centred mindset and skillset and is perceived by the majority of participants as culturally inclusive. Historically, the development of the field of coaching has been primarily informed by a Western world view. There is an increased diversity of educators and learners within educational institutions and the field of coaching has increasing global reach. Given this, future work should explore the implications and opportunities of applying coaching skills across diverse cultures and contexts, including what the coaching field itself can learn from successful educator-learner interactions and approaches in more hierarchical cultural contexts.

Acknowledgements

We are grateful to Dr Lara Shemtob and Dr Ali Cena who undertook the content analysis of free text comments relating to the cultural inclusivity questions, under the supervision of Dr Victoria Collin.

Disclaimer: This is an independent evaluation supported by the National Institute for Health and Care Research Applied Research Collaboration Northwest London. The views expressed in this publication are those of the authors and not necessarily those of the National Institute for Health Research or the Department of Health and Social Care.

References

Aléx, L., & Hammarström, A. (2008). Shift in power during an interview situation: methodological reflections inspired by Foucault and Bourdieu. Nursing Inquiry, 15, 169-176. DOI: 10.1111/j.1440-1800.2008.00398.x.Berman, A.H., Biguet, G., Stathakarou, N., et al. (2017). Virtual patients in a behavioral medicine massive open online course (MOOC): a qualitative and quantitative analysis of participants' perceptions. Academic Psychiatry 41:631–41. DOI: 10.1007/s40596-017-0706-4.Colet, N.M.R. (2017). From content-centred to learning-centred approaches: shifting educational paradigm in higher education. Journal of Educational Administration and History, 49 (1) , 72-86. DOI: 10.1080/00220620.2017.1252737.Cox, E. (2006). An adult learning approach to coaching. In D. Stober & A. M. Grant (Eds.) Evidence based coaching handbook: Putting best practices to work for your clients. London: Wiley 193-217Daniel, J. (2012). Making sense of MOOCs: Musings in a maze of myth, paradox and possibility. Journal of Interactive Media in Education, 3: 18. DOI: 10.5334/2012-18.Deci, E. L., & Ryan, R. M. (1985). Intrinsic Motivation and Self-Determination in Human Behavior, Springer New York, NY. DOI: 10.1007/978-1-4899-2271-7.Foley, K., Alturkistani, A., Carter, A., et al. (2019). Massive open online courses (MOOC) evaluation methods: protocol for a systematic review. JMIR Research Protocols 8:e12087. DOI: 10.2196/12087.Goldberg, L.R., Bell, E., King, C., et al. (2015). Relationship between participants’ level of education and engagement in their completion of the Understanding Dementia Massive Open Online Course. BMC Medical Education 15:60. DOI: 10.1186/s12909-015-0344-z.Guo, S., & Jamal, Z. (2007). Nurturing cultural diversity in higher education: A critical review of selected models. Canadian Journal of Higher Education, 37(3), 27-49.van Ginkel,J. R., Linting, M., Rippe, R. C. A.,& van der Voort, A.(2020). Rebutting existing misconceptions about multiple imputation as a method for handling missing data. Journal of Personality Assessment, 102 (3): 297-308. DOI: 10.1080/00223891.2018.15306.Harding, C., Sofianos, L., Box, M. (2018). Exploring the impact of coaching in higher education. Leadership Foundation for Higher Education, London. Hendriks, R.A., de Jong, P.G.M., Admiraal, W.F., et al. (2019). Teaching modes and social-epistemological dimensions in medical massive open online courses: lessons for integration in campus education. Medical Teacher 41:917–26. DOI: 10.1080/0142159X.2019.1592140.Hossain, M.S., Shofiqul, Islam M., Glinsky, J.V., et al. (2015). A massive open online course (MOOC) can be used to teach physiotherapy students about spinal cord injuries: a randomised trial. 61:21–7. DOI: 10.1016/j.jphys.2014.09.008.Kember, D., Leung, D.Y.P., Jones, A., et al., (2000). Development of a questionnaire to measure the level of reflective thinking. Assessment and Evaluation in Higher Education, 25(4), 382-395.Kirkpatrick, D. L. (1959). Techniques for Evaluation Training Programs, Journal of the American Society of Training Directors, 13, 21-26.Mann, K., & MacLeod, A. (2015). Constructivism learning theories and approaches to research. In J. Cleland, & S.J Durning (Eds.) Researching Medical Education, Wiley Blackwell, pp 49-66.Martin, N., Kelly, N., & Terry, P. (2018). A framework for self-determination in massive open online courses: Design for autonomy, competence, and relatedness. Australasian Journal of Educational Technology, 34(2), 35-55.Mayhew, M. J., & Grunwald, H. E. (2006). Factors contributing to faculty incorporation of diversity-related course content. The Journal of Higher Education, 77(1),148–168.Mezirow, J. (1998). On Critical Reflection. Adult Education Quarterly, 48(3), 185–198. DOI: 10.1177/074171369804800305.Nangalia, L., & Nangalia, A. (2010), The Coach in Asian Society: Impact of social hierarchy on the coaching relationship, International Journal of Evidence Based Coaching & Mentoring, 8(1), 51-66. Available at: https://radar.brookes.ac.uk/radar/items/cb7b963c-655d-430a-b13f-b001ed66dbdc/1/.Piaget, J. (1971). The theory of stages in cognitive development. In D. R. Green, M. P. Ford, & G. B. Flamer (Ed.s) Measurement and Piaget. McGraw-Hill, 1-11.Plant, R. W., & Ryan, R. M. (1985). Intrinsic motivation and the effects of self-consciousness, self-awareness, and ego-involvement: an investigation of internally-controlling styles, Journal of Personality, 53, 435-449. DOI: 10.1111/j.1467-6494.1985.tb00375.x.Sneddon, J., Barlow, G., Bradley, S., et al. (2018). Development and impact of a massive open online course (MOOC) for antimicrobial stewardship. Journal of Antimicrobial Chemotherapy, 73:1091–7. DOI: 10.1093/jac/dkx493.Taheri, M., Hölzle, K., & Meinel, C. (2019). Towards culturally inclusive MOOCs: A design-based approach. In H. Chad Lane, S. Zvacek., & J. Uhomoibhi (Eds.), Computer Supported Education 11th International Conference, CSEDU 2019, Heraklion, Crete, Greece, May 2-4, 2019, Revised Selected Papers, pp. 597-604. DOI: 10.5220/0007715805970604.UK Department for Business Innovation and Skills (BIS). (2013). The maturing of the MOOC: literature review of massive open online courses and other forms of online distance learning. Research paper 130. London, UK Zhu, M., Sabir, N. et al (2021). Addressing learner cultural diversity in MOOC design and delivery: Strategies and practices of experts. Turkish Online Journal of Distance Education, 22: 1-25.

About the authors

Arti Maini is the Director of Undergraduate Primary Care Education and Coaching Lead for the Medical Education Innovation and Research Centre (MEdIC), Imperial College London.

Zoe Moula is a Lecturer in Mental Health at King's College London, and a former Teaching Fellow at the Medical Education Innovation and Research Centre (MEdIC), Imperial College London. 

Celine Esuruoso is an F2 doctor and worked as a student member of the Medical Education Innovation and Research Centre (MEdIC), Imperial College London. 

Victoria Collin is currently a Research Tutor/Lecturer in Psychology at the University of East London and is a former Teaching Fellow at the Medical Education Innovation and Research Centre (MEdIC) at Imperial College London. 

Details

  • Owner: Hazel King
  • Collection: IJEBCM
  • Version: 1 (show all)
  • Status: Live
  • Views (since Sept 2022): 24