Hélène Seiler ✉ (Helsei Consulting)
Formative feedback is likely to improve performance, which has encouraged executive coaches to seek accreditation and supervision. However, many coaches do not consider their clients as suitable providers of formative feedback, due, in part, to a lack of shared knowledge about effective behaviours. The study addressed the issue by developing a client behavioural feedback instrument for the executive coach. The article summarises the key findings of the mixed-methods approach which informed the instrument, highlighting differences and similarities with scales developed by accrediting bodies and experts.
executive coaching, client feedback, coach professional development
Accepted for publication: 01 May 2019 Published online: 31 May 2019
© the Author(s) Published by Oxford Brookes University
While executive coaches routinely use formative feedback for the purpose of developing their clients, few ask them to reciprocate. This is paradoxical since clients spend the most time observing executive coaches in practice and are used to giving formative feedback in the workplace (Millward, Asumeng & McDowall, 2010). Arguably, empirical research about client formative feedback in executive coaching is severely limited. Most coaching accreditation bodies include the competency of seeking client feedback in their models, but stop short of conceptualising the phenomenon. On the practitioner’s side, a survey of the members of the European Mentoring and Coaching Council (EMCC) suggest that 40% of European coaches obtain summative feedback from their clients (Passmore, Brown, Wall, Stokes et al., 2018) but does not mention whether they used client formative feedback for the purpose of developing themselves.
Clients interviewed in qualitative research tend to report coaching outcomes rather than the coach’s actions that led to such outcomes, thus showing a clear preference for summative rather than formative feedback (De Haan, Duckworth, Birch & Jones, 2013; Myers, 2014). The phenomenon may be linked to a pervasive and dominant discourse in which the coach is presented as an expert, and the client, conveniently called the ‘coachee’, becomes the passive recipient of such expertise (Stokes, 2015). Such experts’ discourse has led to building competency models and evaluative tools for executive coaches that do not rely on client data (Blumberg, 2014).
The consequences of not equipping the client to give formative feedback to their executive coach have not been explored in research. Yet, not only may it prevent the coach from accessing useful developmental information, it may also increase the passivity of clients, giving too much responsibility to the coach in the achievement of coaching outcomes (Welman & Bachkirova, 2010). In addition, the lack of shared knowledge about effective executive coaching behaviours between coach and client may explain why there exist discrepancies between clients’ assessments and coaches’ assessments of the coaching process in empirical research (Myers, 2014; De Haan & Nieß, 2015; Linder-Pelz & Lawley, 2016). Such difference, in turn, is likely to feed the self-deception of the coach (Bachkirova, 2015), leading to increasing the discrepancy of perception even further (De Haan & Nilsson, 2017).
If the lack of transparency perpetuates, the coaching process is at risk of becoming less effective and less credible: as Mulvie (2015) summarised, if no-one is in a position of challenging the coach’s position, then the myth and magic of coaching may perpetuate.
To equip clients with adequate knowledge, the study’s aim was to develop a client behavioural feedback instrument for the purpose of providing formative data for the coach. The paper summarises the literature related to the development of such an instrument and presents the methodology that was chosen to build it. Then, it reports key findings about the content of the instrument and discusses its implications for practitioners.
A review of the literature surfaced several knowledge gaps preventing the development of a client feedback instrument supporting the professional development of the coach. Faced with a lack of theory of client feedback in executive coaching, the first objective of the literature review was to cross reference theories of feedback and coaching to find out where they would overlap. Theoretical investigations of feedback interventions (Kluger & Denisi, 1996) and of feedback seeking behaviours (Anseel, Frederik, Beatty, Shen, Lievens & Sackett, 2015) suggest that feedback is likely to be linked to job performance if focused on behaviours, but that its effectiveness is moderated by the characteristics of both the feedback giver and recipient. Notably, if the feedback giver is perceived to have adequate knowledge about the job of the feedback recipient, the intervention is more likely to trigger change for the feedback recipient (Jawahar, 2010).
While behavioural feedback has been critiqued as reductionist (Jackson et al., 2012), a mixed-methods approach, based on the Cognitive Affective Personal System Theory (Mischel & Shoda, 1995) favours the observation of behavioural dimensions while acknowledging that they are situational. The process is based on repeated observations over a range of critical situations typically faced by the individual being assessed (Lievens & Christiansen, 2012). Arguably, research in the root disciplines of coaching such as education or sports coaching provides a strong rationale for considering the coaching conversation as a specific situation during which behavioural observations can be fed back by the client to the coach (Gaertner, 2014; Fletcher & Roberts, 2013), thus triggering a feedback loop (Kolb, 1984). As a result, the mixed-methods approach was selected to operationalise a client feedback intervention.
On such basis, the review of the literature surfaced that a theoretical framework that I described as “client-centred integrative” might support a mixed-methods approach to client feedback. It revolves around the client-centred theory which places the coach and client on equal footing and defines the coaching conversation as a reflexive dialogue between the client and the coach for the purpose of supporting the client’s learning process (Cox, 2013). Drake (2011) and Lane (2016) have acknowledged that, in order for the coach to meet the client’s needs, the choices they make need to be productively discussed with their clients; and that the evidence linking what the coach does to what the client experiences may support the process. This opens the door to viewing the client-centred theory as an integrative framework, within which evidence is provided by both the expertise theory, which supports the development of competency scales (Kilburg, 2016) and the evidence-based theory, which provides summative scales that can be used to assess whether the competency scales are related to client outcomes (Grant, 2016). Finally, the coach-developmental theory (Bachkirova & Lawton-Smith, 2015) provides the framework to include client formative feedback data in the coach self-reflective process.
The literature review indicated that current formative scales were inadequate to support a client feedback process because they have not involved clients in their construction. In addition, it revealed that current knowledge about executive coaching behaviours was highly fragmented (Blumberg, 2014), and based on competency models that were largely a-theoretical and insufficiently validated (Bachkirova, 2016).
In contrast, many summative scales are empirically validated, based on a model of evaluation of workplace training interventions (Kirkpatrick, 1977) adapted for coaching (Ely, Boyce, Nelson, Zaccaro, Hernez-Broome and Whyman, 2010). The model includes four categories of measures: reaction or overall satisfaction with the coaching process, learning and cognitive change, behavioural change and organisational change. The evidence-based theory has also supported the study of important moderators of the coaching outcome, including client characteristics. However, these models have rarely been used to assess the effectiveness of coaching scales (Grover & Furnham, 2016).
In conclusion, the literature review allowed to select a suitable theoretical framework for the purpose of developing of a client behavioural feedback instrument. However, in the absence of an adequate scale of coaching behaviours, it also indicated the need for an exploratory approach to build such an instrument.
The research paradigm underpinning the study was pragmatism, an epistemology which is well suited to address a practitioner’s issue (Fishman, 1999). Feedback was operationalised as a mixed-methods approach (Lievens & Christiansen, 2012), grounded the Cognitive Affective Personal System Theory (Mischel & Shoda, 1985). The development of the instrument was framed in a pragmatic theory which I described as client-centred integrative.
To address knowledge gaps about effective executive coaching behaviours, I used a sequential exploratory, mixed-methods design (Creswell, 2010).
The first, qualitative strand of the research addressed the lack of involvement of clients in the development of coaching scales. It took place between March and May 2017 and involved five focus groups of experienced clients of executive coaching (N=24). Participants were tasked with reviewing and reducing a list of effective coaching behaviours which I had compiled from a review of existing formative scales. To achieve this objective, the focus groups used a method adapted from task analysis: working in a sequential manner, they made changes to the compilation and ranked its components using two commonly used criteria: importance and difficulty (Cadle, 2012). Importance was defined in relation to coaching outcomes such as the trust in the coach, the generation of new insights and the attainment of goals, which are commonly used in summative research (Grover & Furnham, 2016). Difficulty characterised behaviours requiring specific coaching training in order to achieve mastery. The task analysis process ended once saturation was achieved, at the end of the fifth focus group, resulting in a scale of effective coaching behaviours.
The second, quantitative strand of the research assessed the quality of the scale of effective coaching behaviours. It took place between April 2017 and January 2018. I surveyed 107 executives undergoing a 3-4-month coaching programme to ask to what extent they had observed the coaching behaviours contained in the scale. Based on the responses, to assess the construct validity of the scale, I used a statistical method called principal component analysis, a procedure which further reduces the scale and surfaces components clearly interpretable in relation to the executive coaching process, thus leading to a feedback instrument. Subsequently, the reliability of the instrument was assessed by measuring its internal consistency, using the Cronbach Alpha, a coefficient which measures to what extent the items contained in the scale are related to each other.
In addition, the survey contained questions asking participants to what extent they had experienced important coaching outcomes at the end of the coaching process. The answers were used to assess the criterion validity of the instrument, using a model of hypotheses described in Figure 1 below.
Three summative coaching scales selected from the literature review were used to measure coaching outcomes: the Working Alliance Inventory (WAI) (Corbière, Bisson, Lauzon & Ricard, 2006) (hypothesis 1a), the Serendipity Quotient (SQ) (McCay‐Peet & Toms, 2011) (hypothesis 1b) and a goal attainment measure (hypothesis 1c).
The WAI measures the strength of the working alliance between the client and the coach, based on an estimation of their level of agreement and bonding during the coaching process. The summative coaching literature uses it as a measure of satisfaction with the coaching process. The SQ measure the emergence of serendipitous connections in the brain, a precursor of creativity. It has been used to measure to measure cognitive change as a result of a learning process. The goal attainment measure was chosen to measure behavioural change resulting from the coaching process.
Mediation effects have been largely reported in the summative coaching literature (Grover and Furnham, 2016). They occur when a particular outcome of the coaching process may trigger for one or more other outcomes. In particular, several studies indicate that the strength of the working alliance between the client and the coach mediates the relationship between what the coach does and the extent of cognitive change experienced by the client. To attempt to replicate such finding, the WAI was hypothesised to mediate the relationship between the executive coaching scale and the SQ (hypothesis 2a). Similarly, the literature has discussed the mediating effects of the client cognitive change on the relationship between the strength of the relationship between the client and the coach and the client behavioural change. To attempt to replicate such finding, the SQ was hypothesised to mediate the relationship between the WAI and the goal attainment measure (hypothesis 2b).
The third set of hypotheses attempted to replicate literature findings indicating that client characteristics may accelerate or hamper the outcome of the coaching process, thus acting as a moderator of coaching effectiveness. To study the phenomenon, I selected the Self-reflection and Insight Scale (SRIS) of the client (Grant, Franklin & Langford, 2002) at the start of the coaching process. This scale measures aspects of the emotional intelligence of the client. It was hypothesised to moderate the relationship between the coaching and each coaching outcome measure (hypotheses 3a, 3b and 3c).
The research has led to the development of a feedback instrument called the Executive Coaching Behaviour Observation Scale (EXCBOS). Its content, constructs and relationships with summative coaching scale will now be presented.
Figure 2 presents an overview of the changes and decisions made by the focus groups as they reviewed and reduced the compilation of existing formative scales that I had proposed as a starting point. Since these existing scales were largely produced by experts and coaches, the study provided additional evidence that clients have a unique perspective about the role of the coach during the coaching process.
In similarity with coaching scales developed by coaches and experts, the executives participating in the focus groups acknowledged the importance of both compassionate and challenging behaviours, and kept most of them in the scale of effective coaching behaviours. Notably:
However, significant differences were surfaced.
The principal component analysis further reduced the scale of effective coaching behaviours leading to the EXCBOS. Its behavioural items loaded on two clearly interpretable components. As shown in Figure 3, the EXCBOS described executive coaching as a professional transformational learning process, including component 1, describing professional transformational learning actions (Kets De Vries, 2013; Sammut, 2014; Moons, 2015; Theeboom, Van Vianen & Beersma, 2017) and component 2, describing empathic behaviours (Rogers, 1957).
The first component of the EXCBOS represented actions taken by the coach in support of a professional transformational learning process, clearly delineating four non-linear phases of a process described in adult learning (Mezirow, 1990) and conceptualised in coaching (Theeboom, Van Vianen & Beersma, 2017): preparation, creativity, goal setting and maintenance of trust.
The preparatory behaviours included in the EXCBOS (ex: “asked questions about my organisation to better understand the issues I presented”) were considered by Dewey (1910) as the basis of knowledge acquisition for the purpose of getting things done and have been investigated as a key anchor to the adult learning process (Kolb, 1984).
The creativity phase described a co-creative process during which the reflective dialogue is used to generate serendipitous connections between thoughts, evocative of an extended cognition (Cox, 2015). Kets de Vries (2013) argued that creativity comprises two distinct processes: illumination (where the client is in self-reflecting mode) and verification (where client and coaches are in active discussion and both contribute substantively), both of which were represented in the EXCBOS (ex: “when I requested advice, checked first if this is what I really needed and then invited me to reflect on my request”).
Because of the future orientation of coaching (Clutterbuck & Spence, 2016), goal setting is considered an important part of the coaching process. In fact, most coach competency models feature goal-setting skills prominently (Bartlett II et al., 2014) and most coaches report using goal-setting and monitoring behaviours liberally (Vandaveer et al., 2016). However, in the EXCBOS, goal setting behaviours were represented by a very limited number of behaviours focused on ensuring alignment between the individual and the organisation (ex: “invited me to reflect whether my organisation’s culture enables or hampers my development goals”).
In addition, the first component of the EXCBOS contained behaviours related to the maintenance of trust during a coaching process (ex: “when unable to provide expertise, acknowledged it”), so that ruptures which happen often during a transformational learning process can be managed (Moons, 2015).
Empathy was conceptualised by Rogers (1957), in helping disciplines, as the understanding of how clients perceive the world around them and as the ability to communicate it back to them in an appreciative manner. Bachelor (1988, p. 232) distinguished cognitive empathy (which refers to the understanding of how a person feels and what they might be thinking), affective empathy (experiencing the same feeling), sharing empathy (communicating back) and nurturing empathy (manifesting a “sustained, attentive, caring” presence). In the EXCBOS, cognitive, sharing and nurturing empathic behaviours were represented (ex: “stayed non-judgemental”). Affective empathy was not.
During the qualitative strand, the focus groups had discussed how they linked executive coaching behaviours and outcomes in order to rank them by importance. They considered that the sharing and nurturing empathetic behaviours that they selected for the scale were uniquely linked to trust in the coach. Other behaviours included in the scale such as providing career advice, manifesting cognitive empathy or ensuring alignment between the individual and the organisation were linked to both trust in the coach and new insights for the client. In contrast, balancing inquiry and advocacy, seeing patterns and providing challenge were uniquely linked to generating new insights.
Echoing these results, the quantitative analyses indicated that the EXCBOS was a significant predictor of the Working Alliance Inventory and the Serendipity Quotient. Additionally, the Working Alliance Inventory partially mediated the relationship between the EXCBOS and the Serendipity Quotient. While the two components of the instrument had a unique significant contribution to the variance of the Working Alliance Inventory, only the first component, professional transformational learning, had a unique significant contribution to the variance of the Serendipity Quotient. In contrast, while significant differences existed in the scoring of the EXCBOS between respondents who had reported a goal and those of had not reported a goal, the instrument was not a significant predictor of the variance of the goal attainment measure. In addition, the Self-reflection and Insight score of participants at the beginning of the coaching process did not moderate the relationship between the Executive Coaching Behaviour Scale and any of the coaching outcome scales, indicating that the level of emotional intelligence of the client at the start of the process may not have impacted the outcome of the coaching process in this sample.
Involving clients in the development of a feedback instrument for the executive coach surfaced an expectation of professional transformational learning actions and empathic behaviours. In line with the client-centric theoretical framework of the research, the results encourage executive coaches to partner with their clients in designing the coaching conversation, in contrast to a prevalent expert’s discourse in the coaching community which views clients as the passive recipients of the professional expertise of the coach (Stokes, 2015).
In addition, the results of the study suggest that clients expect their executive coach to contribute to the substance of the coaching conversation. Indeed, clients expressed interest in engaging with their executive coach in preparatory behaviours, in particular to help them manage the polarity between individual and organisational needs. They also expected their coach to deploy informing behaviours during the creativity phase of the coaching process to foster their own creativity.
In contrast, participants of the focus groups discarded many goal setting behaviours from the executive coaching scale. In light of the fact that the EXCBOS was not related the achievement of goals, it raises multiple questions about the use of goal attainment as a measure of cognitive change, and more generally about the nature of goals in coaching. As Clutterbuck and Spence have wondered (2016), is the executive coach expected to set and monitor detailed behavioural goals or does the role consist of supporting the generation of “mid-level constructs”?
Both components of the EXCBOS were related to the strength of the working alliance. It suggests that executive coaches used both nurturing / sharing strategies (to increase likeability) and cognitive strategies (to increase credibility) to generate bonding and agreement. In contrast, only the first component, professional transformational learning, was related to the Serendipity Quotient. This indicates that empathy alone may not be sufficient to trigger new insights, challenging a view that the coach need not possess business or organisational knowledge to be effective (Bono, Purvanova, Towler & Peterson, 2009; Grant, 2016).
The reliability of the EXCBOS was solely assessed through internal consistency: it would benefit from being further analysed through confirmatory research in an attempt to replicate the results of the principal component analysis. Its construct validity also needs to be further analysed because the sample of participants was at the lower range of what is deemed acceptable to perform a principal component analysis (Tabachnik & Fidell, 2013). At the same time a functionalist approach to test design states that each test is “specific to a particular situation and cannot be generalised” (Rust & Golumbok, p. 38). In summary, in its present form, the EXCBOS may be best used as a dialogic guide during or after a coaching process. It is not suitable for the purpose of normatively assessing coaches or comparing their ability.
After decades of a prevalent expert’s discourse (Stokes, 2015), it is going to take time for executive coaches to trust that client behavioural feedback can support their development. The literature review suggested that inexperienced clients neither feel confident nor credible and prefer not to report their behavioural observations. Since one of the pillars of trust is perceived credibility (Atkinson & Butcher, 2003) and since perceived credibility is a moderator of the effectiveness of feedback (Jawahar, 2010), an executive coach may need to ensure as a first step that a new client feels knowledgeable enough about the process of coaching before requesting formative feedback. This indicates that further research is needed to design ways to prepare clients ahead of the coaching process on how to use the instrument. For example, if coaching is embedded in a leadership development programme, a module related to effective executive coaching behaviours could be included.
Even when coaching more informed clients, their formative feedback might still be difficult to accept and integrate for coaches. Bachkirova (2015) suggests that the fear of being rejected by the client is a major trigger of self-deception for the coach. She notes that the fear of rejection is amplified by the expert’s discourse, in which coaches and clients alike believe that the coach is solely responsible to choose the ‘right’ technique and to drive the success of the intervention. Once self-deception is in place, it feeds the tendency to block new information, initiating a vicious cycle (Ditto & Lopez, 1992). While supervision might be a way to counter a fear of rejection, Bachkirova (2015, p. 5) remarked that if the coach is in the grip of self-deception, “the relevant material may not reach supervision at all”. As a result, further research should investigate whether coaches might benefit from bringing their personal feelings about client formative feedback to supervision. By addressing their own resistance ahead of the process, they may improve their listening and integrate the information received from their clients in a better way. Indeed, longitudinal studies of student feedback in education and sports coaching indicate that the self-evaluations made by teachers and the evaluations provided by students converge overtime (Gaertner, 2014).
The coach developmental theory has suggested that formative feedback data might become less important as the coach matures professionally (Clutterbuck, 2010). At the same time, it may still be useful to experienced coaches if they believe it can contribute to the process of deliberate practice. Such process has been conceptualised by Ericsson (2006) and has found successful applications in psychotherapy (Chow, Miller, Seidel, Kane, Thornton et al., 2015). Ericsson rejected the claim that mastery exists in unconscious performance. On the contrary, masters need to regain control over the most difficult aspect of their practices and identify specific behaviours that they can improve on, by actively soliciting behavioural feedback from those who benefit from their craft.
In summary, it is hoped that executive coaches will be encouraged to embrace their clients as a major source of formative feedback regardless of their level of experience. Interestingly, the quantitative strand of the study indicated that participating executive coaches, who were all experienced, generally received higher scores on empathic behaviours than on professional transformational learning actions. Perhaps this is an invitation for executive coaches to check with their clients whether they are focusing on the professional transformational learning behaviours which may be needed to achieve not just satisfaction but also personal striving.
Hélène Seiler is an executive coach and supervisor with over 25 years of experience in leadership development and talent management. She has been coaching and training executives since 2001. Hélène is bilingual English / French and has worked and lived in North-America, Western Europe and South-East Asia. Hélène has held the highest certification level of Master Certified Coach with the International Coach Federation since 2007 and is a Board-Certified Coach with the Centre for Credentialing and Education. She holds a Master of Science in Management with HEC School of Management, a Graduate Diploma of Ontological Coaching with Newfield Institute and a Diploma in Coaching Supervision with Coaching Supervision Academy. She was awarded a Doctoral degree in Coaching from Oxford Brookes University in December 2018.