By Brook Rushing

Introduction

Companies of a variety of industries and sizes all over the world realize that to thrive in today’s complex, competitive, and volatile landscape they need strong, capable leaders.  According to several research studies conducted by Center for Creative Leadership, nearly 8 of 10 organizations expect to increase their budgets for leadership development spending with an average increase of 10% over the next year (CCL, 2019). Organizations are investing in a wide range of modalities and delivery methods including assessments and executive coaching.  CCL reported that 7% of leadership development budgets are spent on coaching and 6% of the budget is allocated to assessments.  Multisource Feedback and Executive Coaching have become increasingly popular leadership development tools.  However, despite their widespread use, there are limited studies assessing the effectiveness of this combination and of those studies, the results are insubstantial.  This literature review will summarize and outline themes from several studies that examined the effects of multisource feedback when combined with feedback facilitation or coaching.

Multisource Feedback

Multisource feedback, often referred to as 360-degree feedback, is a type of assessment in which evaluations are gathered about a target participant from multiple rating sources, including self, supervisor, peers, and direct reports (Smither et al., 2003).  The evaluation or survey usually measures work related performance or interpersonal dimensions and the evaluation can be “off the shelf” or customized by the participating organization (Passmore, 2012).  Following the initial evaluation, target participants are provided feedback.  Sometimes that feedback is only provided in the form of a report that contains data and graphics summarizing how they are perceived by various groups and/or individuals with whom they engage regularly in the workplace.  Sometimes, feedback from the report is facilitated by a feedback professional, such as an Executive Coach. 

Advocates of multisource feedback assert that it provides several benefits to managers.  Managers gain more comprehensive perspective on their performance when the feedback is provided by a mix of people with whom they interact and work (Seifert et al., 2003).  The anonymity of a 360-degree feedback emboldens participants to share feedback that they may not be willing to share directly or in person.  When that feedback is compared to the manager’s own assessment of themselves, it evokes greater self-awareness.  When managers use that awareness to improve their effectiveness, it doesn’t just benefit them or their direct reports, but can have a positive ripple effect in the organization. 

The advantages of multisource feedback may depend on how it is delivered to managers (Seifert et al., 2003).  The complexity of feedback increases when there are a high number of sources.  In addition, translation of the feedback report can be overwhelming, depending on format and volume of information.  A facilitator or coach can help managers interpret and prioritize feedback and integrate that feedback into an action plan.

Coaching

Although in its infancy, coaching is becoming increasingly popular.  According to the 2020 ICF Global Coaching Study, it is estimated that there were approximately 71,000 coaching practitioners in 2019, which is an increase of 33% since 2015.  The estimated global total revenue from coaching in 2019 was $2.85 billion, representing a 21% increase over the 2015 estimate (ICF, 2020).

There is no universal definition of coaching, but ICF (International Coaching Federation) has defined coaching as “partnering with clients in a thought-provoking and creative process that inspires them to maximize their personal and professional potential, which is particularly important in today’s uncertain and complex environment.  The origins of the term “executive coach” in business date to the late 1980s, largely as part of leadership development.  By the 1990s, companies began to combine executive coaching with other training programs to enrich leadership skills or avert executive derailment (Ghosh et al., 2014).  360-feedback processes are one of the development initiatives for which we see coaches engaged at an increasing rate. (Thach, 2002).

Research Studies on the Impact of Coaching and Multisource Feedback

Although multisource feedback is growing in popularity as a method of management development, there are a limited number of studies that have assessed its effectiveness.   There is also little empirical evidence about the combined effects of feedback and coaching on manager performance (Kochanowski et al., 2010).  However, there are a few studies, discussed below, that have shown a positive impact when executive coaching or feedback facilitation is combined with multisource feedback.

Study 1:  The Impact of Executive Coaching and 360 Feedback on Leadership

The article, “The Impact of Executive Coaching and 360 Feedback on Leadership Effectiveness,” outlines a three-year research study that demonstrates the positive impact that executive coaching and 360 feedback have on increasing leadership effectiveness.  In the study, a sample of 281 executives from a western USA mid-size, global, telecommunications firm participated in a 360-feedback process.  The 360-feedback survey was co-created by an external consultant and the company’s executive team to measure 17 leadership competencies that aligned to their organization’s five-year strategy.  Following the 360 surveys, participants received four one-hour coaching sessions, including one session dedicated to debriefing on the 360 results and one session focused on creating goals for their consequent development plan.  A mini 360 survey, focusing on leadership effectiveness, was administered after the participants had received coaching and implemented an action plan, which was six months following the original survey.  The study found that the perception of the participants’ leadership effectiveness increased an average of 55 to 60 percent.   Qualitative feedback provided by the executives who participated in the study revealed that coaching was valuable in providing additional feedback and instilling accountability to executing their development plans.  While the outcomes in this study are optimistic, the lack of isolated variables limits visibility to how participants may have responded to feedback without coaching. 

Study 2:  Can Working with an Executive Coach Improve Multisource Feedback Ratings Over Time?  A Quasi-Experimental Field Study

In a study published by Smither et al., 1,361 senior managers in a large, global corporation participated in a multisource feedback program in 1999.   After receiving their feedback, nearly 30% of participating senior managers worked with an executive coach.    The use of an executive coach was either required or optional, depending on what their senior manager elected.  Business lines that elected to utilize coaching provided 5 to 7 hours, the majority of which were allocated to reviewing the multisource feedback.  Early in the process, managers who were working with a coach were surveyed to gather their reactions to the executive coach and coaching process.  Of 286 managers who completed the survey, 79.6% of them indicated that they welcomed the opportunity to work with a coach and it was a requirement in their business unit and 12.7% indicated they requested to work with a coach even though it was not required.  Only 7.7% of participants who were required to work with a coach indicated they were hesitant.  Managers who participated in coaching were surveyed twice throughout the study to provide their reactions to working with an executive coach.  86.3% of senior managers indicated they wanted to work with a coach again and survey responses across six items indicated that senior managers had favorable reactions to their executive coach and the coaching process.  

Feedback from survey raters, at multiple points during the study, were aimed at evaluating four assumptions: managers who worked with a coach set more specific goals; managers who worked with a coach were more likely to share feedback and solicit ideas for improvement from their manager, peers and direct reports; executive coaching will be positively related to multisource ratings; improvements in ratings is related to the number of conversations senior managers have with the coach or to the manager’s rating of the coach’s effectiveness.  The study found that managers who worked with an executive coach were more likely to set specific goals, solicit ideas for improvement from their supervisors and receive enhanced ratings from their direct reports and supervisors (Smither et al., 2003).  The results of this study are important because they are the first to provide data on the impact of working with an executive coach from the perspective of other sources, such as supervisors, direct reports, and peers.

Study 3:  Effects of Multisource Feedback and a Feedback Facilitator on the Influence Behavior of Managers Toward Subordinates

Another study, published in 2003, sought out to find more definitive evidence about the effectiveness of multisource feedback workshops and feedback facilitation on changing managerial behavior through a field experiment (Seifert et al., 2003).  Three groups of managers were a part of the study, all of whom were rated in a multisource survey, specifically measuring the managers’ use of proactive influence tactics.  One group of managers received a feedback report in a workshop with a facilitator.  Another group of managers received the same type of feedback report without a workshop.  Managers in a control group received no feedback until after the study was completed.  Premeasure and post measure surveys were completed 3 months apart.

In advance of the study, the feedback reports were tested for readability and relevance.  The final report included a detailed overview of influence tactics and their effectiveness and an explanation of how to translate the feedback.  Managers were provided instructions on where to focus their attention and given tools to assist them in planning action steps for improvement. 

The feedback workshop was facilitated over a 7-hour period in which the first part of the workshop explained influence tactics and the second part of the workshop was a detailed overview of the feedback reports.  Facilitators went through each section of the report and provided guidance on how to interpret the feedback.  Finally, managers then had an opportunity to work through various scenarios while facilitators supported them in improving their influence behavior. 

The results of this survey demonstrated that managers in the feedback workshop significantly increased the use of two core influencing tactics while managers in the control group did not.  This study was important because it was the first to evaluate the effects of a feedback facilitator who was not the manager’s boss.  It established that having a skilled, encouraging facilitator increases the perceived effectiveness of feedback and results in more behavior change for managers (Seifert et al., 2003).  While the facilitators conducting the feedback workshops in this study were not identified as coaches, the process they followed is comparable to what a manager may experience through a coaching engagement and the findings related to the value of facilitated feedback is applicable to the role an executive coach may play in a 360-degree feedback process. 

Study 4:  Using Coaching to Enhance the Effects of Behavioral Feedback to Managers

In a study published by Journal of Leadership and Organizational Studies in 2010, an experiment was conducted at a midsized grocery store chain to test the hypothesis that coaching enhances the effects of feedback on a manager’s use of four core influence tactics.  All participating managers were involved in a premeasure and post measure survey, in which their direct reports and managers completed an Influence Behavior Questionnaire that measures 11 proactive influence tactics.  All participating managers received a feedback report comparing the ratings and guides to assist them in interpreting the feedback analysis. 

In addition to the report, all participating managers attended a six-hour feedback workshop to help them understand influence tactics, understand the feedback report, develop an action plan and goals to enhance the core influence tactics.  After the workshop, half of the managers selected for the experimental group received 6 weeks of follow-up coaching.  The coaching consisted of weekly 30-minute phone conversations and a 1-hour final in person coaching session. 

Participating managers completed questionnaires at the end of the feedback workshop and the results indicated a positive assessment of the feedback and workshop with no significant differences between the experimental and control conditions.  Managers also completed a questionnaire assessing their perceptions regarding the accuracy of the feedback, their intention to use the feedback, and their capacity to improve performance.  The results indicated that managers who received coaching after the feedback workshop increased their use of collaboration with direct reports more than those that did not receive coaching.  Although both groups of managers increased use of consultation, the increase was twice as large in the experimental group (Kochanowski et al., 2010).  While the results of this study are positive, due to participant attrition, the small sample size reduced the power of the statistical tests.

Key Learnings

While empirical research about the impact of coaching combined with 360-feedback on manager performance and behavior is limited and all the studies outlined above had limitations, each demonstrated some indications that there are positive effects when multisource feedback is combined with coaching or feedback facilitation.  360-degree processes are more effective when they come with support, including coaching.  The support of a coach to aid in processing and prioritizing feedback, as well as assisting in the design of an action plan, can positively impact the employee experience and ultimately, drive more positive change.  In the study outlined by Thach, coaching emerged as the most positive part of the process (Thach, 2002).  

Multiple studies highlighted the importance of a supportive and encouraging climate, including leadership engagement.  Not only is it important for senior management to be involved in introducing the 360-feedback and coaching process, but in maintaining interest along the way (Kochanowski et al., 2010, Seifert et al., 2003).  Organizational changes, particularly when they are perceived as being threatening, can impact the climate and ultimately the success of manager response to feedback and coaching (Seifert et al., 2003).   In addition, organizations that are actively shifting placement of resources or experiencing attrition could see and impact to the quality and participation of 360 raters.   Therefore, it’s important for organizations to consider the business context before introducing a multisource feedback program. 

The studies also suggested that multisource feedback is more effective when there is built in accountability (Smither et al., 2003, Kochanowski et al., 2010, Seifert et al., 2003).  In fact, store managers who participated in the Kochanowski et al. study indicated in a voluntary post-study interview that they had been more motivated to participate in other programs for which they had been held accountable and rewarded (Kochanowski et al., 2010).   Another way to increase commitment to utilizing the 360-feedback is to ensure it includes competencies and behaviors that are built into the performance management and talent management processes and systems (Seifert et al., 2003).  Therefore, as organizations consider implementing 360-feedback processes, customizing the assessment to align with organizational prioritized behaviors and competencies is more likely to drive accountability.  This also enables an executive coach to connect the 360-feedback insights to action plans that correspond to organizational competencies and behaviors.

Future Research

Given the time and financial investment necessary to implement a 360-feedback process, additional research is necessary to measure the impact and help organizations determine whether this is the most valuable leadership development strategy to implement.  Seifert et al. suggests that a better understanding of facilitating conditions, such as relevant skill training, behavior change incentive and supportive climate would be valuable and learning how to create those conditions should be a primary objective of future research on behavioral feedback program (Seifert et al., 2003).  Smither et al. recommends that future research should evaluate the impact of longer-term executive coaching relationships and how that influences manager behavior change over time.  Another avenue for future research is to evaluate individual differences in readiness for change or receptivity to coaching and feedback (Smither et al., 2003).   

Conclusion

As organizations continue to invest in leadership development, 360-degree feedback will remain a popular tool.  The extent of the value of coaching is unknown and there’s no denying that more research is necessary.   Although the research is limited, the studies reviewed in this paper provide an optimistic outlook on the benefits of supporting multisource feedback with feedback facilitation and executive coaching. 

References

Bracken, D., Rose, D. (2011). When Does 360-Degree Feedback Create Behavior Change?  And How Would We Know It When it Does?  Journal of Business Psychology. 26, 183-192.

Center for Creative Leadership (2019).  Benchmarking the State of Leadership Development.  https://www.ccl.org/wp-content/uploads/2019/10/benchmarking-state-of-leadership-development-spending-report-center-for-creative-leadership-ccl.pdf

ICF. (2020) Global Coaching Study Executive Summary.

Fleenor, J.W., Taylor, S., Chappelow, C. (2008).  Leveraging the Impact of 360-degree Feedback.  Center for Creative Leadership. 65-68

Ghosh, R., Marsick, V., Maltbia, T. (2014). Executive and Organizational Coaching:  A Review of Insights Drawn from Literature to Inform HRD Practice.  Advances in Developing Human Resources. 16(2). 161-183.

Kochanowski, S., Seifert, C., Yuki, G. (2010).  Using Coaching to Enhance the Effects of Behavioral Feedback to Managers.  Journal of Leadership and Organizational Studies. 17(4) 363-369

Passmore, J. (2012). Psychometrics in Coaching: Using Psychological and Psychometric Tools for Development. (2^nd ed). Kogan Page Limited. 67-69

Seifert, C., Yukl, G., McDonald, R. (2003).  Effects of Multisource Feedback and a Feedback Facilitator on the Influence Behavior of Managers Toward Subordinates.  Journal of Applied Psychology 88(3), 561-569

Sherman, S., Freas, A. (2004).  The Wild West of Executive Coaching.  Harvard Business Review. November 2004.

Smither, J., London, M., Flautt, R., Vargas, Y., Kucine, I. (2003). Can Working with an Executive Coach Improve Multisource Feedback Ratings Over Time?  A Quasi-Experimental Field Study.  Personnel Psychology. 2003, 56

Thach, E. (2002). The Impact of Executive Coaching and 360 Feedback on Leadership Effectiveness.  Leadership & Organizational Development Journal 3(4), 20