The opening key note address is by Prof Jonathon Passmore [JP] with the title: “Coaching Research: The Good, the Bad and the Ugly”.
The session looks at coaching research especially on coaching in organisations and a critical review of the literature but is these on the good, the bad and the ugly and
1. why research coaching
2. what makes for good quality coaching research
3. key themes in coaching research and
4. suggested direction of research for the coming decade.
Why research is a question he asks in organisations with the response of there’s no need as “we know it works”. But coaching involves risks and there is a need to demonstrate effectiveness and ROI with positive outcomes for individuals and organisations. But this is difficult in terms of agreeing participation, problems of measurement of intangible benefits which can be difficult to publish.
The quality of research depends on the research question that is clearly defined and bounded; that the research method is appropriate, clearly described for purposes of replication and correctly executed; that results are compared with and positioned within earlier research and that conclusions are appropriate and not over-claiming as well as identifying new questions.
Critical questions to ask f the results from research is to query whether a placebo effect is occurring or that other factors contaminated the research, i.e., other training going on or selection of high performers leading to positive outcomes. Also, can the research be replicated. But few studies meet these criteria.
Can look at phases of coaching studies: phase 1 involving case studies and surveys; phase 2 involves theory development through qualitative research which is valuable in immature research areas like coaching – putting up a straw man to be challenged; phase 3 has seen initial randomised controlled trials (RCTs) and a small-scale (25 – 40 people) but provides important evidence on individual and psychological impacts; phase 4 sees larger RCTs (Passmore & Rehman 2012) and phase 5 sees an increased use of meta-analysis and includes the increase ease of access to data sources as well as the impacts of the ‘computational turn’.
These studies have identified a number of popular themes of coach behaviour attracting lots of papers as did the coach-client relationship. But only limited research on client decision-making on coaching and an increase in research on the impact of coaching.
Coach behaviour research, e.g., Hall et al (1999) involving interviews of coaches and clients identified some tentative behaviours but has been validated by future studies especially around the discursive and collaborative approaches and the power relations and dynamics to work collaboratively. Probing and challenge is an emerging area as a distinction from the empathy focus of counselling. JP cites client work and that senior leaders relish challenge. Aspects of confidentiality are critical to effective coaching including risky behaviour as well as commercial confidentiality and maintaining professional distance is also important in the evidence of effective coaching.
Literature on the coach and coach relationship focus on the develop of an alliance between coach and coach but little evidence of what factors make a successful relationship although these can be inferred from other studies, e.g., empathy
Outcome studies (McGivern et al 2001) as a ROI study based on Jack Philips method of ROI leading to an estimate based approach and then decided to cut the number in half – although this was not really justified. JP assessed this as twaddle and rubbish and we need different methods for HRD (the bad research).
Identified 156 outcome studies between 1998 & 2010. Of these, most are small-scale with 30 or so participants and some RCTs. Miller used quasi-experimental study and found no statistical significance on a beneficial impact of coaching but this may be that the coaching intervention was limited and didn’t lead to behavioural change or that managers tended to revert to a more directive styles. Also. a lot of RCT studies involve students not in organisations but these did show psychological benefits of coaching around resilience and mental health. Passmore & Rehman (2012) RCT of military drivers found that a coaching approach reduced time for training and success rates increased.
Some outcome studies have involved longitudinal research evidencing a longer-term effect of coaching that may indicate that coaching is more effective, deeper learning and greater behavioural change than training interventions.
But coaching still only has a small number of studies and these have a small sample sizes compared to studies in health settings. e.g., conducting RCTs in organisations is difficult. Also, the isolation of variables and factors of interest can be difficult (Hawthorne effect), outcome study methods are often not fully described and that research is often undertaken by champions of coaching with inevitable biases.
Meta-analysis research, e.g., De Meuse, Dai and Lee (2009) but this was only based on four papers only so interesting in terms of being a meta-analysis but based on very little data (the ugly). Teeboon, et al (2013) and Jones (in press) more robust papers. Teeboon found positive benefits around factors such as coping, goal directed and self-regulation, performance, attitudes and well-being at about the same level as other L&D interventions. So coaching is one of a number of effective interventions available for L&D practice. Jones study is of 24 RCT studies and looked at effect size on style of coaching and found a larger effect size of internal coaches compared to external coaches. Jones found that coaching had a medium to strong positive impact but the findings should be treated with caution given the small number of papers used.
The future of coaching research may be dominated by either (a) a business school use of case studies; (b) an organisational psychology approach model that disconnects scholarship from practice; (c) a medical based approach with an emphasis on evidence based practice that informs experts including scholar-practitioners.
Research needs to aim for larger RCTs involving random allocations involving two or more interventions, a control group and placebo group. Research needs to identify factors for effective coaching. Need larger scale meta-analysis to identify impact effect sizes.
This will improve understanding of efficacy and appropriateness of coaching or other interventions and then which approaches to coaching are appropriate for different needs and which coaching behaviours are most effective. Also, identifying when is a client ready for coaching in terms of the individual and the organisation (i.e., as managerial support and a supportive culture). Lastly coach behaviour research underpins PG programmes and by professional body competence.