Coaching Bots, Heuristics and Schemata

I attended (or logged into, to be more strictly accurate) a fascinating talk hosted by the EMCC, and presented by a research team who had compared the effectiveness of three coaching bots. 

They had build prompts to get ChatGPT to simulate three different coaching approaches, and had a fairly large sample size of coachees to interact with them.

The results were fascinating.  The team were Jonathan Reitz, Rebecca Rutschmann and Nicky Terblanche, and the three approaches they were comparing were GROW, Solutions Focussed and Cognitive Behavioural Coaching. 

Their initial hypotheses included that the GROWBot would perform best in terms of goal attainment and the goal aspect of the working alliance and would be perceived as easiest to use; and that the Solutions Focussed Bot would be most highly rated for the bond and task aspects of the working alliance, and for the working alliance overall, and would also be perceived as more effective in improving performance, more enjoyable to use, and leave users with a more favourable attitude overall and a higher likelihood of using it again.

None of these hypotheses were supported.

Instead, the analysis showed that all three bots scored similarly for goal attainment, and for every other measure the CBCBot outperformed the other two, and by statistically significant amounts in all but one case (the bond aspect of the working alliance). 

I was less surprised than the researchers (and indeed wondered how they had developed their initial hypotheses). CBT is very well researched and its efficacy is well-known (though not, of course universal) so I would have expected its coaching cousin to do well. 

And it lined nicely to the most recent module of the Psychology of Coaching Programme that I am studying, which was on Cognitive Psychology.

This was fascinating as ever, albeit rather familiar territory to me. My book Shifting Stories is a simple (though of course powerful and sophisticated!) application of cognitive psychology. And what this module did was to provide some depth and some fancy language (heuristics and schemata for example) for concepts I work with quite frequently.  But it is always good to refresh one's understanding and re-visit the assumptions and biases that underpin one's approach. And of course the future modules will probably take me into new territory, so it is quite reassuring to start from a feeling of moderate competence, before launching into the unknown.

It is also interesting that it is in the field of cognitive psychology that the coaching Bots seem to be effective (at least in terms of the aspects measured by the researchers on this project). But my hope is that I will learn more about the relational and other aspects of coaching to understand whether my hope that I am irreplaceable as a coach is well-founded or self-deluding.

My initial hypothesis, for what it's worth, is that the human aspects of coaching that Bots will not be able to simulate generative attention, nor the intuition that is born, perhaps of parallel process, nor compassion.  But how we measure the impact of these is a problem for the researchers...