#ATD2016 An Alternative to Kirkpatrick Evaluation

ATD International Conference Talent Strategy Mapping from Jon Ingham

I've come to ATD's main international conference after speaking at ATD MENA last year.  But also because I've been following the tweets from the last couple of years and have been increasingly concerned by the heavy focus on the Kirpatrick evaluation model here (much, much stronger than the CIPD's Learning and Development Show etc).

Kirkpatrick Partners suggest that "Don Kirkpatrick’s groundbreaking four levels of training evaluation is the most widely used training evaluation model in the world. Ask any group of trainers whether they rely on it in their practice, and you’ll get an enthusiastic affirmation."

The first statement is absolutely true.  The second is nonsense.  Come on!, we all know if doesn't work.

Last year there was also a session on Kaplan and Norton's Balanced Business Scorecard and how that helps training and learning evaluation.  It doesn't.  The BBS works for the rest of the business because it provides balance, flow and the ability to link planning objectives (the business strategy map) to measures (the scorecard).  It doesn't work for talent management and development because it doesn't give us balance or flow.

So I decided to come and talk about my own model, based upon the strategy map, which does provide these benefits (have a look here for an explanation).  It's also very closely related to Kirpatrick's 4 levels but because it's organic vs mechanistic (ie I don't tell you what you need to measure, this comes out of your own business and talent management / development strategy) it works whilst Kirkpatrick doesn't.

Participants in my session had an opportunity to try the approach out too and the strategy maps / value matrix they came up with for a health insurance business represented in the room is provided below (thanks Keith):

The objectives and measures in here are what I suggest WHA should evaluate, and the measures for evaluation are provided by the framework.  There's no need, worry, complexity or confusion to have to consult Kirkpatrick on what a particular measure should be.  And if it's not in the value matrix, you probably don't need to both evaluating it.

I asked our 'client' (Keith, the CHRO) whether the measures we'd identified were mainly the same or different from those they currently use for evaluation and he said they were largely the same.  That was a slight shame but participants had only had a maximum of 15 minutes to play with my framework, so I wasn't too disappointed.

And I think it does demonstrate the main point I was trying to make which is that Kirkpatrick's four levels are too constraining.  Eg up at the top of the matrix there's an objective about improved collaboration which is going to be measured through network mapping.  A great, perfectly aligned measure.  But where is that in Kirkpatrick's system?

It also demonstrates the difficulty measuring strategic attributes (those at the top of the framework) in an objective, quantified way.  Eg we didn't get to develop a measure for change agility, but I bet it won't be a percentage or rating, it's going to be something a bit more complex and therefore qualitative and potentially subjective, which is fine - but of course rules out doing a quantitative ROI study.

But I'm going to leave this post with my criticism of the Kirkpatrick model - if you want a critical review of Jack Phillips ROI methodology have a look here.


Later: actually I am going to extend my discussion into Phillips methodology as well.  This is because I've been reading more of his writing at the ATD conference and I think this demonstrates very effectively why Kirkpatrick / Phillips doesn't work.

Firstly, the article in the ATD's current TD magazine: The Value of Engagement.  Because Phillips uses the Kirkpatrick levels he's forced into completely inappropriate measures.  How do you measure engagement?  Not through 'reaction to engagement'!  What on earth is that?

Or take his new book on the ROI o Organisation Development.  There's not one single mention in the index of complexity, and only a couple of brief mentions of intangibles.  Ie there's no recognition at all that when you focus on something as deep as OD most of the outcomes you're creating fall into Phillips' bucket of intangibles and don't play a role in his perfectly precise ROI equation.  So what's the point of it?  I wasn't in the bookstore long enough to work out whether he recommends a 'reaction to organisation development' measure.  But wouldn't that be bizarre!

Actually, all you've got to do is to go back to the value matrix, check how you're planning to create engagement or a better (in WHA's case more agile) organisation and then measure / evaluate that.  Which is why we need a qualitative measure for change agilty.  And a discussion rather than a single reliable but completely invalid figure for ROI.


You can review the slides from my ATD presentation / workshop session here. 

• Consulting   Research  Speaking  Training  Writing
• Strategy  - Talent - Engagement  - Change and OD 
• Contact me to create more value for your business
• jon [dot] ingham [at] strategic [dash] hcm [dot] com

 

#ATD2016 Simon Sinek on Social Leadership

The first keynote at ATD's International Conference is Simon Sinek speaking about leadership.

We're social animals - we respond to the environment we're in - which is often shaped by the role of the leader (for me it's about the whole team / organisation - not just the manager).

We need this environment to encourage trust and co-operation - but the problem is that these are feelings not instructions - I can't instruct you to do this.  I can tell you to work together but that's different.

This requires people to feel safe.  And if that's not there people experience danger and this breeds cynicism, paranoia, mis-trust and self-interest.

The circle of safety is about not fearing other employees.  It's supported by goals and incentives, because these govern the chemicals which govern our behaviours.   These have developed during our evolution allowing us to sleep soundly at night because we're with people we trust.

Endorphins mask physical pain (eg in strenuous exercise, laughing etc) enabling us to push ourselves that bit further.  We've evolved for endurance. We persist because it feels good, not for intellectual attainment, it's because we crave endorphins. 

Dopamine is triggers when we find something we've been looking for or hit the goal we've been working for eg when we find food, or today - when we're able to cross something off our todolist.  The more concrete vs abstract the better - people won't work just for 'more'.  Metrics help us count progress.  A vision is a crystal clear picture of where we're heading.  A goal provides the finish line and helps us measure progress towards it.  You can't provide a vision but you can help people find it, which helps give meaning and purpose.  Dopamine is the reason we are accomplishment machines, but it can be very highly addictive (nicotine, alcohol, gambling, your cell phone etc).  People can get addicted to hitting their numbers, which also destroys relationships and the environment etc.  Employees stab each other in the back - there is no trust.

Endorphins and dopamine are selfish chemicals - you don't need anyone to help you achieve them.

Serotonin is the chemical for pride - triggered through things like social recognition.  This reinforces the relationships between manager and employee, coach and player etc.  Dopamine doesn't create loyalty, gets very expensive after a while and destroys trust.  But when people give us their time and energy and recognition we'll give them up blood, sweat and tears to make them feel proud.  It's the loyalty chemical.  We're traditionally lived in communities up to 150 people and have become hierarchical animals to help understand how we fit against each other.  We're still judging, assessing and self organising and giving preference to our alphas.  It helps gives us perks and advantages.  However the group is not stupid - there's an expectation that the alpha person - stronger, smarter, more confident (because of all the serotonin flowing in their veins because of all the recognition we give them).  Banks have violated this contract - allowing, or deciding to sacrifice others to protect their own bonuses - offending our sense of morals.  That creates distance and we'll never help them achieve their vision, it becomes a simple transactional relationship.  How do companies have record layoffs at the same time they have record bonuses.  What would anyone volunteer their best ideas?  Madness!

When there's danger leaders put themselves in harms way.  Leadership is a risk which is why not everyone is cut out to be a leader. When we feel leaders have our backs we’ll take care of our companies, and of each other.  It's not about authority over us.  Leadership is a choice, a daily practice, to put other peoples' lives ahead of us and our own interests, like being a parent, driven by evolution and anthropology.  People have to understand a company will invest in them in preference to short-term savings.  Leaders aren't responsible for results, we're responsible for the people who provide the results.

How long will a leadership programme take?  The truth is I don’t know.  But you have to stick with it and eventually the pain goes away.  We just don’t know exactly when.  You have to invest in training and doing the small stuff like putting the kettle back on for the next person who might want a coffee.  Leadership is about lots of consistent ongoing small practices.


Oxytocin is everyone's favourite chemicals, the chemical of love and unicorns, the wishy washy stuff.  We'd rather sit next to someone we know because it feels good and why we leave empty seats between us - because strangers are scary.  We want to be near the people we trust - it helps us feel safe.  Oxytocin is released when a mother gives birth - it bonds them together.  Using Facebook provides oxytocin but it's an unhealthy relationship so Facebook users experience higher rates of depression.  It's why we hug each other in hard times.  It's about giving time and energy with no expectation of a return.  Why bonuses don't work.  We put an emphasis on people who spend time and energy, not money.

Witnessing an act of kindness in someone else also triggers oxytocin and encourages us to be kind to others.  It's the basis for pay it forward.  We call you a leader because you took the risk to go first.  And gave us responsibility before we felt ready for it.

We want to live in a world where people go to work feeling inspired.  Our work is an act of service, an act of selflessness which protects other people.  We have to fight to help protect other people.  Leaders need to give us the space we need to take care of each other.

What we do is hard and stressful and difficult to measure.  We need people who've got our back.  If we take care of each other we'll change the world.

• Consulting   Research  Speaking  Training  Writing
• Strategy  - Talent - Engagement  - Change and OD 
• Contact me to create more value for your business
• jon [dot] ingham [at] strategic [dash] hcm [dot] com