Economist / KMPG Evidence Based HR ~ Strategic Human Capital Management (HCM) Blog

One of the topics I expect to come up at the Economist's Talent Management 2015 summit in a couple of weeks time is evidence based HR, so I've been taking a look through their recent research for KPMG on this topic, which is defined as using the best available evidence to support people-related decision making.

Now I'm a big fan of the Economist's research and know from my experience working as a researcher for one of their sponsored studies how strongly the group upholds its editorial principles in dealing with its clients so I know we can trust the survey results.

However, I am still surprised by, and would argue against some of the report's conclusions. Actually, as a more general point, I do find it interesting that it's particularly in regard to reports on data, evidence and analytics where I tend to find myself most critical about the use of data and analytics within the research! (For example, the worst research report I've ever read in terms conclusions being extrapolated from the data was this one on big data conducted by the CIPD for Oracle.)

Take a look at this:

"Although more than half of all survey respondents (55%) remain skeptical about the potential of evidence-based HR to make a real difference to the HR function, the overwhelming majority of these skeptical executives (82%) plan to increase their use of Big Data and advanced analytics in the short term. We can deduce from these two findings that the obstacles to the successful implementation of these tools are considered to be substantial.

One stumbling block may be the credibility of the HR function. Less than half of non-HR executives (49 percent) agree that HR leaders are able to demonstrate tangible correlations between people management initiatives and business outcomes. In other words, the skepticism is not about big data but about HR practitioners’ ability to use it effectively."

Where exactly is the evidence for this deduction and for these 'other words?'

I think part of the problem with this research is that it is so heavily branded by KPMG - featuring lots of their more general comments before we get into the formal research conducted by the Economist. It also probably doesn't help that although the research focuses on evidence and includes comments from leading academics, KPMG's interest is obviously more focused on big data and analytics and much of my criticism relates to prioritising focus on these.

Prioritising data and analytics?

KPMG suggest that "for the first time in 30 years, HR has a massive opportunity to demonstrate the value it adds to the delivery of business objectives. Now, more than at any other time, the technology is available to access data from many sources and provide predictive insights that will positively impact the delivery of business strategy."

I'd suggest the points in the comment are broadly true but I don't think it's data and technology which should be allowed to drive organisational approaches to analytics and even less, to evidence based HR.

For example, I think this sort of comment from David Crumbly at Coca-Cola is highly damaging: "Mr. Crumley believes that companies need to get the basics right – clean and standardized data, and then the resources, culture and technology to make full use of it – before prioritizing predictive analytics over less ambitious, but still important, investigations. “Many companies want to jump straight to the sexy stuff,” he says. “They try predictive analytics, but they can’t even run a headcount report for their company. They try to run before they can walk. First of all, you need trust in the data; without that trust, you can’t create senior management belief and sponsorship.”

To me, this is completely the wrong advice and is the main reason why so many organisations seem to think it will take them five to seven years before they can start to do predictive analytics. It's simply not true (I know as I've helped clients do it.)

My concern also applies to one of the chief case studies in the report - McDonalds use of their HR system People Stuff to identify hat having at least one employee over 60 dramatically improves customer satisfaction, on average by 20% and boosts performance in a fast food outlet.

"A 2009 study conducted by Lancaster University Management School, which examined the performance of more than 400 McDonald’s restaurants across the U.K., found that employees aged 60 and over deliver a significant business boost for the company.The study revealed that levels of customer satisfaction were on average 20% higher in restaurants that employ at least one worker of that age profile. 'An older population segment was attracted into the stores, and the older workers changed team dynamics for the better,' says Paul Sparrow of Lancaster University. 'We were able to demonstrate the financial uplift.'

This clear connection between people data and business performance is the sort of discovery to make senior executives sit up and take notice. Professor Sparrow notes that this nugget emerged as a by-product of other research. Although not obviating the need to focus on a clear objective at the outset of any evidence analysis, the magic of evidence-based inquiry is that sometimes a jewel will be unearthed when looking for something else entirely."

Magic? Jewel? Or just snake oil and fool's gold? The problem with this sort of approach is that there's no strategic basis for assuming that employees over 60 cause customer satisfaction. They may be correlated together, they may even be predictive, but there may be some completely different variables which actually cause both more applications from older workers and higher levels of customer satisfaction within the same outlet.

So actually, despite all my criticism of the report, I do largely agree with KPMG's evidence based HR methodology shown in the diagramme at the top of this post. And with Jenn Mann of SAS too: "Data mining has little purpose unless the right questions are asked before embarking on the process. “You mustn’t just analyze data for data’s sake.You have to think about the real business challenges that you want to try to answer with your people data,” she says. “That also means asking the business leaders outside HR what they want to gain.”

Identifying a business challenge is the right starting point for analytics and evidence based HR - not data or technology.

Using data and analytics rather than gut instinct?

I also reject the report's suggestion that data and analytics will mean we don't now have to live in a world of suspect validity. Life in organisations and particularly in HR is just too complex for this. Data, evidence and analytics all help, but they're not going to mean that we're living in a machine. HR's not the same as medicine or even consumer profiling - we're operating within a system rather than upon a system and we need to recognise that there's a whole different set of dynamics that come along with this.

And actually, although the report pushes against gut instinct which it suggests is frequently the dominant decision-making mode ("This is not about gut feel and intuition of the past – this is about using data that drives decision making, and links people data with business outcomes" and "The use of evidence will lead to more effective decisions than those based on pure gut instinct") the report uses the words 'belief' or 'believe' - rather than the word 'know' - 24 times.

Please note I'm not actually arguing against using gut instinct - after all I'm using this myself. It's just that my gut instinct is different to KPMG's. My gut instinct is that we'll always need gut instinct. Oh, and at least I'm open in saying that I am using gut instinct rather than trying to dress this up as something which comes from big data.

Summary

All of this means that although I don't disagree with KPMG's suggestion that we all need data scientists to work with the HR function I think Paul Sparrow's point that an advanced social science background ("They need to know when to trust evidence, how to differentiate between good and and bad research.") is much more important.

I'm also not convinced that "HR functions need to be reconfigured so that HR and management work together within an operating model that promotes evidence based people management. As much thought should go into designing the components of the operating model that will allow for the use and consumption of the evidence as is given to the actual generation of evidence."

I do agree about the importance of designing information flows, visualisation, decision making, roles, responsibilities and capabilities - but that's different to the overall organisation model. To me, this puts the cart before the horse. An HR function's operating, or organisation, model needs to to built upon what an HR function needs to do, not just on collecting evidence in order to do what it needs to do. And no HR function is going go get far just by using and consuming evidence. Also there is certainly no evidence offered within the report to support KPMG's conclusion that we need to change the HR structure.

The same applies to the dramatic summary statement that evidence based HR is "the unstoppable force." (That's despite acknowledging the significant stumbling blocks which . But there's an "inescapable logic" dictating it apparently.)

"The HR mindset of the past, negatively regarded by some interviewees, will soon inevitably erode and reconfigure itself along very different lines." Data? Evidence? No, just one belief about the future (and I leave it to you to decide whether there's any bias to the statement perhaps linked to the services that KPMG may be trying to sell you*.)

I'll be blogging with more thoughts on data, analytics and intuition over the next few weeks so if you're interested in the topic, look out for these posts too.

* By the way, I'm not trying to sell you projects on data and analytics as I tend to consult around more well-rounded approaches to decision making that allow for, and try to optimise, gut intuition based processes too. You'll see that bias in my commentary too.

Also see my posts on measurement, analytics and evidence
In particular: Analytics, analytics, analytics, not the future of HR