When Measuring ROI and Soft Skills, Don't Get Bogged Down in Metrics

When Kevin Nieman returned to First Horizon National Corporation, a financial services company, as the organization's Chief Learning Officer six years ago, he helped implement a process to measure the impact of the company's culture.

As a soft skill, culture has traditionally been thought to be difficult to measure, but Nieman used a survey developed by the Hay Group that measures 56 attributes of culture.

While this survey attempted to measure company culture, Nieman cautions that it's important to not get bogged down in metrics. When it comes to showing his department's return on investment to executives at First Horizon, Nieman says he's had the most success listening to the direction the company is headed and then aligning learning objectives with the strategy of the business.

Nieman said he asks leaders, "What would tell you we've been successful?" to determine which metrics leaders are looking for.

Kevin was a part of the panel "Lessons from Learning's Elite" at Corporate Learning Week 2013 that focused on a learning organization's role in developing and growing a company's culture. For more information on Kevin, see our Q&A below.

Can you talk about the culture-related initiatives that L&D has spearheaded at First Horizon?

Back in the early '90s, we began to think about how culture drives performance. But most particularly, since I've been back at the organization in the last six years, we began to look at how culture is actually affecting the workplace today.

We used a company called the Hay Group. They have 56 attributes that measure culture and we had some of our top leaders examine those attributes to determine where our strengths were, where our weaknesses were and how they would impact our business.

If you look at what we want to do as an organization; we want to be there for our customers. And we want to be good at that.

But when it came to actually managing change, we had some growth opportunities. In 2008 and 2010 we measured results and saw we made progress in the very areas we wanted to.

Going a little bit further, we began to say we're moving fast, but, how do we get even better? Where are those situations where you have silos within an organization, where one group is paying attention to itself at the expense of the other parts of the organization?

Other questions might be if you have a situation where you say we want truth to come out and people to be candid with their leaders, where were those situations where the climate  encouraged that kind of behavior and where were those situations where while we said the right things, but in actuality, the climate or the culture prevented us from being candid, actually inhibited candor.

So as you begin to look at this, you begin to think about what are the specific interventions that we could implement that would change that for the better.

What steps have you taken at First Horizon to show executives the ROI on your various training programs?

First, we have done specific programs—one of the easiest one is sales—where we use metrics to identify: Did the intervention we put in place actually result in more sales? Did it increase the revenue per employee in that situation? And we've done some of that.

What we've actually spent more time focused on is what are the actual things that leaders see as their strategy and we've partnered very closely with them.

So, for example, their strategy was to make sure that we were asking the right questions of our customers. Those questions led to a better customer relationship and ultimately we knew wer were in line with where we wanted to go as an organization.

What we tried to do was make sure that we were at the table with our executive leaders so we knew exactly what they were trying to drive and all the learning efforts that have our focused on what they wanted.

I don't mean that we don't push back, because one of the other things is, if we think they're headed in the wrong direction, our culture allows us to say, "You know what, we better raise the issues with that executive, so that we don't allow them to not ask the question." So, while we do some ROI measurement, we really focus on being aligned with the strategy of the business.

That makes a lot of sense with being up front and not just measuring on the back end, like a lot of companies are doing.

That's a great point because if you don't know where it is you want to go, then any road will get you there. So we sat down with the executives or the line of the business leader and we asked them on the front end of any learning initiative whether it's cultural or simply skill-based, if we can get them to define for us exactly where they want to go. 

Then the bottom line is we can ask them, "What would tell you that we've been successful?" And so that's the collaboration that takes place on the front end, which means that we can then measure it on the back end.

Have you bought into any learning management systems? Do you use things like mobile technology or e-learning?

Absolutely. We have a learning management system, which also is a learning content management system, which is also integrated with our performance management system and our succession planning system.

So we bought a package that's integrated all the way, and the theory behind that was, if we understand from the strategic standpoint where we're trying to go, our learning is designed to find what are the gaps in skills and then the learning management and the learning content management system allows us to align competencies that are needed to execute that strategy. We felt like we would have a much better chance of being successful in actually driving business results.

Then, if you look at it from the standpoint of e-learning, in general, we've been developing our own e-learning. We have very strong technical training—both management as well as our designers. We have a small in-house design shop on a relative scale.

We've measured how do we compare from a cost per employee standpoint against what we would pay if we went outside, and we've been able to keep out costs much lower.

But the quality that we do, we get great feedback on, and that's compared to some of the external programs. We are in the process of building social networking. It's a constantly changing environment.

We have some internal tools, but we're constantly looking to upgrade those tools. And as far as mobile learning, we're simply at this place sitting, trying to decide what really works for us.

What are the emerging trends in learning and development that you are specifically focused on?

We would continue on the line of the idea that social learning, particularly where that social learning allows people to be transparent.

I read a case study not too long ago about how organizations make decisions based on limited information with regards to what is the support in the organization for a particular type of effort.

Well, it's one thing to think. It's another thing to do surveys, and then, it's another still when you create a situation where people can be candid, knowing that whatever they say is going to be visible from the chairman, all the way to the front line employee.

Every person's name is attached to what they said, so if they feel that strongly about it, they can be transparent, but they also know what the limits are.

Well, I've seen that in other organizations, and the business results that were achieved from it is people—the executive leadership all the way down to the front line employee—knew exactly how people felt about a particular initiative or idea. And they were able then to make adjustments because they knew exactly what the truth was.

We're looking at that to say, "How do we incorporate that into our organization?" We have some of it, but how do we take that further, so without a formal program, you can actually sort of understand what people are thinking to drive business results.

What are you most excited about at this year's conference?

Well, I think any time you can get together with people in similar roles, who have the ability to influence the bottom-line business results through learning, that's an exciting thing to be a part of.

So being able to both hear those who are speaking, but also to be able to network with people who play a similar role to what you play and in fact many people who have had a lot more experience than I've had. I've been at this for quite a while, but having the opportunity to exchange ideas with those people, having that is great.