The Black Hole of CX
Release Date:
CX leaders have become increasingly effective at taking action on customer feedback. Today many organizations have effective closed-loop programs that efficiently route insights to the right person for timely action. However, there’s a blind spot – the vast majority of customers never provide their feedback. Actually that’s more than a blind spot. It’s a black hole! This episode’s guest has built an impressive early warning system that addresses this problem. Host Pat Gibbons welcomes Maddie Clark, Customer Experience Insights Senior Manager at ServiceNow, for a discussion on how she built an early warning system that addresses the problem of identifying dissatisfied customers that don’t provide feedback.
Maddie Clark
ServiceNow
Connect with Maddie
In this episode:
- Chapter 1: Introduction and Overview (00:01 – 02:12)
- Chapter 2: Maddie’s Background and CX Experience (02:12 – 03:59)
- Chapter 3: Building the Predicted NPS Program (03:59 – 05:54)
- Chapter 4: Establishing Buy-In and Foundation (05:54 – 08:17)
- Chapter 5: Setting Up the Predictive Model (08:17 – 12:00)
- Chapter 6: From Model to Action (12:00 – 16:38)
- Chapter 7: Measuring Success and Continued Innovation (16:38 – 19:22)
- Chapter 8: Closing Thoughts and Take Home Value (19:22 – 23:06)
Highlights
Building a model
“We had to get to a place where we could actually predict whether a customer was at risk or not. And so we went about that by taking over 500 operational variables that we have across customers, regardless of if they respond or not. So what those look like for ServiceNow, a B2B software company where things like product adoption data, usage data, support case data, event attendance data. So all these different avenues that we know are really critical to a customer’s experience with us. We pulled in these variables and we were able to train the model again with our NPS data. So you need that strong foundation and existing relationship program. And we were able to train the model with our existing NPS data to understand for accounts that we don’t hear from with these 500 plus operational variables, how can we predict if they’re at risk for dissatisfaction or not? And we were able to build a model with that and make a really accurate model, because we had that NPS data to train it with.”
Flexing a new muscle
“…I think for us, what was kind of the worst case scenario that we were really working to, to solve for is we build this incredible program, but we don’t get any action from it. And we know that doing that there’s there’s really no point. And so that’s why coaching and training pretty much was the most intensive part of this process for us. And I can’t stress enough how important that is to make something like this successful, because you do need to get your front lines on board. Because this concept is nebulous, you’re not giving them a survey response from a customer, you’re giving them a prediction. And that’s a new muscle for our account teams. It’s not something that they’re used to doing. They’re used to getting that direct feedback. And so what we really focused on is educating them on how this is most helpful to them in terms of making their lives easier.”
Transcript
Pat:00:00:01
How do you help an unhappy customer if they never tell you they’re unhappy?
Maddie:00:00:05
By taking over 500 operational variables that we have across customers, regardless of if they respond or not. And we were able to train the model with our existing NPS data to understand four accounts that we don’t hear from with these 500 plus operational variables, how can we predict if they’re at risk for dissatisfaction or not?
Pat:00:00:27
Innovative ways to identify dissatisfied customers. On this episode of The CX Leader Podcast.
Announcer:00:00:42
The CX Leader Podcast is produced by Walker, an experience management firm that helps our clients accelerate their XM success. You can find out more at walkerinfo.com.
Pat:00:00:53
Hello everyone. I’m Pat Gibbons, host of this episode of The CX Leader Podcast, and thank you for listening. It’s a great time to be a CX leader, and on this podcast we explore topics and themes to help leaders like you develop great programs and deliver amazing experiences for your customers. CX leaders have become increasingly effective at taking action on customer feedback. Today, many organizations have effective closed loop programs that efficiently route insights to the right person for timely action. However, there’s a blind spot. The vast majority of customers, well, they never provide feedback. And actually, if you think about it, that’s more than a blind spot. It’s like a black hole. Well, our guest today has built an impressive early warning system that addresses this problem. Better yet, she’s willing to share her approach so that you can do this too. Maddie Clark is Customer Experience Insights Senior Manager at ServiceNow, an AI platform for business transformation to connect people, processes, data and devices to maximize productivity and business outcomes. Maddie, welcome to The CX Leader Podcast.
Maddie:00:02:12
Thank you.
Pat:00:02:13
It’s it’s really great to have you. And I know you’ve got a really innovative program and you’ve written an article about it and so forth. Um, you know, it’s always a challenge if you rely on surveys, which we know are a great way to get feedback but have their limitations. So really interested in getting into this, uh, this discussion. But first, give us a little bit of background on your experience in CX and your role at ServiceNow.
Maddie:00:02:39
Yes, I am happy to and I’m happy to be here and talk about all things predicted NPS. I have spent my entire career in in CX, so born and raised in this field. I started actually servicing. So building out um, platforms for, for clients. And then I went to the insights side of the house. So helping um, customers in a consulting capacity use their CX data to the fullest. And then I landed at ServiceNow, where I am now. And at ServiceNow one of the things that I do is lead our predicted NPS program, which I know we’re going to talk about today, to really tap into that silent majority to get all the goodness and impact that CX has on customers with that silent majority.
Pat:00:03:27
Yeah. Well ServiceNow is is a client of Walker’s. And of course I know a number of your colleagues and uh, many of the impressive, uh, programs that that your team runs. Um, but as we mentioned, you know, it’s surveying does have its limitations. But, you know, you’ve built this program to reach all customers, you know, even if they’re, you know, not reaching out to you or not giving feedback. So give us a little background on how you came up with this and how you laid the foundation for the program.
Maddie:00:03:59
Yeah. So as you mentioned, surveying does have a huge limitation. And that is due to the fact that to touch your customers in the way that traditional CX is structured, you need to get a survey response from them. And at ServiceNow we know that that is a minority. And so we wanted to be able to touch that silent majority. For us, that’s over 70% of accounts. So we don’t hear from at all.
Pat:00:04:28
Yeah.
Maddie:00:04:28
And so we knew we had a a massive blind spot. And we knew that when we hear from customers and we do follow up with them, it has a massive impact on feature satisfaction and the bottom line. And so that’s where this predicted program comes into play, is we we built a really robust NPS program at ServiceNow. We know it has massive impact on satisfaction and future bottom line numbers. So how do we tap into that for that silent majority. And that’s where predicted comes into play is we now are able to predict their satisfaction and still touch those accounts that we never hear from at all.
Pat:00:05:11
Yeah. So, okay, I’m kind of putting myself in the role of maybe one of the leaders in your organization that would say, okay, interesting, but that sounds like magic to me. You know, how are you…
Maddie:00:05:24
Yeah.
Pat:00:05:24
…going to pull that off? Did you have to get some buy in from some of your leaders?
Maddie:00:05:29
When you hit the nail on the head before we even started. So that was pretty much the first thing that we did from a planning perspective. We didn’t dream up the model. We didn’t think about how to build the program from a tactical perspective. The first thing that we did was we knew we needed to establish buy in for this program, because to your point, it is sort of a nebulous concept. I think a lot of times we hear about, oh, predicting satisfaction or, you know, how do we understand the customer without them even telling us? But what we needed to do was show leaders that this is really something that will impact the bottom line to get them on board. And so the way that we establish that buy in with leaders who we needed to get on board was by showing them a dollar number. It was a really simple case. It was honestly one slide. And we all know that dollars speak louder than anything else, and what we were able to show them is, hey, if we implement this program, if we follow up with these customers who are not hearing from, we’ll have millions of dollars of impact a year. And when you make that clean, easy case, it’s a really pretty simple buy in conversation. But we had to get to those numbers in order to make it happen, and we had to do that before we started building anything at all.
Pat:00:06:54
Yeah. And you know, again, just because I know a little bit about your program, you had established a pretty, um, complex close the loop program already. So…
Maddie:00:07:07
Yes.
Pat:00:07:07
…I’m guessing, I mean, you had some credibility that, hey, we’ve got this system, and I know you had at least two dozen or so different closed loop operations in place. It’s really pretty impressive. But, uh, I guess, did that help in kind of convincing that we’re going to build on this and do something even better?
Maddie:00:07:29
Very much so. I would say if you don’t have a robust CX program in place already, I would strongly encourage you to start there, because you really can’t build a predicted NPS program and a closed loop motion if you don’t already have the foundation in place with an existing NPS program. And we were fortunate to have that at ServiceNow. My amazing colleagues, I can’t take credit for this, had built an extremely strong foundational NPS program, and we had really great adoption of that program. And across our key programs, we’re at over 90% closed loop. And to your point, Pat, because we have that foundation, getting teams to kind of add this on as an extension also made it a much easier sell to leadership, but also made adoption much easier because they are so used to these motions with our other closed loops across our other key programs.
Pat:00:08:31
Yeah. Okay. So you’re trying to identify these customers that are at risk for dissatisfaction. Tell us about how you went about setting up a predictive model for that.
Maddie:00:08:43
Yeah, I think that’s the that was kind of the the most fun part for me of this process. We had to get to a place where we could actually predict whether a customer was at risk or not. And so we went about that by taking over 500 operational variables that we have across customers, regardless of if they respond or not. So what those look like for ServiceNow, a B2B software company where things like product adoption data, usage data, support case data, event attendance data. So all these different avenues that we know are really critical to a customer’s experience with us. We pulled in these variables and we were able to train the model again with our NPS data. So you need that strong foundation and existing relationship program. And we were able to train the model with our existing NPS data to understand for accounts that we don’t hear from with these 500 plus operational variables, how can we predict if they’re at risk for dissatisfaction or not? And we were able to build a model with that and make a really accurate model, because we had that NPS data to train it with. And so once we had that model and we knew it was accurate, we needed it to be actionable as well. And so that’s where thinking through the output was really important, because telling an account team that an account is at risk for dissatisfaction is not the most helpful thing. So our model does that. But then it also gives the top three reasons why in rank order. And so that’s really how we built that, that foundational model to our program is we needed it to be accurate, but we also needed it to tell us why. And those…
Pat:00:10:36
Yeah.
Maddie:00:10:36
…were the key things that we thought through with building the model.
Pat:00:10:39
Yeah. So again, if I’m following it correctly because this gets pretty complex, it’s basically even if they didn’t respond, if they meet certain criteria, that and you’re comparing it to people that did respond, you know, and if they, you know, check these boxes that were checked by the people that did respond, it’s, you know, you’re pretty safe to assume that something that they’re dissatisfied in some way, shape or form and should receive some outreach. Right.
Maddie:00:11:13
You’re exactly right. Yeah. It becomes a really strong model, because what you’re able to do is take all this data that you do have about your accounts and understand where the biggest risk areas are, and that makes things really easy for your account teams, because the biggest problem that we have is that they have too much, too much data to sift through. And what you’re able to do with this is say, okay, we know for these silent accounts that there’s a risk here, and we know that these three areas are the ones that you should pay attention to. So you’re you’re really giving them kind of like a gift by saying, you know, here in rank order are the three things you need to you need to think about for your account.
Pat:00:12:00
Yeah. And it’s, uh, it almost goes without saying, but since we haven’t said it yet, I will say it. A business like yours, a, you know, cloud based, you know, platform is totally dependent on renewals and clients sticking with you. It’s you can’t have a lot of churn. So I you know, I’m just thinking there may be people out there saying, yeah, okay, but it’s okay if I don’t have 100% retention. Well, in your business, retention numbers are critical, right?
Maddie:00:12:31
Very. Yes,
Pat:00:12:33
Yeah. Yeah,
Maddie:00:12:33
Exactly.
Pat:00:12:34
Yeah.
Maddie:00:12:34
Yeah.
Pat:00:12:45
So all of this then, is heavily dependent on your front line and how they use this information. Um, again, I’m putting myself in their shoes and uh, and another one of your stakeholders and saying, okay, so you’re telling me that they didn’t really give you feedback directly, but you think they’re not happy right now? How am I supposed to handle that? And obviously, part of it is you gave them some reasons, but did you have some skeptics? And what kind of training did you do to make sure that, uh, you would have the kind of adoption that would be necessary?
Maddie:00:13:24
Yes, and to be totally transparent, the most challenging part of this whole process, I think everyone thinks like, oh, it’s building the model. No, it’s not building the model. The model is black and white. It probably took us a month or two to build the model. We have an incredible data science team. So that that was definitely part of the reason that that it was not as challenging. But the most challenging part is actually getting your account teams, getting the front line to act on the information that you’re feeding them. Because building something incredible and getting teams to act on it are unfortunately two very different things.
Pat:00:14:03
Yeah, you got people involved, you know.
Maddie:00:14:06
Exactly.
Pat:00:14:07
If you didn’t have people to worry about, you know, it might…
Maddie:00:14:10
It just…
Pat:00:14:10
…be easier.
Maddie:00:14:10
…makes things tricky. And I think for us, what was kind of the worst case scenario that we were really working to, to solve for is we build this incredible program, but we don’t get any action from it. And we know that doing that there’s there’s really no point. And so that’s why coaching and training pretty much was the most intensive part of this process for us. And I can’t stress enough how important that is to make something like this successful, because you do need to get your front lines on board. Because this concept is nebulous, you’re not giving them a survey response from a customer, you’re giving them a prediction. And that’s a new muscle for our account teams. It’s not something that they’re used to doing. They’re used to getting that direct feedback. And so what we really focused on is educating them on how this is most helpful to them in terms of making their lives easier. They don’t need to sift through all of this data anymore. They’re able to just get the three areas that they need to focus on. We also really focused on using their language, so helping them understand with those same ROI numbers that we used for buy in, how this is going to directly affect the bottom line of their accounts. And then lastly, we built extremely comprehensive enablement guides because we wanted to make this program as actionable as possible. And so the way that we orient and guide them is by saying that this is a strategic conversation, you’re never going to have 100% accuracy. It is a prediction. But we do know the data behind your accounts is showing this. And you can argue with the data. And so what we do is we give them a guide which for each area that is a risk for dissatisfaction for their given accounts, outlines everything they would need. So from…
Pat:00:16:17
Wow.
Maddie:00:16:18
…the data behind it to the questions they can ask their customer when they get on the phone with them, to the actions they should drive for a given risk area. So we really handhold and probably err on the side of giving too much information, because what that allowed us to do was make this as actionable as possible for our account teams.
Pat:00:16:38
Yeah. No. Really impressive. And I think, you know, the, the whole point of, uh, it’s not the technology, it’s not the data. It’s it’s, you know, how it’s put to use. And I think that, um, I don’t know, we, we see evidence over and over any time we kind of study it that taking action, you know, getting people to actually use information is still the biggest challenge for CX professionals. So so, you know, I have to ask any success stories that you can share that, uh, you know, where they followed up and, uh, customers, you know, it clicked and you were able to see the results that you were looking for?
Maddie:00:17:20
And what we do and actually the the kind of as we after we launch this program, what we really focused on post launch was how do we ensure continued adoption and measure the impact of this program? Because we started with a pilot and we wanted to expand this across the business, but we needed success stories and we needed proof points in order to do that. So thankfully, we do have both kind of high level success of we were able to measure for the pilot program the impact on the bottom line. So we looked at accounts that actually close the loop and compared them to those that got predicted NPS alerts but did not close the loop. And we see higher ACV, higher gross retention rates and really strong impact on bottom line. And then we also see within individuals, we did a lot of interviews with our front lines because this was a new program. We wanted to make sure it was working. We we’ve really been focused on iterating, making it better for them. And within those conversations, we’ve gotten some really wonderful success stories where we see, for instance, we had one account where our front line had a call with them, and she was based for this account. We were flagging things like product adoption issues as well as other usage related issues. And what that account team was able to do was get on the phone with them, understand the root cause of some of those issues. And she knew these things were issues, but she just didn’t know how big. And this call allowed her the space to have a conversation about those and make a plan for implementation on some of those features that they weren’t using. And we looked at that account six months post her having that conversation. And again, at that individual level, we’re seeing higher ACV, higher higher metrics there as well. So we see it both in aggregate as well as some really strong individual success stories.
Pat:00:19:22
Mm. That’s that is great. That is great. Now, in, uh, the article you wrote and posted on LinkedIn, uh, you mentioned predicted NPS is not a set it and forget it program. So…
Maddie:00:19:36
Yes.
Pat:00:19:36
Yeah. What have you done to just keep it going?
Maddie:00:19:38
Yeah, I think sometimes you wish it was right, but it’s just it’s something that we continue to innovate with. And Walker, as our partner, knows this, I think every month really we’re making changes to improve the program. And we do this by we kind of understand where we need to make changes in two main ways. The first is through interviews, what I what I mentioned, but the second is we review the closed loop notes within Qualtrics to understand how is this going, what are the action plans people are taking? And we can glean best practices and update our guides and things like that with those. And one of the things that we learned through this process is that our account teams want more context into the risk, the dissatisfaction, risk reasons that we’re surfacing. And so we are integrating with gen AI to provide them context within their tickets with the reasoning behind a given risk indicator. So we’re actually going to be able to bring in historical context as well as comparison to similar accounts, and just give them all that information within one place. Whereas before we set them to the guide which linked them to the relevant data, and this will just save them even more time. So we are definitely continuing to innovate. And Gen AI is is the buzzword at the moment for a reason. And that’s kind of our our next frontier for the program.
Pat:00:21:05
Well, it’s all really, really impressive. Um, it’s great stuff. And I think I’m sure our listeners will find all of this, uh, really impressive and delivered on kind of what I said at the beginning. You said you’d be willing to share it so that they can do it, too. So.
Maddie:00:21:22
Yes.
Pat:00:21:23
So we’ve come to that point where we ask for take home value. So this is one tip, ideally, that our listeners, you know, something that they can put to use right away. So Maddie Clark, what is your take home value.
Maddie:00:21:37
I think I can’t stress enough just how important it is to get teams to actually act on this data, and I know I said it, but I’ll say it again because it really was the most important thing for making this program successful. For us, getting a model up and running, getting results is one part of the problem. But the more challenging thing for us was getting our frontline teams to actually act on the data. So the more enablement you can do, the more handholding you can do, the easier that you can make this for your frontline teams, the better. Because getting a model is half the battle. But the really more challenging part is getting people to act on it. And what’s the point of standing up one of these these programs if you don’t get your teams to act on it?
Pat:00:22:31
Well said, well said. Great advice. Uh, Maddie is, uh, if somebody wants to continue the conversation, is it okay if they reach out to you on, I assume LinkedIn would be the way to communicate with you?
Maddie:00:22:43
Yes, yes, please reach out. Don’t be shy. I clearly love talking about this topic and would be happy to connect with anyone on LinkedIn.
Pat:00:22:51
Well, your passion shows and we appreciate it. Maddie Clark is a Customer Experience Insights senior manager at ServiceNow. Maddie, thanks so much for being a guest on The CX Leader Podcast.
Maddie:00:23:04
It’s my pleasure. Thanks for having me.
Pat:00:23:06
And if you want to discuss this topic with one of our experts at Walker, or you have a great idea for a topic for a future episode, email us at podcast@walkerinfo.com. We’d love to hear from you! Be sure to rate The CX Leader Podcast through your podcast service and leave a review. Your feedback will help us improve the show and deliver the best possible value to you, our listeners. Check out our website cxleaderpodcast.com. From there, you can follow the show, find all of our previous episodes, and link to our blog, which we update regularly. The CX Leader Podcast is a production of Walker. We’re an experience management firm that helps companies accelerate their XM success. You can read more about us at walkerinfo.com. Thank you for listening and remember, it’s a great time to be a CX leader. We’ll see you next time.
Tags: Pat Gibbons ServiceNow feedback closed loop Maddie Clark