Is my sample size statistically significant?

The CX Leader Quick Tips Minicast: "How do I know my sample size is statistically significant?": Audio automatically transcribed by Sonix

Download the “The CX Leader Quick Tips Minicast: "How do I know my sample size is statistically significant?" audio file directly.

The CX Leader Quick Tips Minicast: "How do I know my sample size is statistically significant?": this wav audio file was automatically transcribed by Sonix with the best speech-to-text algorithms. This transcript may contain errors.

Chris:
Hello, everyone. I'm Chris Higgins and welcome to the inaugural edition of The CX Leader Quick Tips Minicast, where Walker experts take a few moments to give you helpful advice on how to improve your XM programs. Joining me on this episode of the minicast is Tanner Smith. He's a senior analyst here at Walker. Tanner, thanks for being on the show.

Tanner:
Thanks, Chris. Happy to be here.

Chris:
So what are we going to talk about today in this minicast?

Tanner:
So one of my favorite questions, and that is how do I know my sample size is statistically significant? Another way that I like to ask it, though, is, is our random sample a good representation of our entire population? And before diving into that, though, I did want to add a precursor question to this, and that is why is it important? So as an organization, you've got different types of customers, right? Let's say you want to run an annual relationship survey where you get feedback from all these different segments. For the sake of this metaphor analogy here I'm going to use music. I'm a musician, one of my other passions in life, so it's my go to for this kind of thing. So in this relationship survey that we're running, the goal here is to just hear what the overall tone is of what our customers are singing collectively, right? So musically, I'd call this a chord. What's the chord that all of our customer voices are creating when sung together? Now as a guitar player, that's going to be my go to instrument here. Let's say that the fifth string on your guitar is going to be your biggest customer segment. It's going to be that driving force behind, you know, it's going to be the backbone of the chord that we hear.

Tanner:
So if we listen to what they're singing by themselves, we might get something like the Note C. [Guitar note plays] Now by itself, does that really give us a feel for what is happening, what they're saying? Does that evoke any emotions, thoughts and feelings in us? Does it give us enough information to go on to make decisions that affect all the other segments that we didn't hear from? And what do we assume that what they're singing is going to be the same complimentary of what the other segments are singing? So to get a better idea of this, we do need to add a couple more segments to the mix. So adding a second one, we might get something like this. [Guitar chord plays] Okay, so it sounds like we might have something there. It's obviously harmonious. It goes together, but it still might be a little bit ambiguous in terms of the mood that it's giving off. So we might need to start adding more. And as we start to add more customer segments to our guitar here and layering in these voices, we start adding enough new notes to get a more complete picture of what the chord is as being sung.

Tanner:
And if we accidentally keep a segment excluded from our choir, we might get that ambiguous or incomplete chord. And if we accidentally include a segment or mischaracterize a segment due to poor data quality, we might get an overall chord that sounds like something positive, say like a major chord. [Guitar chord plays] But in reality, our customers are actually singing a minor chord. [Guitar chord plays] And the reason we didn't capture that true minor chord is because we heard from the wrong mix of customer types. And the nuance here is that the major and minor chord are only different by a single note. A single customer group singing a slightly different note, but in the context of the rest of the chord, it completely changed our interpretation of the overall court. And so the shift between a major and minor chord evokes entirely different sets of thoughts and feelings distinct from one another. And it's this different set of thoughts and feelings that go on to drive different attitudes and conversations about how we take action on these results. So that's why it's crucial that we can say we're hearing from the correct mix of customers for our research question and business. Now, it's not a perfect analogy, but I hope that works, Chris.

Chris:
Yeah, that's a great analogy. I'd like to ask a question, though: how do you know what chord to play? How do you know what notes to play in order to get that perfect, harmonious segment of customers?

Tanner:
That's an excellent question. And overall, it's part of our strategic design of the song or the program that we're implementing. We follow a set of structured rules that we've put in place across all of our different instrumentalists or departments, so that way we can all make sure we're landing on the same page to to make the music of successful business. So the next part of this is how do I know my sample is actually statistically significant? Well, this is a bit of a two pronged question. There's the question of how many do we need to make sure that our sample is significant? But then the other prong is, is the distribution of responses in our sample representative of the different segments we have within our population. So the way we go about answering those are slightly different, but tackling the first one first, we're going to look to our margin of error as the primary tool to see if we have enough sample to determine if it's artistically significant. And this takes into consideration the number of respondents in our sample in relation to the overall population and gives us an end result of a percentage that we can use to create a confidence interval. And this confidence interval is going to tell us a couple key things. So going back to our relationship study that we were working on, our song, let's say our global satisfaction score was was 80%. Our confidence interval in that score is going to tell us, given how many we sampled from our population, we obtained an 80% satisfaction score and with 95% confidence, we know that the true score of the population lies somewhere in between 75 and 85%. So now looking at the second part of this whole question, the composition mix between our sample and our population, we can just run some very simple statistic tests to see if we have significant differences between the distribution of our respondents across segments, between that sample and the population to see if it generalizes. All right. So that is everything you might need to know around if your sample is going to be statistically significant or not.

Chris:
Fantastic. And I've just got to say, I am also a musician. So this being the inaugural episode of The Quick Tips, this is just absolutely perfect. Tanner, thanks a lot. If you want to listen to more great advice, go to cxleaderpodcast.com and be sure to subscribe to the show. And if you have a question or would like to hear more on a particular topic from our Walker experts, send us an email at podcast@walkerinfo.com. Thanks for listening.

Sonix is the world’s most advanced automated transcription, translation, and subtitling platform. Fast, accurate, and affordable.

Automatically convert your wav files to text (txt file), Microsoft Word (docx file), and SubRip Subtitle (srt file) in minutes.

Sonix has many features that you’d love including enterprise-grade admin tools, automatic transcription software, automated subtitles, upload many different filetypes, and easily transcribe your Zoom meetings. Try Sonix for free today.