Are customer surveys dead? 3 survey limitations and how to overcome them

Survey fatigue is real. As consumers, we’re bombarded with requests to share our thoughts and experiences after making a purchase or contacting customer service. And let’s face it: very few of us have the time or inclination to respond to every survey we receive.

For contact centers and CX leaders, that’s a big problem. 

Businesses use survey data to improve their customer experience (CX), track brand sentiment, identify dissatisfied customers, and measure their customer service team’s performance. But when only a small percentage of customers complete surveys–and the people most motivated to respond had either highly positive or negative experiences–the survey data isn’t a reliable indicator of performance or CX. 

CX and contact center leaders are searching for new ways to measure customer satisfaction and gain more insights. Below, we’ll explore the three biggest survey limitations and how businesses are overcoming them.

Interested in taking a closer look at specific strategies to overcome survey limitations? Check out our eBook: The death of the customer survey.

The 3 biggest survey limitations

Low response rates

One of the biggest struggles businesses have with surveys is getting enough responses. Delighted, a Qualtrics company, did a study in 2021 to measure survey response rates. They found that website surveys had an average response rate of 8%, while email surveys had a rate of 6%. Depending on the size of the customer base, single-digit response rates may not be statistically valid. 

In other words, businesses are having trouble collecting enough data from surveys to understand their customers' experiences and make customer-focused decisions.  

Sample bias

It’s not just that response rates are low–it’s that the people who take surveys aren’t representative of the entire customer base. Aaron Mickelson, VP of Digital at TwinStar Credit Union, spoke about this phenomenon in a Tethr webinar. “As survey fatigue kicks in, the people who take surveys are polarized,” said Mickelson. “Either you get the person with the really good experience or the bad experience. You never get the forgotten middle, which is actually the most important and largest group of your membership base.”

This sample bias creates a "squeaky wheel gets the grease" effect. Businesses often make decisions based on feedback from their most outspoken critics and fans. Unfortunately, they may overlook valuable insights and trends from customers who don't fill out surveys.

Lack of nuance

The benefit of single-number survey metrics like NPS and CSAT is that they are easy to measure and track. The downside is that they don't leave room for nuance in the customer experience.

For example, NPS buckets customers into three groups: detractors, promoters, and neutrals. In reality, customers may change groups depending on the context, such as who they are talking to or which products they are referring to. According to a study in The Harvard Business Review, 52% of people who discouraged others from buying a brand had previously recommended it. But because of how NPS surveys are set up, customers have to pick one response instead of saying "it depends" and giving more information.

CSAT surveys, meanwhile, tell you that customers had a good or bad experience, but they don’t tell you why. While some surveys include open text fields for customers to share more feedback, the response rates for these questions are even lower than for close-ended questions.

Without understanding the why, or the contextual nuances, businesses can’t make informed decisions to move the CX needle. They need to hear more from their customers and uncover deeper insights than they can get through surveys alone.

3 ways to overcome survey limitations with machine learning

If surveys aren’t giving businesses enough information about the customer experience, what’s the alternative?

Customer conversation transcripts–from phone calls, chats, emails, or other service channels–are an often overlooked source of meaningful insights.

As Aaron Mickelson at TwinStar Credit Union put it: “Why do I need to ask my [customers or] members about their experience? They already told me about it in their conversation.”

The challenge is that language is unstructured data, and it’s labor-intensive to manually organize it and extract insights when your company fields hundreds or thousands of customer service inquiries per day.

Fortunately, machine learning technology makes it possible to automate the data mining process and extract CX insights at scale. Below, we’ll look at three machine learning applications that Tethr uses to help our customers get deeper voice-of-the-customer insights.

Conversation intelligence

Conversation intelligence software uses machine learning to analyze the phrases spoken (or written) in customer conversions and determine their meaning. It organizes phrases into categories and delivers reports with actionable customer insights. Contact centers can analyze all their conversations and get more nuanced information about the customer experience than they could get from survey data alone. 

Example: Contact center leaders can use conversation intelligence to identify phrases associated with customer churn. This helps them understand why customers are churning so they can make policy, process, or agent coaching changes to retain more customers.

Predictive CSAT scoring

Tethr’s CSATai is a machine learning model trained on millions of CSAT surveys and their preceding interactions. By ingesting this data, the model learned about the relationships between survey scores and the words or phrases used in interactions. Today, it can reliably predict the satisfaction of every customer who has a customer service interaction, not just those who complete a survey.

Contact center leaders can combine CSATai with Tethr’s conversation intelligence capabilities to better understand why customers are satisfied or dissatisfied.

Example: Contact center managers can look for trends in the language used in conversations with a negative CSAT score. This helps them see what is driving dissatisfaction and what the contact center can control. From there, they can prioritize initiatives to improve satisfaction.

Sentiment analysis

We trained Tethr's sentiment analysis model to identify 28 emotions based on customer and agent phrases. The model rates each conversation with a customer sentiment and an agent sentiment of positive, negative, or neutral. It also identifies individual instances of different emotions within the conversation and shows where they occur in the transcript.

Sentiment analysis helps contact center leaders understand how customers felt about their service interaction–and why they felt that way.

Example: Contact center leaders can look at the individual emotions present in a sampling of conversations with a negative sentiment rating. This can help them identify specific topics or agent behaviors linked to negative emotions, such as frustration or confusion. They can then train agents on how to avoid specific behaviors and better handle the topics associated with negative emotions.

Final takeaways

Conversation intelligence, predictive CSAT, and sentiment analysis expand what your business can learn about your customers’ experiences. For some businesses, surveys may no longer be necessary. Others may choose to augment their survey program with conversation intelligence to gain deeper insights. 

The real value of conversation intelligence isn’t just that it reduces or eliminates the need to rely on surveys. It provides insights into why customers feel the way they do about their experiences with your business. When your business identifies the factors affecting satisfaction, you can make changes to policies, processes, and agent coaching to improve the customer experience.