How to assess a chatbot background-02
How to assess a chatbot background-02

How should a chatbot be assessed?

How do you assess a chatbot? It's time to clear up something here we feel. We have seen an increasing number of comments from companies that have tested our customers' chatbots and are unhappy with what they see. Why is that so? What are they looking at? 

Here we describe three mistakes that often occur when assessing chatbots. Are you guilty? It's all right. We also explain further down how to do it correctly.  

1. It is believed that a chatbot can answer ALL questions

The most common reason is that the people who tested the chatbot wrote a question that has nothing to do with the company in question. 

Smart home speakers
A chatbot doesn't work the same way virtual assistants

An E trade selling electronics tested with the question "Well, why did Biden actually win the election mr. Chatbot?". Here one misjudges that a chatbot is an omniscient entity that knows the answer to the question "what is the meaning of life?" (the answer to that question is 42 for those wondering). In other words, one expects functionality similar to the general understanding Google Home, Alexa, and Siri are associated with.

Here it is important to understand that the chatbot's purpose is not to be your personal assistant. A chatbot's purpose is to help you with issues related to the organization you are currently visiting. Therefore, a chatbot is often specifically trained to exemplary solve the most common questions their customers ask." Where is my order?" is therefore a much more reasonable question to ask a chatbot for an e-commerce. It is because the majority of actual customers behave in the way that we can achieve over 50% resolved cases with customers in this category

Questions on that? 

2. You treat a chatbot like a shop assistant

Some who test chatbots have thought a step further. They write a question that is relevant to the organization, but the problem is that you bake too much into a single issue. You talk as incoherently and ambiguously as you would in a store when you have not really been prepared to be asked if you need help. The question then looks something like this:

Look, I was thinking about this cord skidding. A cord is a cord, but I thought. Well, that's right, shavers. They need a cord sometimes. When charging. I have a new shaver. And it's corded. What cord is it anyway?

Sure, a good chatbot should also be able to recognize these formulations — you might think so. The truth is, technology hasn't quite gotten that far yet. But that doesn't necessarily mean that the concept of chatbots doesn't work. More on that later. 

3. You would expect it to be a chatbots job to solve all kinds of cases

You have repeatedly contacted customer service because your new glittery eyewear case was broken on delivery. That's enough, you're thinking. They have never reconnected and the case is there on the kitchen table and osar missed customer promise. You fold onto the company's website and start writing with the chatbot that greets you warmly welcome. 

In this situation, the chatbot recommends that it will probably be best if you talk to a human being anyway. This is not a failure of the chatbot. The chatbot exists for this very reason. Taking care of customers like you. 

What you notice is that you hardly have to wait. You'll be paired with a human agent in just a minute. 

The agent says, "Oops, it looks like Rosa has put your case on the court as closed even though it wasn't resolved. Rosa doesn't work with us anymore and you should of course have compensation." 

End good, all good. What about the chatbot? While talking to the agent, the chatbot has helped 10 or so other customers with lighter questions. Customers who would otherwise have been in line and made sure you had to wait longer. 

So no, a chatbot shouldn't do all the work. A chatbot plows the path for agents to help customers faster and more efficiently. 

What have we learned? It's easy to misjudge chatbots. You start writing with it and react with the same facial surface pressure that toothpaste / caviar combo causes when the chatbot can not answer why Lady Gaga wears strange clothes. 

Instead, imagine a real customer situation. When do you usually contact customer service? You may have received the wrong amount on your avi. You may be wondering where your order has gone. You may want to know how to apply for a job at the company. 

If you talk to the chatbot from this perspective, you are more likely to get help. If you do not get help despite this, the chatbot is also more likely to be poor. That said, sending a chatbot over you to an agent also doesn't mean it has failed. Ask yourself instead the questions afterwards: Did I get help? How fast did it go? Will I shop here again? These are the issues customer service should focus on.

How can it work for you?

Get a live demo on the web where we go through the tool, and explore what processes Ebbot could automate in your business.


Share This Post

Share on facebook
Share on linkedin
Share on twitter
Share by email

Read more

Autocorrect spelling mistakes in Swedish language

Understanding that typos can be one of the reasons why Ebbot cannot give the right response, our NLP team at Hello Ebbot decided to develop a new feature to autocorrect spelling mistakes, specifically for Swedish language! Our spellcheck corrector not only considers context to provide better correction, but also has fixed performance.