How to build a chatbot to automate customer service?

Wouldn’t it be awesome to build a chatbot that can automate customer service? Over at China Admissions we have created a chatbot called Marco Polo. With ChatGPT we have created the second version which has successfully automated 95% of our customer service inquiries.

Why it’s awesome

This is great for our customers because:
– They can get instant replies
– In any language
– The replies are often more comprehensive than our customer service team

This is great for our customer service team because:
– They don’t have to spend their time answering the same questions again and again
– They can spend this time on higher leverage activities: training, creating content, spending more time on higher value customers

This is great for the company because:
– Better service
– Lower cost
– Higher revenue

Our problem

We receive hundreds of emails from students each day. Usually they are asking quite similar questions to understand about their options to study in China. Thankfully, the range of questions they can usually ask about is fairly narrow. It’s a perfect problem to automate with a chatbot.

The problem with chatbots

One of the main problems I have with chatbots is that most of them are crap! They don’t properly understand your question, or they just provide a list of possible options for you to choose. This is just a frustrating user experience. Whenever I find a chatbot I am usually frustrated and prefer to try to speak to a human to answer my question.

Our development story

We first started creating Marco Polo chatbot in 2018. We used dialogflow. We wanted to make a chatbot that used natural language (not giving a list of options to click), to answer questions, just as a person could.

The chatbot was ‘surprisingly intelligent’

We spent hundreds of hours creating this chatbot and we did this by typing in all the combinations of questions that the students. The chatbot consisted of two parts:
– FAQs
– Solution Finder

The FAQs is created by having a list of the most popular frequently asked questions that students can have. Then when they ask a question it will give a solution.

The solution finder starts by evaluating if they are looking for us to give them a solution to their problem – in this case recommending which course they should study. This will fire up a series of questions to then give them a solution. We did this by creating a decision tree that would lead them through.

It worked reasonably well for 80% of questions, but we faced two problems, mainly that it was not intelligent. It didn’t understand a range of questions from natural language if it was phrased differently, and because of this we had to manually write out all the thousands of combinations into the chatbot.

Unfortunately we felt that we couldn’t solve these problems that the chatbot just wasn’t good enough, we felt that it was better to just scrap the whole project, and go back to our customer service.

ChatGPT Chatbot

ChatGPT arrived on the scene in 2022 and has become one of the world’s fastest growing services. This solves the previous problems that we had before:
1) Understanding natural language
2) Understanding complex questions without the need us to put in a large volume of data to solve problems.

How it works?

• The chatbot has the following parts

  1. Prompt

     • The prompt is the instructions that is given to the chatbot. Just by going on chatgpt and writing out the prompt. It should be made of the following parts, usually about 150-400 words. It would include 1) Who is the chatbot?, 2) How do we want them to reply, does it have any specific way we want them to answer such as using the cultural values of it. It would also be important to define what you don’t want the chatbot to do too, for example, nothing political, or outside of the industry, answer in this way. It would be a good idea to let the chatbot know of your company values so you can keep those. We can write a simple version of this in ChatGPT, then write it out onto a word doc that we can load later.

  2. Q&A Examples

     • What are the 50-200 most common questions that your customers ask? Write out the questions, and then how you would answer in the perfect way, and put them into a spreadsheet. Try to make it as consistent as possible, answering in a standard way. Instead of having multiple people creating this, it would be a good idea to have someone go over this to make them very standard so its easier to use. It would be good to have a range of questions that cover the core main topics, then have some real questions from real users. Also make sure to include some controversial questions and topics and edge cases that real students ask or might ask, so it knows how to handle those.

  3. KnowledgeBase

     • The next part would be to create a single document which includes the core information about your services and about your company that you want people to know if they have questions. This can include detailed information about the services, the sign up process, the refund process. You can prepare this by taking this information from your website and putting it all in one simple document. Then making it well structured and making sure there is no conflicting information. It would be important to ensure that this is a high quality document and not just a load of content. What data we feed to the chatbot is really important. Less is better than more.

  4. Sitemap

     • What are the URLs you would like to redirect users to on your website? Make a list of all these in a spreadsheet with one column of the title, one of the description of the page, and one of the URL of the page. If students have questions then the chatbot can direct the users to these pages.

  5. Software

     • Now that we have these documents we can then set up the software for it. We basically need software that will link all these parts together. One stack would be:
       • Coolify server https://coolify.io/
       • ChatGPT https://platform.openai.com/
       • Chatwoot https://www.chatwoot.com/

  6. Training

     • All of the above works for a first version, but after that it’s necessary to constantly improve the chatbot. This can be done by checking the answers it gives and then changing the prompt, adding to the knowledgebase, and adding Q&A examples. This is an ongoing process, it can always be improved.

How much does it cost?

• The technical work to set it depends on the developer. We have spent a lot of time and trial and error in trying different stacks and software to find what works best. This would take longer in a bigger company depending on the development process and how many people would be involved. If we were to set this up again we could set something like this up from scratch in about 20-30 hours.
• There are server costs and there is chatgpt costs. The server costs may be a few hundred dollars per month. The chatgpt costs can be calculated at about 4 cents per question. This depends on some factors such as how much data you have, how long is the prompt, Q&A and knowledgebase as to how many tokens it would use.
• There is also a maintenance cost that is important to consider. That would be a technical maintenance – it is necessary to constantly update the software and ensure it is compliant. It is also necessary to keep an eye on the chats to make sure it is answering correctly and well and to keep training it. This depends on developer and staff costs, but would be about 5-10 hours a month.
• Overall the cost of the chatgpt is less than half the cost it would cost to have a person doing it.

Conclusion

This is exciting technology that is going to change the world. Customer service is one of the first areas that is going to be automated with this technology and companies who do automate customer service with this technology are going to be able to provide a better service at lower cost and have a significant advantage in the market compared to competitors.