Answer Quality

See how well your AI agent answers customer questions and find areas to improve.

How to get there: Go to Setup in the top menu, click AI Agent, then click your chatbot. Open the Training tab and select Answer quality.

What it measures

Answer quality analyzes how well your AI answered real customer questions. After each conversation, Boei scores each question-answer pair on a 0-100 scale:

  • 80-100 means the answer was accurate and helpful.
  • 60-79 means the answer was correct but could be more specific.
  • Below 60 means the answer was incomplete or missing important information. These appear as knowledge gaps.

Summary

At the top you see:

  • How many questions were analyzed.
  • How many were answered well (scored 60 or above).
  • The percentage answered well.
  • How many knowledge gaps were found.

If there is not enough data yet, you will see a message instead of percentages.

Knowledge gaps

This is the most useful section. Each row shows a customer question topic where the AI struggled:

  • Question topic is the normalized customer question.
  • Occurrences shows how many times this question came up.
  • Average score shows how well the AI handled it.
  • Unresolved shows how many related conversations were not resolved.
  • Handoffs shows how many were escalated to a human.

What you can do

For each knowledge gap, you have two actions:

  1. View conversations opens the Inbox filtered to conversations from this chatbot, so you can read the actual exchanges.
  2. Add exact answer opens the Exact Answers page where you can write a specific answer for this question. When the AI encounters this question again, it will use your exact answer.

Focus on gaps that:

  • Appear frequently (high occurrence count).
  • Have unresolved conversations.
  • Have a very low average score.

Quality by tag

If you have training tags enabled, this section shows answer quality broken down by tag. This helps you identify which knowledge areas need more content.

Questions answered well

A collapsed section showing topics your AI handles well. Expand it if you want to see what is working.

Period

Use the period selector to view answer quality for different time ranges. The default is Last 7 days. Available options: Today, Last 7 days, Last 30 days.

Tips for improvement

  1. Start with the most frequent knowledge gap.
  2. Read the actual conversations to understand what customers are really asking.
  3. Add an exact answer if the question has a specific, definitive response.
  4. Add or update your website pages or training content if the topic needs broader coverage.
  5. Check back after a week to see if the gap has improved.