Unresolved Questions Analytics

Find and patch blind spots in your AI Agent's knowledge
Written by Konstantine
Updated 2 months ago

AI-Agents in HelpCrunch are designed to handle a wide range of customer inquiries with high accuracy. However, there are cases where they’re unable to confidently answer a question. In these situations, we label the question as “Unresolved” and take additional steps to ensure your customers still receive the help they need.


What Counts as an "Unresolved Question"?

An unresolved question is any query that the AI Agent couldn’t answer confidently based on the connected data sources (Knowledge Base or Custom Responses). This can happen when:

  • The question is outside the scope of the available data

  • The query is too vague or ambiguous

  • The customer is asking something new or highly specific

Each unresolved question is tracked individually, even if it's part of a larger conversation.

What Happens When a Question Goes Unanswered?

When the AI Agent encounters a question it can't answer, it follows one of these fallback paths:

  • Route to Team Offered - The AI offers to transfer the chat to a human agent. This gives the customer the option to continue the conversation with your support team.
  • 🚫 Route to Team Not Offered - In cases where routing is disabled by specific instructions, or customer decided to continue conversation regardless - the AI does not offer escalation.
The question handling result (e.g., whether a route to team was offered) is tracked per individual unresolved question, while the conversation outcome reflects how the entire interaction ended. These are two distinct metrics shown in the report to provide a more detailed view of performance.

How Did Conversations End?

Each unresolved question is part of a larger conversation. Here's how those conversations typically conclude:

  • ✅ Resolved — The customer’s issue was successfully addressed.

  • 🔁 Routed to Team — The conversation was transferred to a human agent.

  • ❌ Abandoned — The customer left the conversation without resolution.

Here you can see the list of questions, their language and tags.
Each question has two indicators:

  • Route Offered - based on whether AI offered or not for transferring the conversation.
  • Outcome -  🟢Resolved , 🔵Routed , 🔴Abandoned

Tags are automatically generated based on the question and help to easily filter out similar questions from the list.


💡 Tips for Reducing Unresolved Questions

  • Improve your Knowledge Base coverage: Make sure key product and service topics are well-documented.

  • Add Custom Responses for commonly missed queries.

  • Monitor tags on unresolved questions to identify knowledge gaps.

  • Use multiple AI Agents trained for specific domains (e.g., Support, Billing, Product).

Once you have added the information to the data sources that address this particular question, you can mark it as resolved in the list and proceed to the next one.


💡 Why This Report Matters

This report is your go-to tool for identifying blind spots in your AI Agents. It shows you

  • What kinds of questions aren’t being answered.
  • How often customers are being handed off to a team member.
  • Where conversations are falling through the cracks.

By keeping a close eye on these metrics, you’re not just improving automation - you’re improving customer experience.

Did this answer your question?