Monitor your AI application for hallucinations

This guide shows you how to monitor your AI application's hallucination behavior using the Live Dashboard and Conversations view.

Use this feature to monitor hallucinations in your RAG application or agent as they happen. You can track metrics over time to catch regressions or verify improvements. When issues arise, drill down into specific conversations to perform error analysis.

Prerequisites

• A Blue Guardrails account with a workspace
• An AI application sending traces to Blue Guardrails

Check your hallucination rate

Open your workspace and click Dashboard in the sidebar.

Blue Guardrails evaluates assistant messages for hallucinations. Each message can contain multiple hallucinations. The hallucination rate is the percentage of evaluated messages that have at least one hallucination.

The stats band at the top shows key metrics:

• Hallucination rate - percentage of evaluated messages with at least one hallucination
• Total hallucinations - total count of detected hallucinations across all messages
• Evaluated messages - number of assistant messages analyzed

If you need a different time range, use the time selector in the top right. You can choose from 15 minutes to 14 days.

Identify problematic messages

Look at the waffle chart below the stats band. Each cell represents a message or group of messages.

• Red cells indicate messages with hallucinations.
• Click a cell to see the messages it represents.

Check the Hallucination types chart on the right to see which categories appear most often.

Scroll to the bottom of the page to see a list of messages as they come in. Click any message to open it in the Conversations view.

Review specific conversations

1. Click Conversations in the sidebar.
2. If you want to narrow down results, filter by date range using the date picker.
3. Browse the conversation list in the left panel.
4. Select a conversation to view its messages in the right panel.
5. Look for pink underlined text in assistant messages. These are detected hallucinations.
6. Hover over an annotation to see:
   • The hallucination type (e.g., "Fabrication")
   • An explanation of why it was flagged

Verify a hallucination

If you want to check whether flagged text is actually a hallucination, use the search box to find that text in earlier messages. If the text doesn't appear in user messages or tool results, the assistant fabricated it.

1. Select a conversation with a flagged hallucination.
2. Note the text marked as a hallucination.
3. Use the search box above the messages to search for that text.
4. Check if it appears in any source messages (user input, tool results, or system prompts).