Adapt issue detection to your use case

This guide shows you how to customize issue detection for your domain.

The default configuration detects general issue types like fabrication and reasoning errors. If your domain has specific terminology, compliance requirements, or error types that matter to your workflow, you can customize both the detection labels and the context the detector uses. For example, a pharmaceutical application might need labels like "dosage_error" or "contraindication_omission", while a legal application might track "citation_missing" or "precedent_misattribution".

Prerequisites

• A Blue Guardrails account with a workspace
• A workspace role of Contributor or higher (see Roles and permissions)
• Traced conversations in your workspace (to verify your changes)

Open the issue config

Click Issue Config in the sidebar.

The page shows your current configuration. If you haven't customized it, you'll see the platform defaults: Fabrication, Context Misattribution, Reasoning Error, and Incorrect Refusal.

Your custom configuration only applies to the current workspace. If you have multiple workspaces, configure each one separately.

Add domain context

Domain context helps the detector understand what constitutes an issue in your specific field.

1. Find the Domain Context panel on the left.
2. Enter text that describes your domain and what counts as an issue. Be specific about terminology, sources, and error types that matter.
3. Click Save Changes.

Example for a pharmaceutical assistant:

This is a medical Q&A assistant reviewing pharmaceutical SmPC documents. Issues include incorrect drug dosages, made-up medical conditions, citing non-existent studies, or misattributing side effects to wrong medications. Be strict about dosage precision and contraindication completeness.

Configure custom labels

Labels define the categories of issues the detector looks for. Each label has a name and a description.

1. Find the Label Configuration panel on the right.
2. If you want to add a label, click Add Label.
3. Enter a name (e.g., "dosage_error") and expand the label to add a description.
4. If you want to remove a label, click the trash icon next to it.
5. Click Save Changes.

Write descriptions that explain what the label means and when it applies. The detector uses these descriptions to classify issues, so be specific.

Example label for pharmaceutical use:

• Name: dosage_error
• Description: Incorrect or fabricated medication dosages, including wrong units, administration frequency, or maximum daily limits that don't match the source document.

Verify your configuration

After saving your configuration, test it with new traces, an experiment, or a live detection request.

1. Send new traces to your workspace, run an experiment on existing conversations, or send a request to the live detection API.
2. Open Dashboard or Conversations in the sidebar (for traces and experiments), or check the response body (for live detection).
3. Look for detected issues. They should now use your custom labels.

If issues aren't being detected as expected, adjust your domain context or label descriptions and test again.