Create Experiment
Run a hallucination detection experiment on your conversations.
Test how a model performs by replaying your traced conversations through it and analyzing the responses for hallucinations. Use this to evaluate models before deploying them or to benchmark your current setup.
The experiment runs asynchronously. Check the status with the GET /experiments/{experiment_id} endpoint.
Header Parameters
Workspace UUID to operate within. Required for session auth and user-scoped API keys. Optional for workspace-scoped API keys (uses key's workspace if omitted).
Request Body
application/json
Name of the experiment
1 <= length <= 255Target model to run inference with
1 <= lengthIf true, copy existing responses instead of running inference. Useful for re-evaluating existing conversations without regenerating responses.
falseFilters for selecting conversations
{}Response Body
application/json
application/json
curl -X POST "https://loading/v1/experiments" \ -H "Content-Type: application/json" \ -d '{ "name": "string", "model": "string" }'{
"id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
"name": "string",
"model": "string",
"pass_through": true,
"filter_snapshot": {},
"created_at": "2019-08-24T14:15:22Z",
"completed_at": "2019-08-24T14:15:22Z",
"created_by": "string",
"total_items_count": 0,
"completed_items_count": 0,
"status_summary": {
"pending": 0,
"in_progress": 0,
"inference_completed": 0,
"completed": 0,
"failed": 0,
"total": 0
},
"metrics": {
"hallucination_rate": 0,
"total_hallucinations": 0,
"cost_per_1k": 0,
"hallucination_labels": {
"property1": 0,
"property2": 0
},
"total_input_tokens": 0,
"total_output_tokens": 0,
"avg_input_tokens": 0,
"avg_output_tokens": 0,
"avg_hallucinations_per_prompt": 0,
"hallucination_free_count": 0,
"total_generation_time_ms": 0,
"avg_response_latency_ms": 0,
"max_response_latency_ms": 0
}
}{
"detail": [
{
"loc": [
"string"
],
"msg": "string",
"type": "string"
}
]
}