Skip to content
Evonic AI

API: Test Management

All test management endpoints are under /api/settings/.

Domains

Section titled “Domains”

List Domains

Section titled “List Domains”
GET /api/settings/domains
{
  "domains": [
    {
      "id": "math",
      "name": "Mathematics",
      "description": "...",
      "evaluator_id": "two_pass",
      "enabled": true
    }
  ]
}

Get Domain

Section titled “Get Domain”
GET /api/settings/domains/<domain_id>

Create Domain

Section titled “Create Domain”
POST /api/settings/domains
Content-Type: application/json


{
  "id": "my_domain",
  "name": "My Domain",
  "description": "Custom evaluation domain",
  "evaluator_id": "keyword",
  "system_prompt": "You are a helpful assistant.",
  "icon": "brain",
  "color": "#3B82F6"
}

Update Domain

Section titled “Update Domain”
PUT /api/settings/domains/<domain_id>

Delete Domain

Section titled “Delete Domain”
DELETE /api/settings/domains/<domain_id>

Deletes the domain and all its tests and level configs.

Levels

Section titled “Levels”

Get Level Config

Section titled “Get Level Config”
GET /api/settings/levels/<domain_id>/<level>

Update Level Config

Section titled “Update Level Config”
PUT /api/settings/levels/<domain_id>/<level>
Content-Type: application/json


{
  "system_prompt": "Level-specific instructions...",
  "system_prompt_mode": "append"
}

Tests

Section titled “Tests”

List Tests

Section titled “List Tests”
GET /api/settings/tests?domain=math&level=1

Both domain and level are optional query parameters.

Get Test

Section titled “Get Test”
GET /api/settings/tests/<test_id>

Create Test

Section titled “Create Test”
POST /api/settings/tests
Content-Type: application/json


{
  "domain_id": "math",
  "level": 1,
  "name": "Addition Test",
  "prompt": "What is 2 + 2?",
  "expected": {"answer": "4", "type": "numeric"},
  "evaluator_id": "two_pass"
}

Update Test

Section titled “Update Test”
PUT /api/settings/tests/<test_id>

Delete Test

Section titled “Delete Test”
DELETE /api/settings/tests/<test_id>

Move Test

Section titled “Move Test”
POST /api/settings/tests/<test_id>/move
Content-Type: application/json


{"domain_id": "reasoning", "level": 3}

Evaluators

Section titled “Evaluators”

List Evaluators

Section titled “List Evaluators”
GET /api/settings/evaluators

Get Evaluator

Section titled “Get Evaluator”
GET /api/settings/evaluators/<evaluator_id>

Create Evaluator

Section titled “Create Evaluator”
POST /api/settings/evaluators
Content-Type: application/json


{
  "id": "my_regex",
  "name": "My Regex Evaluator",
  "type": "regex",
  "extraction_regex": "(\\d+)",
  "uses_pass2": false
}

Update / Delete Evaluator

Section titled “Update / Delete Evaluator”
PUT /api/settings/evaluators/<evaluator_id>
DELETE /api/settings/evaluators/<evaluator_id>

Tools

Section titled “Tools”

List Tools

Section titled “List Tools”
GET /api/settings/tools

Get Tool

Section titled “Get Tool”
GET /api/settings/tools/<tool_id>

Create Tool

Section titled “Create Tool”
POST /api/settings/tools
Content-Type: application/json


{
  "id": "my_tool",
  "name": "My Tool",
  "description": "Does something useful",
  "function": {
    "name": "my_tool",
    "description": "Does something useful",
    "parameters": {
      "type": "object",
      "properties": {
        "input": {"type": "string"}
      },
      "required": ["input"]
    }
  },
  "mock_response": {"result": "mock output"}
}

Update / Delete Tool

Section titled “Update / Delete Tool”
PUT /api/settings/tools/<tool_id>
DELETE /api/settings/tools/<tool_id>

Import / Export

Section titled “Import / Export”

Export All Definitions

Section titled “Export All Definitions”
GET /api/settings/export

Returns a JSON dump of all domains, tests, evaluators, and tools.

Import Definitions

Section titled “Import Definitions”
POST /api/settings/import
Content-Type: application/json


{ "merge": true, "domains": [...], "tests": [...] }

Set merge: false to overwrite instead of merging.

Sync to Database

Section titled “Sync to Database”
POST /api/settings/sync

Re-syncs JSON files on disk to the database cache.