Run the gym
Start a gym session on an agent with an environment, follow it, then read its scores.
Endpoints
All paths start with /api/v1/diagnostics and need X-Space-Slug.
| Method | URL | Does | Auth |
|---|---|---|---|
POST | /agents/{agent_id}/gym/ | Start a session; billed | Key or bearer |
GET | /agents/{agent_id}/gym/ | The agent's sessions | Key or bearer |
GET | /gym/ | Every session the caller can see, newest first; input_search matches a case's input | Key or bearer |
GET | /gym/{session_id}/ | Status, best score, rounds with a report in epoch_reports | Key or bearer |
GET | /gym/{session_id}/epochs/{epoch_index}/kpis/ | Scores of one round; 404 until its report is stored | Key or bearer |
GET | /gym/{session_id}/epochs/{epoch_index}/report/ | What happened in one round: each case's outcome and conversations | Key or bearer |
GET | /gym/{session_id}/conversations/ | [{epoch, row_index, replica, chat_id, …}] | Key or bearer |
POST | /gym/{session_id}/cancel/ | Stop the session, keeping the best so far; 202, 409 once it has ended | Key or bearer |
POST | /gym/{session_id}/decision/ | Answer a session paused at awaiting_approval: {"decision": "accept" | "edit" | "skip" | "stop", "patch"}; patch only with edit | Key or bearer |
POST | /gym/{session_id}/promote/ | Save the best configuration as a new agent; optional {"name"} | Key or bearer |
POST | /gym/{session_id}/reject/ | Discard the improved copy of the agent | Key or bearer |
DELETE | /gym/{session_id}/ | Delete the session and its copy of the agent; 204, 409 while it runs | Key or bearer |
Request fields
| Field | Rule |
|---|---|
kit_id | The environment to run. Without it, send dataset (1–25 rows) and at least one check in assertions |
file_bucket | Without kit_id only: the files the cases hold, [{file_id, description}]; with kit_id the environment's own are used |
lane | "baseline" measures once; "optimization" (default) measures, changes the configuration and measures again |
n_runs | How many times each case is played, 1–50. Default: from the effort level |
judge_votes | How many times a judge scores each check, 1–9; the majority wins. Default: from the effort level |
max_epochs | Optimization only: rounds, 1–50. Default: from the effort level |
objective | Optimization only: "accuracy" (default) or "cost", the same quality at a lower price |
min_delta | Optimization only: the smallest gain, 0–1, that counts as an improvement |
applyable_levers | Optimization only: what it may change, e.g. ["system_prompt"] or ["tools", "skills"]. Default: everything |
mode | Optimization only: "auto" (default) applies each change; "gated" pauses at awaiting_approval before each one |
max_cost_usd | Stop once the session has cost this much |
turn_time_limit_s | Seconds one agent turn may take, 60–3600, default 600. A turn over it ends the conversation there, and that run fails without being judged |
max_turns | Messages each simulated person sends at most, 1–20, the same for every case. Default: the highest any case asks for |
diagnostic_effort | low, medium (default) or high; sets the models and default counts. A default you saved for a level in the web app applies to your sessions |
- The environment must belong to the agent, to the agent it was promoted from, or to no agent.
- A session plays cases ×
n_runsconversations per round; an optimization repeats that for each round.
Measure or improve an agent
-
Start the session:
curl -X POST "$CENTRAL_HUB_URL/api/v1/diagnostics/agents/42/gym/" \ -H "X-API-Key: $ALO_API_KEY" \ -H "X-Space-Slug: $WORKSPACE_SLUG" \ -H "Content-Type: application/json" \ -d '{"kit_id": 7, "lane": "baseline", "n_runs": 3}'The response holds
id,statusand, while it waits for a free slot,queue_position. -
Poll
GET /gym/{id}/about once a minute untilstatusiscompleted,cancelledorfailed; see Statuses. -
Read a round's scores; a baseline has round
1, an optimization lists its rounds inepoch_reportsand its best inbest_epoch:curl "$CENTRAL_HUB_URL/api/v1/diagnostics/gym/89/epochs/1/kpis/" \ -H "X-API-Key: $ALO_API_KEY" \ -H "X-Space-Slug: $WORKSPACE_SLUG"The response holds
objective_value,total_executions,errored_executions,checks[]with{label, passed, evaluated}, latency andcost. -
After an optimization,
POST /gym/{id}/promote/saves the best configuration as a new agent, orPOST /gym/{id}/reject/discards it. Rejecting deletes the gym's copy of the agent and its conversations: read them first.
- The gym's conversations belong to its copy of the agent: read them with
fork_agent_id, as in Read the conversations.
Statuses
status | Means |
|---|---|
pending, queued | Waiting to start; a queued session starts on its own |
running, cancelling | In progress |
awaiting_approval | A "gated" session waits for POST /gym/{id}/decision/ |
completed, cancelled, failed | Ended; failed carries error_message |
promoted, rejected | An optimization's result was kept or discarded |
stop_reason | Why an optimization ended |
|---|---|
max_epochs | It ran every round |
perfect_score | The first measurement already scored 100%; nothing to improve |
budget_exhausted, quota_exhausted | It ran out of budget or credit quota |
cancelled | It was cancelled |
Troubleshooting
| Response | Cause | Fix |
|---|---|---|
422 An ad-hoc gym (no kit_id) needs … | No kit_id, and dataset or assertions is empty | Send kit_id, or both |
409 on promote or reject | The session is not completed or cancelled | Wait for it to end |
409 on delete | The session is still running | Cancel it first |
409 on decision | The session is not awaiting_approval | Read its status |
422 `file_bucket` is for an ad-hoc gym | file_bucket sent with kit_id | Send one or the other |
404 on /epochs/{n}/kpis/ | That round has no report yet, or does not exist | Use a round from epoch_reports |