> Documentation index: [/llms.txt](/llms.txt). Every page as one file: [/llms-full.txt](/llms-full.txt). # Run the gym Run the gym [#run-the-gym] Start a gym session on an agent with an environment, follow it, then read its scores. Endpoints [#endpoints] All paths start with `/api/v1/diagnostics` and need `X-Space-Slug`. | Method | URL | Does | Auth | | -------- | ------------------------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------- | ------------- | | `POST` | `/agents/{agent_id}/gym/` | Start a session; billed | Key or bearer | | `GET` | `/agents/{agent_id}/gym/` | The agent's sessions | Key or bearer | | `GET` | `/gym/` | Every session the caller can see, newest first; `input_search` matches a case's input | Key or bearer | | `GET` | `/gym/{session_id}/` | Status, best score, rounds with a report in `epoch_reports` | Key or bearer | | `GET` | `/gym/{session_id}/epochs/{epoch_index}/kpis/` | Scores of one round; `404` until its report is stored | Key or bearer | | `GET` | `/gym/{session_id}/epochs/{epoch_index}/report/` | What happened in one round: each case's outcome and conversations | Key or bearer | | `GET` | `/gym/{session_id}/conversations/` | `[{epoch, row_index, replica, chat_id, …}]` | Key or bearer | | `POST` | `/gym/{session_id}/cancel/` | Stop the session, keeping the best so far; `202`, `409` once it has ended | Key or bearer | | `POST` | `/gym/{session_id}/decision/` | Answer a session paused at `awaiting_approval`: `{"decision": "accept" \| "edit" \| "skip" \| "stop", "patch"}`; `patch` only with `edit` | Key or bearer | | `POST` | `/gym/{session_id}/promote/` | Save the best configuration as a new agent; optional `{"name"}` | Key or bearer | | `POST` | `/gym/{session_id}/reject/` | Discard the improved copy of the agent | Key or bearer | | `DELETE` | `/gym/{session_id}/` | Delete the session and its copy of the agent; `204`, `409` while it runs | Key or bearer | Request fields [#request-fields] | Field | Rule | | ------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------ | | `kit_id` | The environment to run. Without it, send `dataset` (1–25 rows) and at least one check in `assertions` | | `file_bucket` | Without `kit_id` only: the files the cases hold, `[{file_id, description}]`; with `kit_id` the environment's own are used | | `lane` | `"baseline"` measures once; `"optimization"` (default) measures, changes the configuration and measures again | | `n_runs` | How many times each case is played, 1–50. Default: from the effort level | | `judge_votes` | How many times a judge scores each check, 1–9; the majority wins. Default: from the effort level | | `max_epochs` | Optimization only: rounds, 1–50. Default: from the effort level | | `objective` | Optimization only: `"accuracy"` (default) or `"cost"`, the same quality at a lower price | | `min_delta` | Optimization only: the smallest gain, 0–1, that counts as an improvement | | `applyable_levers` | Optimization only: what it may change, e.g. `["system_prompt"]` or `["tools", "skills"]`. Default: everything | | `mode` | Optimization only: `"auto"` (default) applies each change; `"gated"` pauses at `awaiting_approval` before each one | | `max_cost_usd` | Stop once the session has cost this much | | `turn_time_limit_s` | Seconds one agent turn may take, 60–3600, default 600. A turn over it ends the conversation there, and that run fails without being judged | | `max_turns` | Messages each simulated person sends at most, 1–20, the same for every case. Default: the highest any case asks for | | `diagnostic_effort` | `low`, `medium` (default) or `high`; sets the models and default counts. A default you saved for a level in the web app applies to your sessions | * The environment must belong to the agent, to the agent it was promoted from, or to no agent. * A session plays cases × `n_runs` conversations per round; an optimization repeats that for each round. Measure or improve an agent [#measure-or-improve-an-agent] 1. Start the session: ```bash curl -X POST "$CENTRAL_HUB_URL/api/v1/diagnostics/agents/42/gym/" \ -H "X-API-Key: $ALO_API_KEY" \ -H "X-Space-Slug: $WORKSPACE_SLUG" \ -H "Content-Type: application/json" \ -d '{"kit_id": 7, "lane": "baseline", "n_runs": 3}' ``` The response holds `id`, `status` and, while it waits for a free slot, `queue_position`. 2. Poll `GET /gym/{id}/` about once a minute until `status` is `completed`, `cancelled` or `failed`; see [Statuses](#statuses). 3. Read a round's scores; a baseline has round `1`, an optimization lists its rounds in `epoch_reports` and its best in `best_epoch`: ```bash curl "$CENTRAL_HUB_URL/api/v1/diagnostics/gym/89/epochs/1/kpis/" \ -H "X-API-Key: $ALO_API_KEY" \ -H "X-Space-Slug: $WORKSPACE_SLUG" ``` The response holds `objective_value`, `total_executions`, `errored_executions`, `checks[]` with `{label, passed, evaluated}`, latency and `cost`. 4. After an optimization, `POST /gym/{id}/promote/` saves the best configuration as a new agent, or `POST /gym/{id}/reject/` discards it. Rejecting deletes the gym's copy of the agent and its conversations: read them first. * The gym's conversations belong to its copy of the agent: read them with `fork_agent_id`, as in [Read the conversations](./overview#read-the-conversations-behind-a-report). Statuses [#statuses] | `status` | Means | | ---------------------------------- | -------------------------------------------------------- | | `pending`, `queued` | Waiting to start; a queued session starts on its own | | `running`, `cancelling` | In progress | | `awaiting_approval` | A `"gated"` session waits for `POST /gym/{id}/decision/` | | `completed`, `cancelled`, `failed` | Ended; `failed` carries `error_message` | | `promoted`, `rejected` | An optimization's result was kept or discarded | | `stop_reason` | Why an optimization ended | | ------------------------------------- | ------------------------------------------------------------- | | `max_epochs` | It ran every round | | `perfect_score` | The first measurement already scored 100%; nothing to improve | | `budget_exhausted`, `quota_exhausted` | It ran out of budget or credit quota | | `cancelled` | It was cancelled | Troubleshooting [#troubleshooting] | Response | Cause | Fix | | ---------------------------------------------- | --------------------------------------------------- | -------------------------------- | | `422` `An ad-hoc gym (no kit_id) needs …` | No `kit_id`, and `dataset` or `assertions` is empty | Send `kit_id`, or both | | `409` on promote or reject | The session is not `completed` or `cancelled` | Wait for it to end | | `409` on delete | The session is still running | Cancel it first | | `409` on decision | The session is not `awaiting_approval` | Read its `status` | | `422` `` `file_bucket` is for an ad-hoc gym `` | `file_bucket` sent with `kit_id` | Send one or the other | | `404` on `/epochs/{n}/kpis/` | That round has no report yet, or does not exist | Use a round from `epoch_reports` |