Lab

Run the gym

Start a gym session on an agent with an environment, follow it, then read its scores.

Endpoints

All paths start with /api/v1/diagnostics and need X-Space-Slug.

MethodURLDoesAuth
POST/agents/{agent_id}/gym/Start a session; billedKey or bearer
GET/agents/{agent_id}/gym/The agent's sessionsKey or bearer
GET/gym/Every session the caller can see, newest first; input_search matches a case's inputKey or bearer
GET/gym/{session_id}/Status, best score, rounds with a report in epoch_reportsKey or bearer
GET/gym/{session_id}/epochs/{epoch_index}/kpis/Scores of one round; 404 until its report is storedKey or bearer
GET/gym/{session_id}/epochs/{epoch_index}/report/What happened in one round: each case's outcome and conversationsKey or bearer
GET/gym/{session_id}/conversations/[{epoch, row_index, replica, chat_id, …}]Key or bearer
POST/gym/{session_id}/cancel/Stop the session, keeping the best so far; 202, 409 once it has endedKey or bearer
POST/gym/{session_id}/decision/Answer a session paused at awaiting_approval: {"decision": "accept" | "edit" | "skip" | "stop", "patch"}; patch only with editKey or bearer
POST/gym/{session_id}/promote/Save the best configuration as a new agent; optional {"name"}Key or bearer
POST/gym/{session_id}/reject/Discard the improved copy of the agentKey or bearer
DELETE/gym/{session_id}/Delete the session and its copy of the agent; 204, 409 while it runsKey or bearer

Request fields

FieldRule
kit_idThe environment to run. Without it, send dataset (1–25 rows) and at least one check in assertions
file_bucketWithout kit_id only: the files the cases hold, [{file_id, description}]; with kit_id the environment's own are used
lane"baseline" measures once; "optimization" (default) measures, changes the configuration and measures again
n_runsHow many times each case is played, 1–50. Default: from the effort level
judge_votesHow many times a judge scores each check, 1–9; the majority wins. Default: from the effort level
max_epochsOptimization only: rounds, 1–50. Default: from the effort level
objectiveOptimization only: "accuracy" (default) or "cost", the same quality at a lower price
min_deltaOptimization only: the smallest gain, 0–1, that counts as an improvement
applyable_leversOptimization only: what it may change, e.g. ["system_prompt"] or ["tools", "skills"]. Default: everything
modeOptimization only: "auto" (default) applies each change; "gated" pauses at awaiting_approval before each one
max_cost_usdStop once the session has cost this much
turn_time_limit_sSeconds one agent turn may take, 60–3600, default 600. A turn over it ends the conversation there, and that run fails without being judged
max_turnsMessages each simulated person sends at most, 1–20, the same for every case. Default: the highest any case asks for
diagnostic_effortlow, medium (default) or high; sets the models and default counts. A default you saved for a level in the web app applies to your sessions
  • The environment must belong to the agent, to the agent it was promoted from, or to no agent.
  • A session plays cases × n_runs conversations per round; an optimization repeats that for each round.

Measure or improve an agent

  1. Start the session:

    curl -X POST "$CENTRAL_HUB_URL/api/v1/diagnostics/agents/42/gym/" \
      -H "X-API-Key: $ALO_API_KEY" \
      -H "X-Space-Slug: $WORKSPACE_SLUG" \
      -H "Content-Type: application/json" \
      -d '{"kit_id": 7, "lane": "baseline", "n_runs": 3}'

    The response holds id, status and, while it waits for a free slot, queue_position.

  2. Poll GET /gym/{id}/ about once a minute until status is completed, cancelled or failed; see Statuses.

  3. Read a round's scores; a baseline has round 1, an optimization lists its rounds in epoch_reports and its best in best_epoch:

    curl "$CENTRAL_HUB_URL/api/v1/diagnostics/gym/89/epochs/1/kpis/" \
      -H "X-API-Key: $ALO_API_KEY" \
      -H "X-Space-Slug: $WORKSPACE_SLUG"

    The response holds objective_value, total_executions, errored_executions, checks[] with {label, passed, evaluated}, latency and cost.

  4. After an optimization, POST /gym/{id}/promote/ saves the best configuration as a new agent, or POST /gym/{id}/reject/ discards it. Rejecting deletes the gym's copy of the agent and its conversations: read them first.

  • The gym's conversations belong to its copy of the agent: read them with fork_agent_id, as in Read the conversations.

Statuses

statusMeans
pending, queuedWaiting to start; a queued session starts on its own
running, cancellingIn progress
awaiting_approvalA "gated" session waits for POST /gym/{id}/decision/
completed, cancelled, failedEnded; failed carries error_message
promoted, rejectedAn optimization's result was kept or discarded
stop_reasonWhy an optimization ended
max_epochsIt ran every round
perfect_scoreThe first measurement already scored 100%; nothing to improve
budget_exhausted, quota_exhaustedIt ran out of budget or credit quota
cancelledIt was cancelled

Troubleshooting

ResponseCauseFix
422 An ad-hoc gym (no kit_id) needs …No kit_id, and dataset or assertions is emptySend kit_id, or both
409 on promote or rejectThe session is not completed or cancelledWait for it to end
409 on deleteThe session is still runningCancel it first
409 on decisionThe session is not awaiting_approvalRead its status
422 `file_bucket` is for an ad-hoc gymfile_bucket sent with kit_idSend one or the other
404 on /epochs/{n}/kpis/That round has no report yet, or does not existUse a round from epoch_reports

On this page