Surfaces

API reference

The REST API: submit a verification, poll it, read the verdicts.

The engine is a small REST API at https://calma-engine.fly.dev. Authenticate every request with a bearer API key. Verification is async: you submit a repo, get a job id, and poll it.

Submit a verification

curl https://calma-engine.fly.dev/api/verify \
  -H "Authorization: Bearer $CALMA_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"repo":"owner/repo","deep":true}'

Body fields

Field	Type	Default	Meaning
`repo`	string	—	`owner/name` or a GitHub URL
`deep`	bool	`false`	re-run the code (vs a fast, no-run validity check)
`entry`	string	auto	entrypoint override, e.g. `run.py`
`pip_install`	string[]	auto	extra dependencies
`artifact_repro`	bool	`false`	also verify committed result files regenerate (see below)

Returns { "id": "…", "tier": "pro", "deep": true }.

Poll a job

curl https://calma-engine.fly.dev/api/jobs/<id> -H "Authorization: Bearer $CALMA_API_KEY"

The job carries status (queued → running → done/error), a live progress line, per-claim claims (each with a verdict), and counts. List your recent scans with GET /api/jobs; get the full run log as plain text with GET /api/jobs/<id>/logs.

Endpoints

Method	Path	Returns
`POST`	`/api/verify`	submit a scan → `{id}`
`GET`	`/api/jobs`	your recent scans (newest first)
`GET`	`/api/jobs/{id}`	one scan: claims + verdicts + counts
`GET`	`/api/jobs/{id}/logs`	the full run log (plain text)
`GET`	`/api/usage`	your live meter (scans, budget, tier)

Artifact reproduction

For a "reproduce the paper" repo — one that ships a repro script + committed result files (leaderboard.csv, metrics.json) whose numbers come from custom code — set "artifact_repro": true. Calma re-runs the repro entrypoint and diffs the regenerated result files against the committed ones, reporting how many committed numbers reproduce identically (and naming any that don't). It reports reproduced / not-reproduced / incomplete — never a false "reproduced".