pinecallDocs
Testing / pinecall runs

pinecall runs

The runs of a suite, and one run's matrix.

pinecall runs list [--limit n] | show <id> | diff <a> <b>
pinecall runs promote <call-id> [--name x] [--out test/candidates] [--from-seq n]
pinecall runs drift --agent <slug> [--window 7d] [--baseline 30d] [--threshold 10] [--limit 200]
                … any of them with --json: what the gateway answered, for a pipe
Terminal
$ pinecall runs list --limit 3
run_8a8870b59bc1  2026-09-11 12:47:23  clinica-norte     done     1/1
run_5c6b559d6093  2026-09-11 11:56:49  clinica-norte     done     2/2

show prints one run the way test printed its matrix when it happened — without the first line, which carries the run id and belongs to a run that is still happening; diff says what moved between two, golden by golden — a measurement that HELD and is new to the later run is not a change and is left out, one that is BROKEN is printed (not measured → broken), and one the later run stopped making is printed too, so a golden nobody ran is never read as a fix. promote writes one real call down as a golden candidate in test/candidates — the golden the gateway derives from it (GET /v1/calls/{call}/golden): the state it was in at --from-seq, every caller turn after it, the facts the app gave it, what memory recalled, the day it ran, and an expect from its broken verdicts — for a person to edit before it counts as a golden. Promote is for the call that broke: a verdict says what went wrong, never what was right, so the expect says what must not happen again — the tool a broken consent ran unasked in not_tools, grounded: true, the other judges that broke in judges, asked again by name — and the verb prints a note for each. A call that held gives an empty expect, and you write what it must keep doing. The same golden is what cases keep keeps on the gateway instead, for a case that belongs in the dataset rather than in the repository. drift counts each judge's held-rate over two windows of finished calls and exits 1 when one fell further than --threshold points: nothing is judged again, a held-rate is a count of the verdicts call.score already carries. The window is the last --window; the baseline is the time before it, back to --baseline ago, so --baseline must be longer than --window or it is refused with exit 2. It reads the agent's newest --limit calls, 200 by default, which is the sessions endpoint's own ceiling, so a window wider than that is counted over those 200.