pinecallDocs
Testing / pinecall cases

pinecall cases

Real calls and simulations a judge broke on, grouped by judge: diagnosed, a setting applied, decided whole, pulled into the repository.

pinecall cases [list] [--status pending|approved|dismissed] [--agent <name>] [--json]
pinecall cases groups · propose <group> · apply <group> · persona <group>
pinecall cases show <name|group> · approve <name|group> · reopen <name> · forget <name>
pinecall cases dismiss <name> [--judge-was-wrong <judge>] [--note '…']
pinecall cases pull <name> [--out test/<agent>/goldens]
pinecall cases keep <call-id> --name x [--held-out]

A case joins its group. Every case kept at hang-up — a simulation's too, with the persona that played it — joins its agent's group for the judge that broke it, so fifty calls one judge broke on are one thing to read. pinecall cases groups lists them, the pending first: the judge, how many calls and how many were simulated, the cause, the fix proposed, and the group's id (grp_…), which every group verb takes. propose <group> asks the judge model to read the group's newest calls with the lines each verdict cites, the prompt, the settings and the tools, and to name the cause and the smallest fix — a setting (apply <group> writes it as a new version of your settings; replay it with pinecall test --case), code (the tool or prompt block and what it must do: your assistant makes it, through pinecall mcp's cases tool), or knowledge; one eval, none on your own judge key. persona <group> keeps a caller like the group's as the agent's own. approve and dismiss with a group's id decide every case in it at once; --judge-was-wrong then needs no judge's name: it is the group's.

Terminal
$ pinecall cases groups
pending   promises           4 call(s) (1 simulated) · it promised a call back nobody will make  grp_4c81e2a07d1b

$ pinecall cases propose grp_4c81e2a07d1b
pending   promises           4 call(s) (1 simulated) · the agent promises call backs no tool books · fix: code book_callback  grp_4c81e2a07d1b
  the agent promises call backs no tool books
  proposes code · book_callback · call book_callback before promising · 82% sure
  your assistant makes it: `pinecall mcp`, the cases tool, then `pinecall test --case`

Real calls are the dataset. A call a judge broke on is kept by the gateway at hang-up as a pending case — its caller's lines, the state it opened in, the facts the app gave it, what memory recalled, the day it ran — with an expect that says what must not happen again: the tool a broken consent ran unasked in not_tools, grounded: true, and every other judge that broke (Pinecall's others, the org's own and the agent's own) in judges, asked again of the replay by name. A person reads it, reproduces it, fixes the agent, and approves it into the nightly or dismisses it:

Terminal
$ pinecall cases
1 waiting of at most 50
pending    promises-youll-call-me-tomorrow-29d7c7  promises  production  3h
approved   thursday-afternoon                      —         sandbox     2d

$ pinecall cases show promises-youll-call-me-tomorrow-29d7c7
promises-youll-call-me-tomorrow-29d7c7 · pending · clinica-norte
  from call_29d7c7b6cdd643de9c659984a0125c8c (production, settings v4) · kept by the hang-up panel · 2026-10-08 11:02

  broke
    promises  it promised a call back nobody will make
  the caller
    "You'll call me tomorrow about Thursday, then"
  state   {"stage":"book"}
  today   2026-10-08
  expect  {"judges":["promises"]}

  pinecall test --case promises-youll-call-me-tomorrow-29d7c7     play it again through the agent this terminal serves
  pinecall cases approve promises-youll-call-me-tomorrow-29d7c7   the nightly plays it from now on
  pinecall cases dismiss promises-youll-call-me-tomorrow-29d7c7   nothing to fix; --judge-was-wrong <judge> when the judge was

The inbox lists the agent's cases, the pending first, newest first: status, name, the judges that broke, the world the call ran in, its age — and above them how many wait of the most that may (past it a broken call is judged as ever and not kept until a person decides some). --status lists one. A case is addressed by its name within the agent — the project's one, or the one --agent names; a name the agent has none of is said and exit 1.

approve puts it in the nightly (pinecall test --dataset). dismiss takes it out: --judge-was-wrong <judge> says the judge that broke should have held, which the gateway keeps as a calibration label of the call, --note beside it; a judge the case did not break on is the gateway's refusal, and --note without a judge is refused here, since it is kept on that label. reopen makes it pending again. pull writes the case's golden as <name>.json in test/<agent>/goldens/ (--out moves it) — the golden as the case holds it, promoted_from and all — and only then marks the case as kept in the repository, so the nightly plays the file and not the case. keep <call-id> --name x keeps a finished call as an approved case, with the expect its broken verdicts give; --held-out plays it only when a run names it — a release's check rather than the nightly's. forget drops the case; the call it came from stays.

A case is the org's, kept from either world, and played in the sandbox only. --json prints what the gateway answered — the list, or the case — with the fields it leaves at their defaults written in; pull --json prints {path, case}. A flag that is another verb's is refused with exit 2, before anything travels.


Running it#