Sign in
Pending benchmarksTracking since Nov 3, 2026

Is Surgeon state of the art?Pending benchmarks.Announced Nov 3, 2026; no independent results yet.

Crosby's in-house verifier model that judges whether a contract redline is surgical (precise, targeted edits that keep the legal substance), returning reasoning plus a yes or no verdict. Post-trained from Qwen-3.5-27B; internal, live only inside Crosby's service for its customers.

Announced facts

The details so far

Lab
Crosby
Announced
Nov 3, 2026
API
Not yet
Open weights
Closed
Parameters
27B (Qwen-3.5-27B base)
Type
Verifier: reads one redline and outputs a reasoning trace and a yes or no verdict on surgicality
Base model
Qwen-3.5-27B (Crosby's blog)
Training
One epoch of SFT on rejection-sampled teacher traces, then GRPO with a reward gated on a correct verdict times an LLM-judge score of the trace
Training data
Crosby's agent redlines labeled against attorney-sent versions, plus synthetic negatives; validated on 500 redlines relabeled by Crosby attorneys
Use
Offline evaluation, live monitoring, auto-acceptance of surgical redlines and steering Crosby's agentic harness
Availability
Internal/customers only: in production inside Crosby's service; no public weights or API

Tags marked unsure are not stated on the lab's own pages.

Next steps

What would let us call it

Verdicts use only independent boards we already track. Two consecutive snapshots are required before any crown.

  • Publicly announced with a source
  • Usable by the public (API or weights)
  • Listed on an independent board we track
  • Enough results for a rank in a category
  • Holds across two snapshots (crowns only)
Self-reported

What the lab claims

Self-reported. Shown for context; not used in any verdict.

Lab claim: 90.6% balanced accuracy on Crosby's internal 500-redline surgicality set, against 84.3% for Gemini 4 Argon, 83.1% for GPT-6 Astra and GPT-6 Sol, and 60.4% for a heuristic baseline.

Lab claim: roughly 10-100x lower cost per million tokens than most closed models in that comparison.

Receipts

Sources

LABHow to Train your Surgeon (Crosby blog)crosby.ai/blog/how-to-train-your-surgeon · Nov 3, 2026LABCrosby announcement on Xx.com/crosbylegal/status/2117664204568109344 · Nov 3, 2026

Tracking ends Feb 1, 2027 unless the API or weights become available.