Development report · November 2026
UM Review takes a submitted clinical review and gives the physician advisor an organized, criteria-referenced picture of the case, a proposed decision, and the documentation written both ways. Its purpose is to enrich the information in front of a physician, not to automate utilization management.
Principles
These determine how the tool behaves, and they are the standard a plan, a regulator or a reviewer should measure it against.
01
When the physician sets a decision, the tool writes to it. When the submitted review proposes one, the tool uses that and says so. Only when neither exists does it propose a decision of its own, labelled as a proposal. Who decided is recorded on every output. This is the shape California's SB 1120 requires: AI may inform a medical-necessity determination; a physician makes it.
02
The target is a supportable, transparent decision with documentation a physician can edit in a minute, not a tool that is right on its own. A physician who changes the proposal for reasons that do not generalize has not found an error; they have done their job with better information in front of them.
03
Every review has the same four parts, in the same order, with the same fixed sentences: a verdict for the physician, the criteria attestation, structured notes whose Advisor Review always states the findings against the decision before the findings that decide it, and a member letter in plain language. The model writes the prose; it never writes the fixed sentences.
04
The reasoning runs on service profiles the author wrote and owns, with shared rules on fairness and proportionality. They can be read, edited and published without a license. Licensed criteria a review names (ASAM, LOCUS, InterQual) appear as labels, which matters because ASAM's terms prohibit entering its criteria into AI systems at all.
05
The mechanical parts of review, such as how appealed days are counted, how long a single case agreement runs, and what happens when a person's needs exceed the level requested, are applied by code, identically every time. Whenever a rule changes what the model proposed, or cannot be applied, a flag beside the decision says so.
06
Every case leaves a record of who decided, what the referral proposed, what the model proposed before any rule applied, which rules acted, and which version of the tool and model produced it. A plan can measure its own override rates and anchoring from those fields; an auditor can check a decision against the criteria without the model in the room.
What it looks like
The whole pass on one synthetic case: the empty interface, the submitted review pasted in, the facts for and against lifted out of the record, one call to the model, and the review as it comes back, with the Advisor Review stating the findings against the decision before the findings that decide it, and the member letter in plain language. The wait for the model is shortened here; in practice it is about seventeen seconds. Nothing in it is a real member.
What it does well
The two evaluations used different scorers and different standards, and they agree on the shape of the result: the tool reproduces the record faithfully, reaches decisions a second reader finds supportable, and makes its reasoning visible enough that its errors can be found and corrected.
01 · Fidelity
98.8%factual fidelity, author-calibrated scoring, 108 cases; decision-making 94.4%
The clinical review it writes is a condensation of the record with the contradictions left in, which is what a physician needs to read and what an auditor needs to check.
02 · Supportability
96%of decisions matched a second model's independent reading, 55 of 57 held-out cases
A model from a different vendor read each case cold, reached its own decision, then graded the output. It rated the decision line as fully supported by the record in 51 of 57. Single case agreements were the weakest category, and the one where it found a defect rather than a judgment call.
03 · Correctability
6 → 3major defects on the same 60 cases, one version to the next; decision agreement 86% → 96%
The grader found a reasoning pattern the author had missed: denials that left a member with less care when their needs exceeded the level requested. It was corrected in a day as a rule, not by prompt-tuning, and the grader confirmed the fix on the same cases. A tool whose mistakes are legible to a second reader is one that can be governed.
Limitations
All 120 were written for this work. They carry no missing pages, no contradictory faxes, and no referral recommendations in the held-out set, so the tool's effect on a physician anchored to a nurse reviewer's proposal, the author's main experience of using it, has not been measured.
The criteria, the format and the reference determinations are the author's. The first evaluation measures agreement with the author's own method.
The second evaluation is independent of the vendor that wrote the reviews but not of model judgment. Two gradings of identical text differed by up to 0.08 on a 2-point scale, and the rubric was revised once after seeing results; both gradings are reported.
Requests for a specific procedure or treatment do not fit the format.
About 100 cases, part derived from public independent-review decisions, physician advisors reviewing with and without the tool, and blinded raters scoring the documentation.
Collaborate
01
Behavioral health physician advisors to review cases in both arms, and raters to score documentation blind. Paid.
02
A health plan, delegated review vendor or regulator that wants to see a physician-first, auditable tool on its own synthetic or de-identified cases.
03
A partner so the same format and audit record can run on criteria a plan already uses.
04
The study is small. Reviewer time is the cost.
Gerrit van Schalkwyk, MD
Child and adolescent psychiatrist; payer-side physician advisor in behavioral health utilization management
Gerrit@resilientmindsmd.comCopied