Jev 1.13: decision points in AI code review
Decision points in AI code review: small yes/no and pick-one questions with calibrated probabilities, not one big model for every step. Notes on Jev 1.13.
Decision points in AI code review are small, specialized judgments: is this finding true, about new lines, matching a rule, and worth a human’s time? A fast yes/no or pick-one model answers those more cheaply than a chat or coding LLM. High confidence is not correctness, so application code still decides when to trust an answer. The bet is many clear questions instead of one mega-model doing the whole review.
This is early. We wired a decision model (Jev 1.13, via TypeSafe / OpenRouter) into Revix for the judgment steps on the PR review path. It is not another chat or coding model. Read what follows as architecture notes, not a claim that every team should copy it tomorrow.
Decision points in AI code review, defined
A decision point is a discrete yes/no or pick-one judgment that the review system asks at a specific step. Each question is narrow and answerable. Application logic owns what happens with the answer. It is not a free-form chat about the PR or a request to rewrite the diff.
In practice, a decision point is a tight prompt plus a constrained output. Examples:
- True or false.
- Pick the best matching rule from a short list.
- Score whether a comment deserves a human glance.
The model is there to judge, not to narrate. This matters because PR review is a pipeline of different jobs: finding candidates, matching rules mined from the repo, ranking attention, and writing comments. For the basics, see what is AI code review. For why review load grows as shipping speeds up, see more PRs, slower merges.
Why one large LLM is the wrong fit
Asking one large LLM to find issues, assign severity, invent confidence, match team standards, and write the thread mixes incompatible jobs. The output sounds fluent while the judgment underneath is mushy. Teams then mute the bot.
Split the jobs instead, so each answer stays small and checkable:
- A coding-oriented pass proposes candidate findings.
- A decision model answers whether each candidate is true, scoped to new lines, tied to a rule, and worth attention.
- Application code applies thresholds.
- Humans still review architecture and product risk.
Useful comments matter more than comment volume. Decision points are one architecture move toward a better ratio of useful comments to total comments.
What a decision model is good and bad at
A decision model is good at fast yes/no and pick-one answers to clear questions. It is cheap and quick compared with long-form chat. It is bad at open conversation, essay-length critique, and replacing either the coding agent or human review.
Clear questions are the contract. A vague prompt gets a vague judgment. If the allowed answer is open prose, you are back to a chat model with a confidence score attached. Design the question first, then call the model, then let application code decide trust.
We treat Jev 1.13 as that judgment layer on the review path. We do not ask it to author the PR or own the merge. Standards still come from rules mined from the repository and from humans editing those rules. For how repo-derived review differs from generic diff review, see repo-aware vs diff-based AI code review.
High confidence is not correctness
High confidence is a model score, not proof that a finding is right. Calibrated probabilities help with ranking and thresholds. They do not decide when to suppress, post, or escalate. Application code must own that trust boundary, or authors learn to ignore every badge.
Calibrated probabilities are more useful than one vague confidence badge. They let application code ask harder questions: how sure, relative to what, and what action follows? A 0.9 that a finding is “interesting” differs from a 0.9 that it is true on new lines against a specific rule.
If your bot collapses all of that into one “high confidence” label, authors learn to ignore the label. A single label trains people to mute the bot, just as comment volume does.
Four questions on the PR review path
We split the judgment into separate questions so one fuzzy score cannot hide four different failures. Each answer can gate posting, ranking, or suppression on its own.
| Split question | What it gates |
|---|---|
| Is it true? | Suppress invented or unsupported claims |
| About new lines? | Prefer findings on the diff, not drive-by nits |
| Match the rule? | Tie to a repo-derived standard, not vibes |
| Worth a human’s time? | Cap attention; cut noise that trains people to mute |
The goal is fewer comments that people ignore on sight, not more comments. Rules mined from the repo give the “match the rule” question something real to check. Generic best-practice lists invent findings the team never enforced by hand.
This runs on PRs from GitHub.com, GitLab.com, and Bitbucket Cloud. The decision layer does not replace Slack notifications, human threads, or your merge policy. For how review can show up in Slack without becoming another muted channel, see AI code review in Slack.
Meaning-matched threads protect signal
Matching review threads by meaning keeps related findings grouped, so fixing one issue does not hide another. Matching by exact string or by a line position that moved is brittle. A legitimate fix can then erase a sibling problem that still matters.
Meaning match is a decision-shaped problem too: are these two findings the same concern, or two different ones that sit near each other? Getting it wrong either duplicates noise or silently drops signal. We care about the second failure as much as the first.
Matching by meaning is hard, and this part is still early. The goal is that resolving one thread never buries a different rule break on the same diff.
Graceful degrade when the model fails
The review should still finish. Graceful degrade means an unavailable, empty, or nonsense decision answer does not abort the PR review. You may lose some ranking or filtering help. You should not lose the review.
Decision points are helpers on the path, not a single point of failure. Application code owns timeouts, fallbacks, and the definition of junk. If the decision layer is down, authors and reviewers still get a completed review.
Junk handling also protects trust. A confident wrong answer that always posts is worse than a skipped judgment that stays quiet. When the decision output is unusable, suppress and continue.
What this architecture does not claim
A decision model does not replace the coding agent or human review. One mega-LLM owning find, score, match, and comment is not “smarter.” A confidence badge is no substitute for the split questions above.
The takeaway: smaller decision points, clear questions, calibrated probabilities, meaning-matched threads, and graceful degrade. It is still early. Measure useful comments on your own repos before you treat any vendor story as settled. If you are comparing tools by noise and fit, Greptile alternatives covers when recall-heavy review still floods the thread.
Revix mines your repository, builds review rules from how your team already writes code, and reviews every pull request against them. You edit the rules. To see decision points on a real PR, try a review on revix.ai.
Frequently asked questions
Keep reading
- What Is AI Code Review and How Does It Work?How AI code review works stage by stage, what it reliably catches, where it fails, and the one difference between tools that decides what yours can find.
- AI code review in Slack: get the nudge, keep the PRAI code review in Slack should DM the person who has to act, not flood a channel. How Revix does it on Pro, what ships today, and when to skip Slack.