Decision models
Decision models vs. LLMs: choose the smallest useful decision
Choose rules, a bounded decision model, a general LLM, or human review for each workflow step. Includes an illustrative triage example and a selection worksheet.
Use a decision model when your workflow needs a bounded answer—such as a category, priority, or choice of next step—from messy input. Use a general large language model (LLM) when it needs to generate an explanation, synthesize evidence, or draft something new. Use ordinary rules when the answer follows directly from reliable fields. Keep a person responsible for consequential exceptions.
These are workflow choices, not competing beliefs about which model is best. One useful system can combine all 4 approaches.
Why decision models are worth a look now
Ollama’s September 29, 2026 release added Jev-style decision models, with state and named questions producing bounded answers. Cloudflare’s October 1, 2026 announcement introduced Clef and Clef-flash, available on Workers AI and as open models under Apache 2.0. Cloudflare describes typed answers with probabilities for classification and routing.
Those releases make the approach more accessible to evaluate. They do not establish accuracy, speed, or cost for your workload. A vendor benchmark is not a result from your workflow, and a probability is not permission to act.
Separate the decision from the work
A question such as “Which of these 3 queues should receive this request?” has a bounded answer. “Explain the options and prepare a response” is open-ended. Splitting them lets you judge the classification separately from the draft.
Start with rules. If a verified account field uniquely identifies the destination, code can select it without model inference. Consider a decision model when free text contains the relevant meaning but the allowed choices are stable. Consider an LLM when you need synthesis or generation. Choose human review when the evidence is incomplete or the consequence exceeds the workflow’s authority.
Avoid forcing a bounded classifier to answer questions that need investigation. Also avoid giving a general model tools merely because it can select a category. Classification and authorized action are separate responsibilities.
Worked example: synthetic request triage
Imagine a fictional equipment-service inbox with 3 queues: maintenance, billing, and general questions. A request says, “The unit stopped yesterday, and the invoice still looks wrong.”
A rule checks whether the request has a verified account identifier. A decision model proposes a queue and an ambiguity flag. Because this input spans 2 categories, the review rule sends it to a person rather than automatically choosing one. A general LLM may draft a short summary for that reviewer, but it cannot send a reply or change the account.
To test the design, assemble labeled examples with a single issue, multiple issues, missing details, misleading instructions, and requests outside the categories. Compare rules alone, a decision model, and a straightforward LLM baseline using the same inputs. Check the actual destination after any permitted routing action.
This is a proposed example, not a measured test or a client system.
Worksheet
Choose one workflow step
Fill in the boundary before choosing a model. Scroll the table horizontally on a narrow screen.
| Question | Your answer |
|---|---|
| Exact decision needed | ___ |
| Allowed choices, including unknown | ___ |
| Reliable fields a rule can use | ___ |
| Input evidence the model needs | ___ |
| Examples with more than one valid answer | ___ |
| Errors that require human review | ___ |
| Permitted action after classification | ___ |
| Labeled test set and baseline | ___ |
| Acceptance criteria and review owner | ___ |
| Data boundary, version, and rollback | ___ |
Failure mode: a typed answer that is wrong
A valid category can still misclassify a request. A model’s reported probability can also be poorly calibrated on unfamiliar inputs. Do not assume that a particular threshold guarantees a particular error rate. Evaluate it on labeled examples from the intended workload, inspect failure categories, and preserve an unknown or review path.
Missing context is another failure mode. A classifier cannot recover facts that were never supplied. Let it flag insufficient evidence instead of turning an incomplete input into a confident-looking decision.
Acceptance checklist
- Try deterministic rules before adding inference.
- Define choices, unknowns, and review triggers.
- Test ambiguous, missing, and out-of-scope inputs.
- Separate classification from permission to act.
- Compare accepted outcomes, review effort, and completion time.
- Keep versioned examples and a rollback path.
Use the completion scorecard to check the final result, then compare cost per accepted task. The smallest useful decision is the one you can evaluate and safely use—not simply the smallest model.
Sources were checked October 7, 2026. This guide proposes an evaluation approach, not a performance or safety guarantee.