Measured. Wins and losses.
DecisionNode against TypeSafe Jev, Cloudflare Clef and five open decision models: 15 text suites, 25,134 items, every suite shown, including the ones we lose. Then two image suites, places in photos, video and drone views, and our own latency.
- DecisionNode, 15-suite mean
- 0.817
- DecisionNode Flash
- 0.783
- DecisionNode Flash, short request
- 5 ms
Preliminary results, October 2026
The 15-suite mean
Accuracy averaged over the 15 text suites, one dot per system. DecisionNode scores 0.817 and TypeSafe Jev 0.803. DecisionNode Flash, the tier built for speed and price, scores 0.783.
- our models
- other systems
Preliminary results, October 2026
- 1DecisionNodeBynn Intelligenceours0.817
- 2TypeSafe Jev 1.13TypeSafe AI0.803
- 3Cloudflare Clef 27BCloudflare0.786
- 4DecisionNode FlashBynn Intelligenceours0.783
- 5Cloudflare Clef-flash 9BCloudflare0.759
- 6H2O-Lightning-4Bopen model0.748
- 7pplx-decider-27bopen model0.746
- 8Cygnet (Gemma 4 12B)open model0.717
- 9Kev-4Bopen model0.717
- 10Strands Decider 2Bopen model0.622
accuracy, 15 text suites
Where we lead, where we trail
DecisionNode minus TypeSafe Jev on every text suite. A bar is solid where the two 95% intervals do not overlap and hollow where the difference is inside the noise.
- DecisionNode ahead, intervals apart
- TypeSafe Jev ahead, intervals apart
- inside the noise
- Prompt injection+16.8 pts, DecisionNode 0.768, TypeSafe Jev 0.600, intervals apart
- Web tasks+6.2 pts, DecisionNode 0.760, TypeSafe Jev 0.698, inside the noise
- Rules and policies+5.2 pts, DecisionNode 0.883, TypeSafe Jev 0.831, intervals apart
- Public decision set+5.1 pts, DecisionNode 0.906, TypeSafe Jev 0.855, inside the noise
- Large option sets+3.8 pts, DecisionNode 0.813, TypeSafe Jev 0.775, inside the noise
- Familiar task families+1.9 pts, DecisionNode 0.793, TypeSafe Jev 0.774, inside the noise
- Multilingual+1.9 pts, DecisionNode 0.801, TypeSafe Jev 0.782, inside the noise
- Typed decisions+0.5 pts, DecisionNode 0.740, TypeSafe Jev 0.735, inside the noise
- Short classifications−0.5 pts, DecisionNode 0.899, TypeSafe Jev 0.904, inside the noise
- Long inputs−0.7 pts, DecisionNode 0.846, TypeSafe Jev 0.853, inside the noise
- Transfer tasks−0.8 pts, DecisionNode 0.844, TypeSafe Jev 0.852, inside the noise
- Open decision tasks−1.0 pts, DecisionNode 0.788, TypeSafe Jev 0.798, inside the noise
- Held-out decisions−1.6 pts, DecisionNode 0.932, TypeSafe Jev 0.948, inside the noise
- New task families−3.5 pts, DecisionNode 0.788, TypeSafe Jev 0.823, intervals apart
- Knowledge (MMLU-Pro)−12.8 pts, DecisionNode 0.692, TypeSafe Jev 0.820, intervals apart
← TypeSafe Jev aheadDecisionNode ahead →
difference in accuracyLeads
Clear leads on prompt injection (+16.8 pts) and rules and policies (+5.2 pts). Ahead too on web tasks (+6.2 pts) and public decision set (+5.1 pts), by margins inside the noise. That is the everyday work of a workflow: hold to the task when the input argues, apply a written policy and act on a page.
Trails
Jev is clearly ahead on knowledge (−12.8 pts) and new task families (−3.5 pts), and ahead by a margin inside the noise on held-out decisions (−1.6 pts) and open decision tasks (−1.0 pts). If your decisions lean on broad world knowledge, weigh that suite first.
Every suite, every system
Accuracy per suite, with the 95% interval under it for our models and TypeSafe Jev. The highest score on each row is marked. Scroll sideways on a small screen.
| Suite | DecisionNodeBynn Intelligence | DecisionNode FlashBynn Intelligence | TypeSafe Jev 1.13TypeSafe AI | Cloudflare Clef 27BCloudflare | H2O-Lightning-4Bopen model | pplx-decider-27bopen model | Cygnet (Gemma 4 12B)open model | Kev-4Bopen model |
|---|---|---|---|---|---|---|---|---|
| Held-out decisions636 items | 0.9320.91 to 0.95 | 0.9360.92 to 0.95 | 0.9480.93 to 0.96 | 0.933 | 0.926 | 0.959 (highest on this suite) | 0.924 | 0.892 |
| Public decision set117 items | 0.906 (highest on this suite)0.84 to 0.95 | 0.8120.73 to 0.88 | 0.8550.78 to 0.92 | 0.855 | 0.872 | 0.850 | 0.868 | 0.744 |
| Transfer tasks730 items | 0.8440.81 to 0.88 | 0.7790.74 to 0.82 | 0.852 (highest on this suite)0.81 to 0.89 | 0.810 | 0.753 | 0.829 | 0.786 | 0.762 |
| Web tasks1,106 items | 0.760 (highest on this suite)0.73 to 0.79 | 0.7510.72 to 0.78 | 0.6980.66 to 0.73 | 0.697 | 0.612 | 0.672 | 0.632 | 0.612 |
| Knowledge (MMLU-Pro)1,479 items | 0.6920.67 to 0.71 | 0.5720.55 to 0.60 | 0.820 (highest on this suite)0.80 to 0.84 | 0.634 | 0.482 | 0.627 | 0.544 | 0.494 |
| Short classifications178 items | 0.8990.84 to 0.94 | 0.8650.81 to 0.92 | 0.9040.86 to 0.94 | 0.921 | 0.831 | 0.941 (highest on this suite) | 0.913 | 0.792 |
| Open decision tasks5,011 items | 0.7880.78 to 0.80 | 0.7700.76 to 0.78 | 0.7980.79 to 0.81 | 0.834 (highest on this suite) | 0.809 | 0.790 | 0.759 | 0.811 |
| Typed decisions955 items | 0.740 (highest on this suite)0.71 to 0.78 | 0.7130.68 to 0.75 | 0.7350.70 to 0.77 | 0.705 | 0.688 | 0.720 | 0.663 | 0.663 |
| New task families3,629 items | 0.7880.78 to 0.80 | 0.7500.74 to 0.76 | 0.823 (highest on this suite)0.81 to 0.83 | 0.800 | 0.776 | 0.800 | 0.758 | 0.738 |
| Familiar task families4,100 items | 0.793 (highest on this suite)0.78 to 0.81 | 0.7670.76 to 0.78 | 0.7740.76 to 0.79 | 0.772 | 0.728 | 0.777 | 0.727 | 0.707 |
| Large option sets1,677 items | 0.8130.79 to 0.83 | 0.7950.78 to 0.81 | 0.7750.76 to 0.80 | 0.842 (highest on this suite) | 0.781 | 0.775 | 0.694 | 0.701 |
| Long inputs1,147 items | 0.8460.82 to 0.87 | 0.8150.79 to 0.84 | 0.853 (highest on this suite)0.83 to 0.87 | 0.845 | 0.828 | 0.492 | 0.634 | 0.758 |
| Multilingual1,812 items | 0.801 (highest on this suite)0.78 to 0.82 | 0.7810.76 to 0.80 | 0.7820.76 to 0.80 | 0.760 | 0.737 | 0.800 | 0.756 | 0.727 |
| Rules and policies1,759 items | 0.883 (highest on this suite)0.87 to 0.90 | 0.8660.85 to 0.88 | 0.8310.81 to 0.85 | 0.798 | 0.752 | 0.845 | 0.778 | 0.704 |
| Prompt injection798 items | 0.7680.74 to 0.80 | 0.769 (highest on this suite)0.74 to 0.80 | 0.6000.56 to 0.63 | 0.582 | 0.648 | 0.318 | 0.327 | 0.652 |
Cloudflare Clef-flash 9B and Strands Decider 2B are in the mean above; their per-suite results are not published.
Text and images, one request
Two image suites: questions about real photos, and statements judged from rendered scenes. DecisionNode reads images and text together and answers through the same question types. TypeSafe Jev reads text only, so it is shown as not supported rather than as zero.
Photos
Choice and truth questions about real photos. 1,668 items.
- DecisionNode
- 0.945
- DecisionNode Flash
- 0.934
- TypeSafe Jev 1.13
- not supported, reads text only
- Cloudflare Clef 27B
- 0.882
- Cloudflare Clef-flash 9B
- 0.905
- pplx-decider-27b
- 0.891
- H2O-Lightning-4B
- 0.890
Not run on the image suites: Cygnet (Gemma 4 12B), Kev-4B and Strands Decider 2B.
Rendered scenes
How likely a statement is, judged from a computer-rendered scene. 450 items.
- DecisionNode
- 0.762
- DecisionNode Flash
- 0.680
- TypeSafe Jev 1.13
- not supported, reads text only
- Cloudflare Clef 27B
- 0.783
- Cloudflare Clef-flash 9B
- 0.742
- pplx-decider-27b
- 0.809
- H2O-Lightning-4B
- 0.684
Not run on the image suites: Cygnet (Gemma 4 12B), Kev-4B and Strands Decider 2B.
Places, video and drone views
Readings only our two models were measured on: where things are in a photo, questions about video and distances from drone views with Depth Vision. No other system on this board was run on them, so none is shown here. TypeSafe Jev reads text only.
One place in a photo
Point to a named thing, or draw a box around it. A point counts when it lands on the thing; a box counts when its overlap with the true box (IoU) is 0.5 or more. 300 photos.
- Point on the thing
- DecisionNode0.960DecisionNode Flash0.950
- Box hit, IoU 0.5 or more
- DecisionNode0.927DecisionNode Flash0.897
- Box overlap, mean IoU
- DecisionNode0.849DecisionNode Flash0.812
Every match in a photo
Find every match of a named thing, as points and as boxes. Precision is the share of returned places that are right; recall, the share of the real ones found. 197 photos.
- Points, precision
- DecisionNode0.721DecisionNode Flash0.688
- Points, recall
- DecisionNode0.672DecisionNode Flash0.663
- Boxes, precision
- DecisionNode0.692DecisionNode Flash0.646
- Boxes, recall
- DecisionNode0.547DecisionNode Flash0.522
Video
Choice, score and truth questions about short clips sent as frames, in four kinds of footage. 1,381 questions.
- All questions
- DecisionNode0.71595% interval 0.69 to 0.74DecisionNode Flash0.71895% interval 0.69 to 0.74
- General scenes461 questions
- DecisionNode0.86695% interval 0.83 to 0.89DecisionNode Flash0.84695% interval 0.81 to 0.88
- Factory lines362 questions
- DecisionNode0.71595% interval 0.67 to 0.76DecisionNode Flash0.74995% interval 0.71 to 0.79
- Security cameras334 questions
- DecisionNode0.64795% interval 0.60 to 0.70DecisionNode Flash0.66295% interval 0.61 to 0.71
- Traffic224 questions
- DecisionNode0.50995% interval 0.43 to 0.59DecisionNode Flash0.48795% interval 0.41 to 0.57
Drone views, with Depth Vision
Questions about pictures taken from drones: how far to the first obstacle ahead, whether the path is clear, which way is most open, how urgent it is to stop, and how far away the middle of the picture is. A distance counts when it is within 20% of the truth, or half a meter. Most questions are about rendered flights; 263 are about real drone photos. Each kind below is read with Depth Vision. 1,924 questions.
- All questions, with Depth Vision
- DecisionNode0.584DecisionNode Flash0.563
- All questions, without it
- DecisionNode0.525DecisionNode Flash0.524
- Is the path clear for a given distance?400 questions
- DecisionNode0.812DecisionNode Flash0.800
- Which way is most open?176 questions
- DecisionNode0.784DecisionNode Flash0.750
- How urgent is it to stop?374 questions
- DecisionNode0.802DecisionNode Flash0.775
- How far to the obstacle ahead?338 questions
- DecisionNode0.482DecisionNode Flash0.429
- How far is the middle of the picture?636 questions
- DecisionNode0.310DecisionNode Flash0.310
Real photos taken from high up are the hard part: there, most distance answers miss. Depth Vision's distances are estimates from one picture, for decisions and planning, never a safety stop.
Answered while a chat model is still typing
A chat LLM writes its answer token by token: GPT-5.6 Luna took 1,300 ms for a short decision, timed end to end from one client. DecisionNode reads the input once and answers in one pass: 19 ms end to end on DecisionNode Flash, 5 ms of it on our side.
up to 68×faster than a chat LLM on the same decision, end to end, on DecisionNode Flash
- DecisionNode Flash19 ms end to end, 5 ms on our side
- GPT-5.6 Luna1,300 ms end to end
milliseconds end to end, log scale
End to end from one client, network included, one short decision, against GPT-5.6 Luna. October 2026. DecisionNode's latency is not published yet.
Accuracy is half of it. Calibration is the other half.
Every Choice and Score answer carries a confidence, and every Truth answer is the probability that a statement is true. Calibrated means those numbers mean what they say: of all the answers given at 0.9, about nine in ten are right.
That is what lets your code act alone. Set a threshold once and every answer above it is acted on instantly; below it, your code takes the safe path or re-checks with the full model. Calibration results per question type will be published with the full run.
- right
- right
- right
- right
- right
- right
- right
- right
- right
- wrong
How we measured
15 text suites, 25,134 items, development splits. 95% intervals for our models and Jev from a bootstrap that resamples whole inputs. Jev was called through its official API; open models ran on our GPUs as their authors serve them. October 2026.
The suites
- Held-out decisions636
- Typed decisions on tasks kept out of development.
- Public decision set117
- A small public set of choice, score and truth questions.
- Transfer tasks730
- Decisions phrased the way other decision APIs phrase them.
- Web tasks1,106
- Pick the right element or action on a real web page.
- Knowledge (MMLU-Pro)1,479
- Hard multiple-choice questions that need world knowledge.
- Short classifications178
- One-line inputs sorted into a few labels.
- Open decision tasks5,011
- A broad public mix of typed decision tasks.
- Typed decisions955
- Mixed choice, score and truth questions in one request.
- New task families3,629
- Task families the model never saw while it was built.
- Familiar task families4,100
- New items from task families the model knows.
- Large option sets1,677
- Choices with dozens of options to pick from.
- Long inputs1,147
- Long threads and documents with the answer deep inside.
- Multilingual1,812
- The same decisions asked in many languages.
- Rules and policies1,759
- Apply a written policy to a case and decide.
- Prompt injection798
- Inputs that try to talk the model into the wrong answer.
- Photos1,668, image
- Choice and truth questions about real photos.
- Rendered scenes450, image
- How likely a statement is, judged from a computer-rendered scene.
- Each suite is scored as accuracy: the share of items where the answer matches the reference. Intervals are 95%, from a bootstrap that resamples whole inputs, and are shown where they are published: for our models and TypeSafe Jev.
- TypeSafe Jev 1.13 was called through its official API with its documented request shape. Open models ran on our GPUs, served the way their authors publish them.
- Strands Decider 2B: its published training data contains items of two of our test sets, so its mean may read high.
- Places, video and drone views were measured on our two models only, scored as each panel says. No other system here was run on them.
Information last reviewed: October 2026.
Every system here changes over time, ours included. Scores describe the versions measured on the date above; check each provider's current documentation before you choose.
DecisionNode is not affiliated with TypeSafe AI, Cloudflare, OpenAI or the other model publishers. Their names identify the systems measured.
Preliminary results, October 2026
Questions about the numbers
Why show the suites you lose?
Because a mean hides the shape. You should know that DecisionNode trails on knowledge-heavy questions and on task families it has never seen before you build on it, and that it leads on prompt injection, web tasks and written policies.
Are these numbers final?
No. They are preliminary results, october 2026, measured on the development splits of each suite, and the page says so wherever they appear.
Why are only your models on places, video and drone views?
Because no other system on this page was measured on them, so there is nothing to compare. TypeSafe Jev reads text only, according to its public documentation. The other systems were run on the suites above, not on these.
Why is Jev missing from the image suites?
TypeSafe Jev reads text only, according to its public documentation, so there is nothing to score. It is shown as not supported, not as zero.
Can I check the numbers on my own data?
Yes. Once you sign in, the playground runs any request against both models, and every answer is deterministic, so the same request gives the same answer every time you run it.
How fast is it next to the others?
DecisionNode Flash answers a short request in about 5 ms on our side and 19 ms end to end. GPT-5.6 Luna, timed from the same client on the same decision, took 1,300 ms, so DecisionNode Flash is up to 68× faster.