Confidence-gated routing

Set a threshold once: every answer above it is acted on instantly, and everything below it is re-checked by the full model or takes the safe path. Calibrated confidence makes the split predictable.

The shape of it#

1.Ask your questions in one request to DecisionNode-⁠1.0 Flash, the fastest tier.
2.If confidence clears the threshold, act on the answer at once.
3.If it does not, send the same request to DecisionNode-⁠1.0 and act on that answer when it clears the bar.
4.If neither clears it, take the safe action your product defines. Log every decision with the model that made it.

THRESHOLDS = {"route": 0.7}   # per question, in config

def handle(ticket, request):
    for model in ("decisionnode-flash-latest", "decisionnode-latest"):
        route = decide({**request, "model": model})["route"]
        if route["confidence"] >= THRESHOLDS["route"]:
            queues[route["choice"]].put(ticket)
            return log(ticket, decided_by=model, answer=route)
    queues["general"].put(ticket)           # the safe default, still automatic
    log(ticket, decided_by="safe_default", answer=route)

Tuning the threshold#

Your outcomes are the labels: reopened tickets, refunds charged back, comments restored on appeal. After a week, compute for a few thresholds how much traffic Flash settles on its own, how much the full model settles, and how often either was wrong. Raise the threshold if the error rate is too high; lower it if the full model agrees with Flash on nearly every re-check.

Reading a threshold sweep
Signal	What it means	Move
The full model agrees with almost every re-check	The threshold is stricter than it needs to be	Lower it
Automatic answers are wrong too often	The threshold is too loose	Raise it
Many items sit just below the line	The question may be ambiguous	Rewrite the options or split the question

The shape of it#

1.Ask your questions in one request to DecisionNode-⁠1.0 Flash, the fastest tier.

2.If confidence clears the threshold, act on the answer at once.

3.If it does not, send the same request to DecisionNode-⁠1.0 and act on that answer when it clears the bar.

4.If neither clears it, take the safe action your product defines. Log every decision with the model that made it.

THRESHOLDS = {"route": 0.7}   # per question, in config

def handle(ticket, request):
    for model in ("decisionnode-flash-latest", "decisionnode-latest"):
        route = decide({**request, "model": model})["route"]
        if route["confidence"] >= THRESHOLDS["route"]:
            queues[route["choice"]].put(ticket)
            return log(ticket, decided_by=model, answer=route)
    queues["general"].put(ticket)           # the safe default, still automatic
    log(ticket, decided_by="safe_default", answer=route)

Tuning the threshold#

Reading a threshold sweep
Signal	What it means	Move
The full model agrees with almost every re-check	The threshold is stricter than it needs to be	Lower it
Automatic answers are wrong too often	The threshold is too loose	Raise it
Many items sit just below the line	The question may be ambiguous	Rewrite the options or split the question