Versions and retirement

• DecisionNode-⁠1.0

  Alias you send as model (moves)

  decisionnode-latest

  Version, a pinned id (never changes)

  decisionnode-1.0

• DecisionNode-⁠1.0 Flash

  Alias you send as model (moves)

  decisionnode-flash-latest

  Version, a pinned id (never changes)

  decisionnode-1.0-flash

Aliases and pinned ids#

Two words, used the same way on every page: the alias is the model you send, and the version is the pinned id the response names. You may also send a version as model, to pin it.

• An alias such as decisionnode-latest answers from the newest version of its model. It moves when we release one, without notice, so answers can change on the day of a release.
• A pinned id such as decisionnode-1.0 answers from exactly that version for as long as it is available. The same request gets the same answer every time. See Determinism.
• Every response names the version that answered: model in the body and x-decisionnode-model in the headers, whichever id you sent.
• GET /v1/models lists each alias with the pinned version it points to today and its release_date.

How long a version lives#

• A pinned version stays available

  Value

  At least 12 months after its successor is released

• Notice before a pinned version retires

  Value

  At least 12 months

• How notice is given

  Value

  In the console and by email to your account's contact

• Evaluation and calibration results

  Value

  Published with each new version, before you switch to it

The binding commitment is section 3.3 of the Terms of Service. A version may be withdrawn sooner only where the law, security or the prevention of harm requires it.

After a version retires#

A call that names a retired pinned id gets 400, with the body every unknown model name gets; nothing is billed. An alias never retires: it moves to the newest version. A batch that names a retired pinned id is refused when you finalize it, with the same error, so move your batch files to the new id with your live code.

{
  "detail": {
    "error_type": "api_usage_error",
    "message": "Unknown model: decisionnode-1.0"
  }
}

Moving to a new version#

1. 1

   Read what changed#

   Each release lists its evaluation and calibration results. Check the suites closest to your work on the Benchmarks page.

2. 2

   Replay your own requests#

   Send requests you have kept with known outcomes to your current pinned id and to the new one, and compare. The program below counts the top answers that change: the choice, the score's nearest level, a truth on the other side of your cut-off, or the number.

   PythonTypeScript

   Copy

   import json, os, requests
   
   KEY = os.environ["DECISIONNODE_API_KEY"]
   HEADERS = {"Authorization": f"Bearer {KEY}"}
   CURRENT = "decisionnode-1.0"  # the pinned id your code sends today
   
   
   def decide(body: dict, model: str) -> dict:
       r = requests.post(
           "https://api.decisionnode.com/v1/decide",
           json={**body, "model": model},
           headers=HEADERS,
           timeout=10,
       )
       r.raise_for_status()
       return r.json()
   
   
   # the version the alias points to now
   listed = requests.get("https://api.decisionnode.com/v1/models", headers=HEADERS, timeout=10)
   models = listed.json()["models"]
   candidate = next(m["version"] for m in models if m["name"] == "decisionnode-latest")
   
   TRUE_AT = 0.5  # the cut-off your code acts at on truth answers
   
   
   def top(answer: dict):
       """What your code acts on: the choice, the level, true or false, the value."""
       kind = answer["type"]
       if kind == "choice":
           return answer["choice"]
       if kind == "score":
           return round(answer["score"])
       if kind == "number":
           return answer["number"]
       return answer[kind] >= TRUE_AT  # truth, or noul from a Jev-shaped client
   
   
   # replay requests you have kept with known outcomes, one per line: {"request": {...}}
   changed = 0
   with open("labelled.jsonl") as f:
       for line in f:
           if not line.strip():
               continue
           case = json.loads(line)
           old = decide(case["request"], CURRENT)["answers"]
           new = decide(case["request"], candidate)["answers"]
           changed += sum(top(new[key]) != top(old[key]) for key in new)
   print(f"{candidate}: {changed} top answers differ from {CURRENT}")

3. 3

   Re-tune your thresholds#

   Probabilities are calibrated per version, so the same cut-off can act on a different share of traffic. Re-pick each threshold on the new version from your replay, the way you first chose it. See Confidence.

4. 4

   Switch the pinned id#

   Change the id in config with the new thresholds, in one change, so a rollback restores both. Keep the old id until its retirement date for that rollback.